U.S. EPA Office of Research and Development’s Response to Recommendations in Strategic Research Planning for 2016-2019: A Joint Report of the Science Advisory Board and Board of Scientific Counselors (January 27, 2015)

THE ADMINISTRATOR OF THE ENVIRONMENTAL PROTECTION AGENCY

WASHINGTON, D.C 20460

AUG - h 2015

Deborah L. Swackhamer, Ph.D.

Chairwoman

Board of Scientific Counselors
Science Advisory Board
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington D.C. 20460

Dear Dr. Swackhamer:

1 want to thank the U.S. Environmental Protection Agency's Science Advisory Board and the Board of
Scientific Counselors for their joint review of the Office of Research and Development's strategic-
research action plans for 2016-19 and its roadmaps on cross-cutting research issues. The EPA's research
programs help build the scientific foundation upon which the EPA and others rely for making decisions
that protect public health and the environment.

This joint report provides valuable insight and very thoughtful recommendations that will strengthen our
research plans and approaches for federal-agency collaborations. I am especially pleased that ORD
received this report early in the development process so that the contributions from your committees can
be carefully considered as ORD works with EPA colleagues and stakeholders to determine research
priorities for 2016-19.

I appreciate the boards' recognition of the difficult budget climate during the past few years. These
uncertainties have a particular impact on our research enterprise that requires advance planning to carry
out multi-year research. ORD has been leading the way to ensure that the EPA is not only thinking about
the scientific and technical information needed for imminent decisions but also anticipating the
decisions and issues of the next decade. To develop the scientific foundation to address the
environmental challenges ahead of us, t he EPA must plan and embark on that research now.

As your report emphasized, implementation is as important as research planning. ORD is focusing on a
number of issues that you raised, including research integration on cross-cutting issues, work-force
planning and communication. We have made a substantial commitment to convene the BOSC during the
next several years to advise ORD as they implement these plans and roadmaps.

1 have enclosed a detailed response to each recommendation in the joint report so that members of both
boards can see how ORD is considering your guidance.

This paper is printed with vegetable-oil-based inks and is 100-percent postconsumer recycled material, chlorine-tree-processed and recyclable

-------
,r4 ,i- epf r<\ -in- \-i * ckUi u«l" u < h: kuiH L-. ,n» i ,i« uu| • • <\ in>' an; r-c.uJ > tiki pi
ivis \ r.„il it i(k IL1 \ ,i. '«¦ c	.id-, a:j* s - m .iKik c .hi*! Uy Iuio^l^ ii.u | toiw • im, :i iu. >pI

i Ml' O'V I r. >F flUMl

Mncereiv.

r, , \Kt

-------

THE ADMINISTRATOR OF THE ENVIRONMENTAL PROTECTION AGENCY

LLI

WASHINGTON, D C 20460

AUG - 2015

Peter S. Thorne, Ph.D.

Chairman

Science Advisory Board
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington D.C. 20460

Dear Dr. Thorne:

I want to thank the U.S. Environmental Protection Agency's Science Advisory Board and the Board of
Scientific Counselors for their joint review of the Office of Research and Development's strategic-
research action plans for 2016-19 and its roadmaps on cross-cutting research issues. The EPA's research
programs help build the scientific foundation upon which the EPA and others rely for making decisions
that protect public health and the environment.

This joint report provides valuable insight and very thoughtful recommendations that will strengthen our
research plans and approaches for federal-agency collaborations. 1 am especially pleased that ORD
received this report early in the development process so that the contributions from your committees can
be carefully considered as ORD works with EPA colleagues and stakeholders to determine research
priorities for 2016-19.

1 appreciate the boards' recognition of the difficult budget climate during the past few years. These
uncertainties have a particular impact on our research enterprise that requires advance planning to carry
out multi-year research. ORD has been leading the way to ensure that the EPA is not only thinking about
the scientific and technical information needed for imminent decisions but also anticipating the
decisions and issues of the next decade. To develop the scientific foundation to address the
environmental challenges ahead of us. the EPA must plan and embark on that research now.

I have enclosed a detailed response to each recommendation in the joint report so that members of both
boards can see how ORD is considering your guidance.

This paper is printed with vegetable-oil-based inks and is 100-percent postconsumer recycled material, chlorine-free-processed and recyclable.

-------
r.i iiH % ivc v "i.'hlihL' ,-m L: Kt.jvl- J«. Ju i «-i« l«> Ini(>Y• mj .>ii, |.i>r,iit Im'»iu r ptt , • '* >j i

,<>h is ¦ i', ,1 Mk If' \ .1- 'Vi res .jk i. i dtivt, i i >i ten . > ikI bt- >1.^3 n ir i'n>ti -s 'i ji.t.r; I ¦ : i.t
jl*. fio . ;r |!> i iivirl

)sure

-------
U.S. EPA Office of Research and Development's

Response to Recommendations in

Strategic Research Planning for 2016-2019: A Joint Report of the Science
Advisory Board and Board of Scientific Counselors (January 27, 2015)

-------
Table of Contents

General Findings and Overarching Recommendations	1

Program-Specific Recommendations for Research	9

4.1. Air, Climate and Energy	9

4.2	Safe and Sustainable Water Resources	14

4.3	Chemical Safety for Sustainability and Human Health Risk Assessment	19

4.4	Sustainable and Healthy Communities	26

4.5. Homeland Security	36

Recommendations for Integration	41

5.1	Childrens Environmental Health Roadmap	41

5.2	Nitrogen and Co-pollutants Roadmap	45

5.3. Climate Change Roadmap	54

5.4 Environmental Justice Roadmap	60

5.5. Integration across ORD Programs	63

-------
General Findings and Overarching Recommendations

NOTE: The recommendations within the SAB/BOSC report to EPA are repeated in this document for
convenience of the reader. Each recommendation is followed by an EPA response in italics (in a few
cases, two recommendations are grouped due to their similarity, followed by one response). For ease of
reference, section numbers and headings correspond to section numbers and headings in the SAB/BOSC
report.

3.1. Relationship to the EPA Strategic Plan

Charge Question 1(a). Considering the proposed research directions and focus, how well is ORD as a
whole poised to support EPA in meeting the goals of the EPA Strategic Plan?

In its strategic plan (U.S. EPA 2014), the EPA has committed itself to five environmental goals
(Addressing Climate Change and Improving Air Quality; Protecting America's Waters; Cleaning Up
Communities and Advancing Sustainable Development; Ensuring the Safety of Chemicals and Preventing
Pollution; Protecting Human Health and the Environment by Enforcing Laws and Assuring Compliance)
and to four cross-cutting strategies (Working Toward a Sustainable Future; Working to Make a Visible
Difference in Communities; Launching a New Era of State, Tribal, Local, and International Partnerships;
and Embracing EPA as a High-Performing Organization). ORD's draft StRAPs and roadmaps show high-
level strategic thinking in linking ORD's research efforts to the EPA strategic plan and in framing, in a
coordinated way, how ORD programs support progress toward the goal of sustainability. This represents
a sea change in EPA's research planning. The draft documents are effective tools for communicating
how the EPA's complex and inter-related research relates to the agency's mission, with one important
caveat. The reorganization and redirection of ORD over the past decade and the specific research plans
presented in the draft 2016-2019 StRAPs are directed toward providing the scientific foundation for a
new integrated systems approach that attends to broad goals of enhanced and sustained health of the
environment and the health and well-being of human communities. In contrast to the more traditional
focus on informing regulation and compliance, this new broader mission may not be well understood
and accepted by other parts of EPA, by the regulated community, and by the public. To the extent that
understanding and support is lacking, the ORD should consider making a more concerted effort to
"prepare the ground" for the new directions they intend to pursue in the coming decades.

Although the ORD plans are impressive, evaluating the StRAPs and roadmaps as internal strategic
planning documents proves challenging without knowing more information about ORD resources,
personnel, and personnel backgrounds and capabilities. Given stable or declining funding (in real terms)
overall across program areas, there is some concern that the ambitious language in the StRAPs and
roadmaps is not likely to match concrete research deliverables. The SAB and BOSC offer the
recommendations immediately below to help ORD develop more effective research plans to support the
EPA in meeting the goals of the agency Strategic Plan. Recommendations relating to strengthening the
relationship of specific national programs to the EPA Strategic Plan may be found in the discussion of
each program in section 4.

-------
Recommendation: Communicate more consistently in the StRAPs and roadmaps the EPA's specific
research niche and how ORD plans to partner with other entities, including international
organizations and other federal agencies. Given the complexity of environmental problems and stable
or declining resources, the StRAPs and roadmaps should communicate ORD's specific research focus
within larger environmental issues and describe how ORD or the agency more broadly is meeting the
science and research needs associated with those issues. The need for international cooperation is
appropriately emphasized generally, but the StRAPs and roadmaps do not describe how the EPA plans to
take advantage of knowledge created outside the United States and, in some cases, the StRAPs appear
to be "reinventing the wheel." For example, the Chemical Safety for Sustainability StRAP does not
mention two major international programs of direct relevance to that StRAP, namely the European
Union Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) Program, and the
Canadian Priority Substances List. Similarly, the ORD can only focus on a small component of climate
change research important to the EPA. How will other agency climate change research needs be met
and how can ORD best leverage the many national and international efforts in this area? The EPA could
also enhance its use of Cooperative Research and Development Agreements (CRADA) authorized under
the Federal Technology Transfer Act. This program allows the EPA to transfer federal technologies into
the marketplace, and collaborate on environmental research and development projects with outside
entities, such as industry, consortia, academia, trade associations, and state and local agencies.

EPA Response: The final StRAPs and cross-cutting research roadmaps will put greater emphasis on
clarifying EPA's specific research niche and how ORD plans to partner with other research organizations.
ORD places a very high value on collaboration with other research partners, exchanging information and
leveraging resources with organizations around the world. Cooperative Research and Development
Agreements are indeed another avenue for enhancing the transfer of environmental technologies into
the marketplace and supporting innovative ideas through interactions with the public and private
sectors. We agree that these points need to be very clear. To keep the documents at a strategic level,
they won't include extensive description of our many interactions and collaborations, but rather a strong
set of illustrative examples.

Recommendation: Use the StRAPs and roadmaps to communicate the most important priorities for
ORD to address. As noted above, the plans and research directions for the six ORD research programs
are generally well aligned to support EPA in meeting the goals of the EPA Strategic Plan. The challenge is
that many of the planned activities are under-funded, often leading to narrow project scopes with
modest projected impact. Considering the broad mission and range of expectations for ORD research
programs and the reality of steadily declining ORD budgets, there is no easy way to address this
shortcoming. ORD should seek advice from the SAB and BOSC to help it prioritize, rather than just
expand the list of general research that EPA should be doing by virtue of its broad mission.

EPA Response: ORD appreciates the observation that the plans and research directions for the research
programs are generally well aligned with the EPA Strategic Plan. We agree that there are many
challenges in addressing the broad range of EPA science needs, and that hard choices must be made in
identifying priority areas for ORD research. Based on input from this SAB/BOSC report and from our EPA
partner offices and external stakeholders, the research plans will be refined to convey our priorities for
2016-2019. Further interaction with the BOSC Executive Committee and Subcommittees in the coming
years will help us to make decisions as we deal with newly emerging research needs and with budget
constraints.

-------
Recommendation: Communicate more clearly how planned research will inform specific agency
decisions. ORD can best support the EPA's strategic goals if it develops research that is clearly linked to
the information needs of agency decision makers and adheres to the EPA Strategic Plan core value of
"transparency." ORD should consider expanding the use of the "dashboard for decision makers/' which
provides access to ORD-generated tools and research, as articulated by the Chemical Safety for
Sustainability program, into other ORD programs.

EPA Response: ORD has made great strides in working very closely with EPA program offices and
Regions in planning its research. From the staff level up through the Assistant and Regional
Administrators, ORD programs engage our policy and regional partners throughout the year to ensure
that our research plans reflect the highest Agency priorities and will be useful for upcoming EPA
decisions. For example, each research program holds an annual spring meeting involving both the
research scientists and EPA program and regional office staff providing the opportunity for direct
interaction and developing greater understanding of Agency needs and priorities. The Deputy Assistant
Administrator meets quarterly with his counterparts in the Agency and ORD holds annual meetings with
Assistant and Regional Administrators on their program areas of interest. There is a section in each
StRAP titled EPA partner and stakeholder involvement that will clearly describe how this multi-stage
engagement throughout the year ensures the relevance of ORD research to actual environmental
decisions.

In this document and throughout our work, ORD refers to our EPA colleagues in the program offices and
regions as our partners, in that we are all partners, working together to achieve EPA's goals. We also
use the term partners to include other researchers we collaborate with (e.g., from other agencies). The
term stakeholders generally refers to those outside of EPA who have an interest in our research and may
use it for a variety of applications.

We appreciate the Boards' support for the CSS program's development of "dashboards for decision
makers" to provide access to ORD-generated tools and research. We are learning quite a bit from CSS'
experience with this approach to help make sure the "dashboards" are most effective at providing the
needed information. The SHC program is now beginning development of an EPA web portal to SHC tools
and research for users to access. The ACE program recently released the "Air Sensor Toolbox for Citizen
Scientists." We expect to continually improve and expand user-friendly access to ORD tools and
research.

Recommendation: Describe how decision makers will access and integrate information about the
uncertainties associated with ORD-generated tools and data. In general, users of ORD research need
more information on the reliability of models and forecasts, how to interpret the results of uncertainty
analyses, and how uncertainty analysis will be incorporated into systems-based modeling approaches.
Environmental complexity and biological variability make it critical to conduct broad uncertainty
analyses and present a comprehensive set of results for statistical reliability of models. These models
should include contexts where models are tested and found to be reliable and contexts where reliability
is poor or has not been well-established. Essential elements include model verification, calibration and
sensitivity analyses, particularly in the context of complex, linked models and systems-based models in
which feedback loops may lead to unexpected outcomes.

-------
EPA Response: We agree that understanding the reliability of models is extremely important, and ORD
uses a variety of methods to conduct uncertainty and sensitivity analysis on a wide range of models. EPA
and the National Academy of Sciences have provided guidance on conducting such analyses. Projects,
products and outputs that are developed by the research programs will address this issue more
specifically than the overall strategic plans. For example, in development of the coastal ocean models to
examine Gulf of Mexico hypoxia, a quantitative uncertainty analysis is planned for FY15. In order to
improve estimates of model uncertainty, two different ecosystem models are included in the research
effort. Additionally, ORD participates in the NOAA model inter-comparison study with academic
modelers.

We appreciate this recommendation and agree that we have a responsibility to translate these analyses
to be most meaningful to the end users. As with our research findings, ORD is placing greater emphasis
on translating our work so that it can have the greatest impact. For further discussion of how the
research programs are addressing model uncertainty, please see responses below to specific
recommendations in the SSWR and Nitrogen Roadmap sections of the report.

Recommendation: Clarify how sustainability is operationally defined at ORD and the agency and
articulate how sustainability relates to the specific research planned in each program. Although
sustainability is presented as a central cross-cutting focus of the StRAPs, the formal role of sustainability
is presented only in abstract terms, and sustainability tradeoffs and synergies are not discussed. The
StRAPs do not convey a research agenda driven by the type of detailed, transparent consideration of
sustainability recommended by the National Research Council (NRC 2011). The overall agency strategic
plan appears to adopt the original National Environmental Policy Act definition of sustainability:
"conditions under which humans and nature can exist in productive harmony and fulfill the social,
economic, and other requirements of present and future generations," yet the specific manifestation of
the sustainability concept and relevance for each program's research remains largely abstract and
unstated. In at least one of the StRAPs (HHRA), the word sustainability could be entirely stricken from
the document without any obvious impact. All of the StRAPs would benefit from a more explicit
treatment of sustainability, including how the concept is defined and how it influences the specific
research that is proposed.

EPA Response: Sustainability is a central cross-cutting focus of the ORD research programs and one of
EPA's cross-agency strategies. EPA recognizes that our future depends on environmental protection
efforts that consider tradeoffs and synergies in environmental, social and economic systems. Currently,
EPA's Strategic Plan defines the following principles to integrate sustainability into the Agency's day to
day operations:

1. Conserve, protect, restore and improve the supply and quality of natural resources and
environmental media (energy, water, materials, ecosystems, land and air) over the long term;

2. Align and integrate programs, tools, incentives, and indicators to achieve as many positive
outcomes as possible in environmental, economic, and social systems; and,

3. Consider the full life cycles of multiple natural resources, processes, and pollutants in order to
prevent pollution, reduce waste, and create a sustainable future.

Emphasizing these principles, EPA is working to support states, local governments and communities in
making sustainable environmental decisions. EPA research is developing the science needed to support
such decision-making. The research programs are focusing on how to improve life cycle analysis to
support EPA in making more sustainable decisions. Just a few examples of the sustainability emphasis in
the research programs include the following.

-------
• The Air, Climate and Energy research program is looking at the consequences of different energy
choices to help highlight sustainable choices.

• A major focus of our Safe and Sustainable Water Resources research program is evaluation of
green infrastructure approaches that may be more sustainable. Also, SSWR is emphasizing water
and wastewater re-use, and the concept of wastewater as a resource.

• The Sustainable and Healthy Communities research program is leading the way on sustainability
indicators and decision support tools to help the Agency and the States evaluate decisions
through a sustainability lens.

• The Chemical Safety for Sustainability research program includes work on life cycle analysis
methods, and in molecular design to make products safer from the start.

• The Homeland Security research program is broadening its view to encompass "resiliency" - to
help communities prepare for and recover from environmental aspects of all disasters.

• In Human Health Risk Assessment, development of methods for cumulative risk assessment will
support the consideration of multiple exposure and cumulative impacts in environmental policy
and decisions.

We agree that the StRAPs need to do a better job of communicating how some of the most significant
research efforts are contributing to the Agency's emphasis on a sustainable future.

For decades, EPA has been focused on risk reduction and pollution control. Embracing sustainability is a
very different approach for a large Agency, guided by multiple, specific statutes. As hiring within the
Agency has been very constrained in recent years, our ability to build in-house expertise in social sciences
to help address the economic and social systems aspects of sustainability is moving slowly.

We appreciate the SAB and BOSC raising this issue and keeping it front and center as we prepare for a
new era when systems approaches become more central to our efforts. We continue working with advice
from you and the NRC to take concrete actions to make greater progress in working toward our
sustainability goals. It is a high priority for ORD and your recommendations will help us reconsider how
the StRAPs build sustainability concepts into the research programs.

Recommendation: Explain how ORD will develop or access the social, economic and behavioral
science needed to achieve the goals of the EPA's Strategic Plan. In terms of strategic emphasis, clarity,
and motivations, the StRAPs are best developed in ORD's traditional areas of strength such as the
natural sciences, risk assessment and human health. The draft StRAPs provide good roadmaps for
continuing progress in these areas. Supporting the EPA Strategic Plan's six major goals and focus on
sustainability, however, requires the agency to develop or access social, economic, and behavioral
sciences. Past reviews of ORD research planning (U.S. EPA SAB 2011a, 2012) have noted the lack of a
clear research agenda and expertise required to address important social, economic and behavioral
dimensions of the EPA's goals. This shortcoming remains evident in the current ORD planning
documents. The draft StRAPs are not well developed in the social, economic and behavioral sciences
and in cross-cutting areas that involve these sciences. Although social, economic and behavioral aspects
are mentioned in the StRAPs, the discussion is less sophisticated and developed compared to parallel
topics in natural science, risk and health. Although the relative importance of social sciences varies
across ORD program areas, social and human dimensions are relevant to all areas. ORD should describe
its plans for hiring in the social, economic, and behavioral sciences and for training existing EPA staff in
these areas in collaboration with appropriate academic institutions or professional associations.

-------
EPA Response: We appreciate the SAB/BOSC comments bringing increased attention to the social
economic and behavioral sciences. We are expecting to move forward in this area, and this will be
reflected as we develop the final versions ofStRAPS. Additionally, recent hiring guidance to ORD
managers emphasizes the importance of addressing the science needs identified in the StRAPs. ORD is in
the process of developing a short- and long-term hiring strategy (see discussion in section 5.5 below).

Several StRAPS illustrate the increased focus on challenges related to the socialeconomic and decision
sciences:

• Safe and Sustainable Water Resources StRAP includes as a major Research Objective,

"Quantitate the benefits of water quality." By partnering with EPA's Office of Policy and Office of
Water, and through external STAR grants, the SSWR program will develop the models and tools
needed to estimate the economic benefits of water quality improvements.

• Homeland Security StRAP identifies the science question, "What social and environmental
variables affect a community's environmental resilience? What are indicators and metrics of
resilient communities?" To help the research program address this and related issues, ORD
recently hired a social scientist for the National Center for Homeland Security Research.

• Air, Climate and Energy StRAP includes the science challenge, "What are the social, behavioral,
and economic factors that may hinder the ability of communities and individuals to prepare for
and implement adaptation strategies for climate change and make informed decisions regarding
air quality and carbon pollution ?"

• Sustainable and Healthy Communities StRAP includes a focus on assessing environmental health
disparities in vulnerable groups. ORD plans to expand on work that had emphasized
disproportionate exposure to chemicals. Research will address how environmental stressors
interact with social and economic factors to drive health disparities.

Additionally, we recognize that other research organizations have interest in and are working on these
issues. ORD is making progress in communicating and collaborating with external organizations and
experts; for example, we collaborated with NIH to fund Centers of Excellence on environmental health
disparities research. Through such connections, we can identify and build on existing work and focus our
efforts where EPA can make unique contributions.

-------
3.2 Overall perspectives on proposed research to address environmental issues
of 2020 and beyond

Charge Question 1(b). What are the SAB/BOSC perspectives overall on the proposed research
directions providing research to address environmental issues of 2020 and beyond?

ORD has made significant progress through the StRAPs and roadmaps in placing its research in a
framework of the anticipated major environmental challenges confronting the United States. However,
this charge question is difficult for the SAB and BOSC to address because the ORD draft planning
documents did not distinguish between longer-term vs. near-term needs. The SAB and BOSC offer
recommendations to position ORD research so it explicitly addresses environmental issues of 2020 and
beyond and also offers recommendations for particular areas of focus. Recommendations relating to the
capacity of proposed research described in specific StRAPs to address environmental issues of 2020 and
beyond may be found in the discussion of each program in section 4 of this report.

Recommendation: Provide a more explicit description of the approach used to identify research
necessary to anticipate emerging environmental issues.

Although the draft StRAPs and roadmaps clearly acknowledge the complexity of emerging issues
indicated in the EPA Strategic Plan, more detail could be provided on how these longer-term focal points
and emergent issues might be better anticipated. ORD should better articulate early risk detection
efforts at multiple geographic and temporal scales. A considerable amount of work outlined in the
strategic plans involves assessment efforts, yet these may not be tuned to pick up on outliers that might
be emerging and trend analysis to monitor trajectories of issues not yet in the cross-hairs of EPA
programs and research. Early detection requires ongoing consultation with advisors and a wide
spectrum of partners, stakeholders and experts to identify emerging problems and research needs.

Recommendation: Add a section to each StRAP and roadmap whose purpose is to describe research
needs for the next decade. ORD's proposed research directions are focused on the near future (e.g.,
2016-2019) rather than on 2020 and beyond. Adding to each StRAP and roadmap a section whose
purpose is to describe research needs of the next decade (2020s) and how current research relates to
anticipated future issues would force the discussion. The section should discuss how those anticipated
future needs will be identified and the process for managing the evolution of the research program. The
exercise would also help direct ORD's current research projects and objectives into the future. Although
the SAB and BOSC review does not permit an extended discussion of future environmental issues, this
report suggests that many anticipated issues will arise as a direct result of the tension between growth
(e.g., population, consumption, economic) and finite resources (e.g., natural resources, biodiversity) and
the ways in which a constrained ecological system can best be managed to meet human needs. SAB and
BOSC members suggest that ORD consider as focal points issues that are a combination of stressors,
drivers, and impacts, including, for example: (1) climate change; (2) habitat loss; (3) introduced/invasive
species; (4) eutrophication; (5) chemical contamination; (6) evolving demographics and social systems;
(7) technologies affecting the extraction and use of energy; and (8) continued transformations of land
use and land cover.

-------
EPA Response: The intent of the StRAPs is to describe the strategic research directions for 2016-2019,
addressing the priorities in the EPA strategic plan, including the environmental challenges of the next
decade. The StRAPs articulate a balance of short-term research for near-term use, and long-term,
anticipatory research that will address issues in the 2020s. How well ORD has anticipated the issues of
the next decade in the current StRAPs is an issue we will continue to consider.

Describing research directions and plans beyond 2020 is a separate question. Rather than adding a
distinct futures section to each StRAP, ORD believes that it would be more appropriate to think broadly
about potential future issues and how to begin to address them. For example, we are starting to think
about how to take an integrated systems approach to environmental protection and public health, which
have long been viewed through separate lenses. We can play a leadership role in using new methods and
technologies to assess health and environmental impacts. We need to build partnerships with the public
health community and identify areas for collaboration to advance integrated approaches.

ORD also uses other means to encourage forward-thinking inquiry. Our scientists are continually
scanning the horizon for emerging issues. Our innovation programs, such as the Pathfinder Innovation
Projects, provide opportunities to explore truly creative solutions to complex environmental problems.
Interactions with a wide spectrum of experts have contributed to recent ORD endeavors such as
computational toxicology, air pollution sensors, and green infrastructure for stormwater management.
We agree with the SAB/BOSC that more work is needed to anticipate future issues, and our efforts will
include consultation with science advisors and other partners and stakeholders.

-------
Program-Specific Recommendations for Research

4.1. Air. Climate and Energy

Executive Summary: The EPA's Air, Climate, and Energy (ACE) research program has a strong strategic
plan, linking well to the EPA Strategic Plan and agency priorities, and addressing some of the most
important current and emerging issues facing environmental quality, human health, and society in the
coming decades. The program is exceptionally broad, with its scope encompassing criteria air pollutants,
greenhouse gases, climate change, and energy. Energy, in particular, is an extensive component, as the
life cycle of energy influences all elements of the environment, and overlaps with the other five program
areas.

EPA Response: Thank you for your supportive comments of the ACE program especially as the ACE StRAP
relates to and supports EPA's Strategic Plan. We agree the program is broad and addresses key areas of
importance to the Agency and the public.

4.1.1. Charge Questions 2a and 2b. How well will the research directions in each Early Draft StRAP
(2016-2019) support EPA in achieving the relevant Agency objectives and cross-cutting strategies as
described in the EPA Strategic Plan (2014-2018)? What are the SAB/BOSC perspectives on the
proposed research directions in the StRAP providing research to address environmental issues of 2020
and beyond?

Key Recommendation: Document the progress addressing greenhouse gases resulting from agency
programs and plan the research needed to inform future decisions. While the SAB and BOSC found the
overall structure and substance of the ACE strategic plan to be sound, this report suggests a few changes
that will strengthen the presentation and sharpen the focus. The plan is ambitious, and as such, may
seem to be difficult to achievable. The StRAP should be revised to include a bold statement of what the
EPA can do to forge a better future. A compelling introduction might begin by reporting on the potential
success of the new greenhouse gas regulations in reducing emissions. While the Climate Action Plan is
mentioned, the transformational nature of the EPA's new role in greenhouse gas emissions should be
declared as an example of how the new vision and strategic plan can compel major advances for air,
climate, and energy both nationally and internationally.

EPA Response: We strongly agree with the Committee's visionary perspective and enthusiastic support
for EPA's role and are revising the StRAP and the Climate Research Roadmap to better describe work that
ORD, in partnership with the rest of EPA, can do to move the Agency forward toward that "better
future." We are also revising these documents to more explicitly address emerging issues, such as
application of life cycle analysis to future energy system scenarios, to help prepare for consequences of
energy choices ahead. . Finally, the enormous scope of the problem demands that EPA and ORD work
across federal agencies, with state and local organizations, and with business, industry, academic, and
non-governmental organizations domestically as well as internationally. EPA is committed to this
approach, as demonstrated by its participation in interagency bodies, community outreach, and public-
private partnerships. ORD is fully supportive of EPA's directions, as described in the EPA Strategic Plan
and other Agency directives.

-------
Key Recommendation: Include a conceptual framework linking program elements in the ACE StRAP. A

graphical representation of a conceptual framework at the outset will allow the reader to understand
the scope, focus, and anticipated impact of the program (such as that provided in the presentation given
by Dr. Daniel Costa, side #4 at the July 24-25, 2014 meeting). Such a figure should show linkages among
the elements of the complicated program hierarchy: (a) 3 elements of the program (air, climate, and
energy); (b) the 3 research objectives (assess impacts, prevent and reduce emissions, and
adapt/mitigate); (c) the 5 research topics; and (d) their short- and long-term aims. Such a diagram and
description at the outset might further crystallize motivations and necessary interactions. The objectives
are not presented until quite late in the document, but represent the driving force for the strategic plan.
A crisp vision for each of the research objectives should be succinctly presented at the very outset in
format that links clearly to the conceptual framework.

EPA Response: We have incorporated a conceptual diagram in the StRAP section on Research Program
Objectives that aims to capture the program hierarchy in a simplified manner. The diagram illustrates
the integrated nature of the program with the Topic areas collectively feeding into the Research
Objectives which are encompassed in the overall Program elements of Air, Climate and Energy. The
additional layers recommended in the comments, however, were too many to capture in a
straightforward way. As a result, the long- and short-term aims of the various research topics remain in a
separate table along with examples of research related to the aims.

Recommendation: Clarify relative priorities, with respect to budget distribution and interactions with
other agencies. The scope of the program is very large, but investments in the individual components
(air, climate, and energy) are skewed, with a very large proportion of the effort focused on the "air",
relative to the "climate" and "energy" elements. This occurs both as a result of the traditional focus on
criteria pollutants within the EPA, and because other federal agencies deploy enormous resources
toward climate and energy research issues [e.g. National Oceanic and Atmospheric Administration
(NOAA), the Department of Energy (DOE), the U.S. Global Change Research Program], Explicit
recognition of the priorities, how they are reflected in the budget, and in turn how ACE anticipates
resource allocations shifting as a result of the strategic plan should occur early and clearly in the
document. Targets for inter-agency actions that will assist ORD in meeting its climate and energy goals
should be elaborated. Research on mitigation represents a special opportunity for collaborative work
that could be led by the EPA.

EPA Response: We appreciate the need to provide an indication of relative priorities among the three
primary elements of the program, namely Air, Climate and Energy. The relative levels of investment are
determined by Congressional appropriation of resources to the Clean Air and Global Change research
categories. Within those bounds, the relative priorities are envisioned to change over the course of this
StRAP (2016-19) consistent with the strategic directions of the Agency. We have indicated in the StRAP
that we anticipate growth and emphasis in climate change and energy research over the course of this
StRAP with a commensurate reduction in historic air pollution related research. The vision of the revised
StRAP is to be even more integrative in both approach and in interpretation and recommendation. A
solutions perspective requires a strong interdisciplinary core in design. The StRAP also addresses our
strong and continuing interactions with other Federal agencies, particularly through the subcommittees
of the Committee on Environment, Natural Resources and Sustainability (CENRS) and their staff-level
interagency working groups. Collectively, the participating Federal agencies assist EPA in meeting our
climate and energy research goals.

-------
Recommendation: Elaborate and/or expand the research to be conducted on mitigation of climate
change impacts. The current document (in the research topic on Climate Change Impacts, Mitigation,
and Adaptation) is unclear with respect to what work is planned related to mitigation. For instance,
there are no short-term goals at all related to mitigation. The SAB and BOSC recommend proposing
tractable work in this arena. Connecting the work to the Intergovernmental Panel on Climate Change
(IPCC) report on mitigation would be helpful.

EPA Response: We recognize the need to more effectively communicate the work we are conducting
related to mitigation, much of which is taking place under the Sustainable Energy and Mitigation (SEM)
projects. These projects were formerly referred to as Sustainable Energy Evaluation. The topic is being
renamed to reflect this additional focus on mitigation. We -with ORD and other EPA partners - are
actively engaged in updating the ACE StRAP to develop the most effective structure for addressing
mitigation-related issues Such a structure will include short-term goals, and is being developed with a
clear understanding of the breadth of issues and challenges addressed in the IPCC Working Group III
report on mitigation and by other federal and state agencies. This information will be covered in the
section of the StRAP that describes our research topics.

Recommendation: Focus the distributed monitoring of air quality on quality data collection and
distribution to citizens. The current document (in the research topic on emissions and monitoring)
mixes two endpoints of a spectrum of environmental data from, on one end, accurate and precise
regulatory-quality data from a limited number of sites, to the other end, ubiquitous citizen-science
generated data of uneven overall accuracy and precision. The SAB and BOSC recommend that ACE work
with and motivate entrepreneurs for the development of extensive high quality data that are available
to civil society in real time and potentially available for use for regulatory purposes. The rapid advances
in sensing technology, the concomitant increases in accuracy and precision, and decreases in cost have
highlighted the potential to deploy environmental sensors at orders of magnitude greater density than is
currently the case. When combined with effective visualization, it is possible to provide civil society with
a much greater understanding of variations in environmental quality at a scale that matters to
individuals. ORD is well positioned to help ensure that the quality of the data that flows from this sensor
revolution is both accurate and inter-compatible. Given the large private investment already being made
in the development of sensors, a convening and coordinating role is where ORD's investment would
have the largest leverage.

EPA Response: We appreciate the insights of the SAB and BOSC on our sensors research. We fully
support the recommendations particularly related to working with and motivating entrepreneurs for the
development of sensors. The StRAP has been revised to clarify our approach to sensor development and
evaluation under the Emissions and Measurements Research Topic. A coordinating and motivating role
has been one approach among several that we have taken as the field has progressed. We will be
focusing our efforts on sensors related research to be consistent with this motivating role to maximize
our utility in helping to advance the field of air pollution sensors. We will also maintain our efforts in
methods development for ambient and source measurements, including a role for sensors. We are
continuing to work closely with EPA's Office of Enforcement and Compliance Assurance (OECA) to test
new technologies at industrial facilities. OECA and EPA regional offices have been great supporters of our
Village Green project which uses new, smaller instruments installed beneath a park bench to collect real-
time hiqh-quality air pollution data, and the data is made easily web-accessible. This project is a good
example of how we can use new technologies to make local-scale, high quality data readily available to
the public. Also, ORD has sponsored three citizen-entrepreneur workshops on sensors, provided access to
a testing platform for assessment of sensor performance, and will be sponsoring a workshop in the

-------
summer of 2015 specifically targeting citizen use of air sensors. We agree with your perspective and
envision in time that the sensors will be an essential part of a large integrated network of various sensing
modalities blending regulatory monitoring networks, satellite based sensors, citizen and community
sensors and an integrated air quality modeling fabric fusing these approaches.

Recommendation: Consider explicit focus and analysis of agricultural sources and other land use
contributions to air pollution. Agricultural sources of air pollutants are significant. These sources include
hazardous air pollutants, ammonia, methane and nitrogen dioxide fluxes stemming from fertilization
and livestock, particulate matter from cultivation practices, and both direct and indirect impacts of
energy use in agricultural production. The draft StRAP (essentially all research topics: Climate Impacts,
Mitigation and Adaptation; Emissions and Measurements; Modeling and Decision Support; National
Ambient Air Quality Standards and Multipollutant; Sustainable Energy Evaluation) is silent on
agricultural sources, and their inclusion in the strategic plan and subsequent research is important.

EPA Response: We agree that agriculture is an important source of air pollutants for both an air and
climate change perspective. We have added discussion in the StRAP to point out the important linkages
between agricultural sources and air pollutants and climate change. In addition, the Nitrogen & Co-
pollutants draft Cross-cutting Research Roadmap discusses research on agricultural sources of nitrogen
across the EPA research programs. Moreover, we have recently been working closely with EPA's Office of
Air and Radiation on the analyses of data from the National Air Emissions Monitoring Study, whose
purpose is to develop improved emissions from agricultural operations.

Recommendation: Provide more specific targets for the short-term research aims. The current table in
the strategic plan provides both short- and long-term research aims for each of the research topics. The
short-term goals are likely still too diffuse and it will be difficult to identify metrics that will allow
evaluation of success. The SAB and BOSC recommend more specific targets focused on key knowledge
gaps that can be used to define those metrics and actionable work plans.

EPA Response: We agree and have included specific examples of short-term research to illustrate that
key knowledge gaps are being addressed with specific research. The final StRAPs will include an appendix
of planned research outputs and year of completion.

4.1.2. Charge Question 2c. Did the presentations and plans indicate that ORD is designing for
integration, where appropriate, on topics that are relevant to other research programs?

Recommendation: Consider specifying projects that integrate ACE with other programs. The current
ACE strategic plan presents opportunities for integration with other programs but does not identify
goals for integrated research. Such goals for integrated projects would assure that the work occurs. The
Sustainable and Healthy Communities program provides especially good synergy, with respect to the
Emissions and Monitoring research aim, and the Sustainable Energy Evaluation research aim. Neither of
these relationships is currently identified.

EPA Response: The StRAP has been modified to include a more detailed discussion of integration across
the research programs, along with specific examples. Additional detail has also been included regarding
the ACE connections to the four ORD cross-cutting research roadmaps.

-------
4.1.3. Charge Question 3: Does the SAB/BOSC have suggestions regarding how ACE should target its
efforts to understand, model, and convey the potential environmental impacts of possible energy
choices?

Recommendation: Consider incorporating energy efficiency/conservation research. The StRAP
(Research Topic 5: Sustainable Energy Evaluation) avoids the topic of energy efficiency and energy
conservation, even though energy use represents the single largest source of pollutants and increased
efficiency is one of the most achievable means for reducing energy-related impacts. The EPA's Office of
Atmospheric Programs manages the Energy Star program, which emphasizes the implementation of
incentives for energy efficiency on a small-scale. Will ACE conduct research related to the behavioral and
economic forces related to energy efficiency or on the opportunities for innovation in the arena of
energy conservation and efficiency?

EPA Response: We have modified the StRAP to include a discussion that describes our energy research
that incorporates the ability to evaluate energy efficiency and conservation into analyses using the
Market Allocation model (MARKAL). Models such as MARKAL are at the core of our current and planned
efforts to address mitigation through decision making on energy choices. We do not see ourselves
heavily invested in mitigation technologies as this falls to the purview of the Department of Energy
(DOE). ACE is also connected to the work of EPA's National Center for Environmental Economics and the
analyses supported by the Office of Atmospheric Programs, both of which are looking into economic and
behavioral factors as they relate to energy consumption and efficiency. These cross-EPA connections, as
well as interactions with energy modelers in DOE and outside institutions, have resulted in initial steps
toward additional research in this area of behavioral science.

Key Recommendation: Plan to incorporate renewable energy scenarios and pathways developed by
other organizations in ORD's analysis of environmental impacts. ORD should plan to use the renewable
energy scenarios and pathways developed by other organizations as bases to analyze environmental
impacts of importance to the EPA. For example, because of EPA's significant expertise in applied life
sciences, it would be useful to engage DOE and other relevant agencies in the development of synthetic
biology methods, which are already in research and development in the private sector as an alternative
means of chemical synthesis and renewable biobased energy. The StRAP should include more discussion
of ORD's coordinating efforts with DOE, the U.S Department of Transportation, U.S. Department of
Agriculture and NOAA around likely scenarios for alternative fuels, vehicle standards, conservation,
renewables and the reasons for their selection.

EPA Response: ORD has strong ties to the energy modeling community, both directly with DOE's Energy
Information Administration and through the Energy Modeling Forum. This community shares insights
and approaches across a range of topics, including alternative fuels, vehicle technology advances,
biomass energy, and behavioral responses. ORD, as well as EPA's Office of Air and Radiation and
National Center for Environmental Economics, follow advances in these areas and interact as
appropriate with experts from DOE, USD A, and other agencies and institutions to ensure we are aware of
possible new trends and approaches related to energy scenarios and pathways. Regarding synthetic
biology, the state of development of these technologies is such that incorporating them into current
scenarios at a meaningful scale is not possible; it is just too speculative at this stage. While we are paying
attention to these developments, at this point they are more appropriately addressed in terms of new
chemicals and materials.

13

-------
4.2 Safe and Sustainable Water Resources

Executive Summary: The Safe and Sustainable Water Resources (SSWR) StRAP outlines research
activities in support of the EPA's Strategic Plan's goal of Protecting America's Waters. The major
research topics were developed from an overarching theme of maintaining environmental, social and
economic sustainability in the face of significant stressors, including climate change, extreme events,
land use, aging infrastructure and population growth

4.2.1. Charge Question 2a. How well will the research directions in each Early Draft StRAP (2016-2019)
support EPA in achieving the relevant Agency objectives and cross-cutting strategies, as described in
the EPA Strategic Plan (2014 -2018)?

Key Recommendation: Prioritize research in order to allocate resources across research areas,
balance immediate and long-term needs, and leverage areas of strength completed by partnerships.

The four priority areas in the StRAP - watershed sustainability, nutrients, green infrastructure, and
water systems - align well with the Strategic Plan at a high level and represent a balanced plan for the
next four years. ORD is commended for the efforts taken to parse all of the potential research
activities into four focused target areas, but it is essential to prioritize even among these objectives
given declining budgets. SSWR must determine how to allocate resources across research areas and
strike a balance between meeting immediate needs of the Office of Water while continuing to work
toward longer-term strategic objectives.

There are several agencies actively involved in maintaining the health of the nation's waters. SSWR
should identify areas of strength for the EPA and strive to make significant advances in those areas
while developing complementary partnerships with other agencies and stakeholders.

EPA Response: Thank you for the supportive comments. We agree there is a need to indicate relative
priorities among these four areas and the final StRAP will address this is greater detail. The SSWR
program has worked closely with the Office of Water and the Regions to develop a prioritized list of
research needs that serves as the basis for planning the scope of work and timelines for our research
products and outputs. We envision the priorities will be modified during 2016-2019 to be consistent with
the Agency's strategic directions and the allocation of resources as determined by Congressional
appropriations.

The program allocates resources for near-term and longer-term objectives in several ways, and this will
be made more explicit in the final StRAP. For example, the Water Systems topic has three continuous,
integrated projects that aim to advance the next generation of drinking water and wastewater systems.
The projects transition from current systems and regulatory support (immediate research needs) to next
steps in technology advancement and transformative approaches and technologies (intermediate- to
longer-term research needs). The NCER grant program is another example of how SSWR allocates
resources to intermediate- and longer-term needs.

We recognize there are more than twenty federal agencies with jurisdiction of water-related research.
Coordination of research activities among many of these agencies is enabled through the SSWR National
Program Director's role as co-chair of the Subcommittee on Water Availability and Quality (SWAQ)-an
interagency forum on water resources that advises and assists the White House Office of Science and

-------
Technology Policy (OSTP) Committee on Environment, Natural Resources, and Sustainability (CENRS).

The SSWR program has benefitted from having knowledge of science and technology advancements
by the public and private sectors, and strategically identified EPA's areas of strength where we can
make significant advances. For example, several agencies and research institutions are investing in
research to project water availability, especially in terms of potential shortages related to natural
drought cycles and climate change. SSWR will build on this information to develop solutions for
water demand, such as water reuse, that rely on EPA's expertise in water quality.

4.2.2. Charge Question 2b. What are the SAB/BOSC perspectives on the proposed research
directions in the StRAP providing research to address environmental issues of 2020 and beyond?

Recommendation: Plan for the human resource needs required by increased collaboration,
integration and partnerships. As SSWR works to meet its research goals, partnerships will be
essential. While the EPA should focus on identified research areas, the agency cannot move forward
without strong collaborations across other agencies, both within the United States and
internationally. Within the EPA, collaboration should occur at the ORD level, not just at the level of
individual researchers. As the need for collaboration increases, additional staff dedicated specifically
to developing and maintaining integration within and outside of the agency will be needed.

As SSWR plans for future research areas and begins to recruit researchers with the necessary
expertise, it should develop a strategic plan to adequately meet human capital needs. Some
examples include: strategic use of postdoctoral fellows, development of methods to foster mobility
for current EPA scientists, and hiring of social scientists.

EPA Response: We agree. In addition to active participation in SWAQ with U.S. federal agencies, since
2003 EPA has a formal partnership the Global Water Research Coalition (GWRC). The aim ofGWRC is to
leverage funding and expertise among its international member organizations, coordinate research
strategies, and actively manage a centralized approach to global issues.

As described above in response to the over-arching recommendation on the need for social sciences
expertise, ORD is working across its Research Laboratories and Centers to develop a corporate workforce
strategy that addresses identified gaps and includes near-term and longer-term elements for enhancing
the workforce.

Recommendation: Anticipate regional needs and changing demographics. SSWR should continue to
anticipate regional issues as well as understand the implications of changing population
demographics. Research on regional and watershed scales (e.g., wildfires, drought) should be
included in overall research activities. Changing populations will affect the size and characteristics of
populations being exposed to certain contaminants, as well as the types of contaminants present in
water and other environmental media. De-population of urban areas (e.g., Detroit) should also be
considered as this phenomenon may affect resiliency to extreme events, for example.

A stronger focus on invasive species related to regional and national needs should be
included, including implications of climate change and modeling/prediction of secondary
transport.

-------
EPA Response: We agree. The SSWR program has already invested $4.1M in NCER grants for
research on the impacts of drought and/or wildfire on water quality. This topic was directly
responsive to discussions with EPA Regions in the Southeast, Central Rockies, West and Southwest.
Changing populations is one of the overarching drivers of the four topic areas. We note 'population
change' is a better term than 'population growth' and have made this edit in the StRAP.

We also agree invasive species is an important topic; however, this is a classic example of the need
to prioritize research objectives and leverage resources with other agencies in the era of declining
budgets and FTEs. The departments of Commerce, Agriculture and Interior are the leaders in this
area, and as such they co-chair the National Invasive Species Council, which was established by
Executive Order to ensure effective and efficient coordination of federal programs and activities.
The Council will release its draft strategic plan in spring 2015. This information is expected to
identify how the EPA can uniquely support invasive species efforts. SSWR will fulfill its existing
commitments to research on ballast water treatment technology and to support Region 5 and the
Great Lakes National Program office on early detection sampling methodologies and DNA
sequencing of samples collected in ports for the upcoming field season. However, we plan to phase
out this work in the next two years and redirect resources to other consolidated efforts, such as
early detection of harmful algal blooms.

Recommendation: Develop models with appropriate capability to communicate uncertainties. In

general, model results should clearly communicate uncertainties and limitations to decision makers
and the public. Wherever possible, uncertainty should be quantified. This can be done by
complementing the use of complex process models that do not allow estimation of uncertainties
with simpler probabilistic models that lend themselves to uncertainty analysis.

EPA Response: We agree and SSWR is keenly aware of and routinely conducts sensitivity and
uncertainty analyses using various methods on a wide range of models. We provide one example below,
this and other examples are provided in the response to comments on the Nitrogen and Co-Pollutant
Roadmap.

The use of two different ecosystem models (GEM and GoMDOM) and participation in the NOAA coastal
ocean modeling testbed (COMT) model intercomparison study with academic modelers will improve
estimates of model uncertainty. Uncertainty in parameterization of model coefficients can be mitigated
by calibrating the model to a good observation dataset. ORD oceanographic data collection has resulted
in an unprecedented 5+ year peer-reviewed dataset to calibrate against. In GEM the subjectivity of
calibration is removed by using an evolutionary optimization algorithm that searches for the best
parameter set (within specified bounds) to fit the observations. In GoMDOM, a more traditional
approach is used of iteratively changing a parameter and examining model i/s observation. The degree of
fit between the model and observations is then quantified using standard error metrics such as bias and
root mean square error and calculate model prediction skill. To put error bars on the model results,
parameter uncertainties must be propagated through the model.

Efforts in FY15 include error propagation work using a Markov Chain Monte Carlo approach through the
EPA Computer Center, EMVL. Other uncertainty issues that will be specifically addressed in the research
include the issue of non-unique parameter sets that may provide the same or very similar results,
uncertainty in the model structure, and uncertainty in the boundary conditions.

-------
We are looking forward to future interactions with the BOSC to articulate our approach to uncertainty
and to communicate the results of the analyses.

Key Recommendation: Communicate the concept that wastewater is a resource. SSWR should actively
promote the transition of the term "wastewater" to "resource" to reflect that wastewater is a
secondary resource that has potential for multiple beneficial reuses, both as a source of water but also
as a source of trace metals, nutrients and other resources.

EPA Response: We agree. A new and substantial area of investment in the SSWR research program
is on transforming the concept of 'wastewater' to 'resource'. To promote adoption of this concept,
the conventional term 'wastewater' has been replaced with 'post-use water' in the revised StRAP;
however, we would appreciate advice from the SAB and BOSC on the choice of this term.

4.2.3. Charge Question 2c. Did the presentations and plans indicate that ORD is designing for
integration, where appropriate, on topics that are relevant to other research programs?

Recommendation: Increase integration with ORD's Human Health Risk Assessment and ACE
programs and with other federal agencies. The SSWR research plan is generally well integrated with
other programs. The plans indicate linkages between the four priority research areas in SSWR and
other programs. Strong linkages exist with Sustainable and Healthy Communities, but integration
with Human Health Risk Assessment should be increased. Additional focus on the water-energy nexus
would strengthen the existing integration with the ACE StRAP. Beyond integration among the ORD
programs, the SSWR StRAP should reflect improved engagement and collaboration with other
agencies, particularly USDAand DOE.

EPA Response: We are in the process of exploring linkages with HHRAfor human health impacts related
to three of our four topic areas (watershed sustainability, nutrients, and water systems). Representatives
from HHRA will be attending our upcoming annual research meeting, where they will learn more about
SSWR research activities during our poster sessions and then participate in a panel discussion on cross-
Agency opportunities for collaboration and integration.

The water-energy-food nexus is another growth area for the SSWR program, and we appreciate the
response below (4.2.4) to our program-specific charge question 5.a on how EPA can make a significant
research contribution in this area. SSWR is very active in this area, not only by leading an initiative
through SWAQ, but also serving on multi-sector panels. Recent examples in one month alone include a
joint NSF-DOE-EPA workshop on Energy Positive Water Resource Recovery, an invited roundtable
discussion on the Energy-Water Nexus hosted by Secretary of Energy Ernest Moniz, and the President's
Council of Advisors on Science and Technology (PCAST) plenary session on how science and technology
can foster adaptation and mitigation of such problems as drought - a significant issue for both the
energy and water sectors. In August 2015, SSWR will also serve on a panel with representatives from the,
States, Congress, DOE, Navy and others at the Intermountain Energy Summit on the Water-Energy
Nexus. SSWR and ACE are working productively to integrate efforts specific to water and energy, and
also study how climate may impact the food-energy-water nexus.

-------
4.2.4. Charge Question 5a. Where can EPA make a significant research contribution in moving toward
a sustainable water-energy future, with consideration of energy, water, nutrients, and other
resources?

Key Recommendation: Build on the EPA's dual role of research and regulation to identify a unique
research role in moving toward a sustainable water-energy future. The StRAP positions the EPA well
in the water-food-energy-climate space and builds on the EPA's dual role of research and regulation.
Since there are many agencies with interest, expertise and resources dedicated to the water-energy
nexus, it is important that the EPA carve out a unique role by partnering with other players and actors
in this area and avoid duplication of efforts.

The EPA's work in providing technical assistance to municipalities is useful, particularly for small
systems that lack a research budget. Expertise in areas such as water reuse and systems analysis will
help to reduce energy needs in water treatment and distribution, as well as to develop a life-cycle
approach for water. Strengths in fate and transport modeling, uptake and human and environmental
risk analysis are additional assets for the EPA. Additional focus could be added to climate
uncertainties, understanding extreme conditions and events, and developing mechanisms for adapting
to and managing these situations.

The EPA could make a significant contribution in deployment of policies and technologies to address
the water-energy nexus (e.g., social and technological aspects of green infrastructure adoption,
development of monitoring technologies and approaches).

EPA Response: We agree. SSWR is already actively engaged with other agencies as well as other public
and private sector organizations to carve out the unique niche area where EPA can make the most
substantial contribution. The guidance provided by the SAB/BOSC is greatly appreciated and will be
reflected in the final StRAP on areas of investment where EPA can make the greatest impact.

-------
4.3 Chemical Safety for Sustainabilitvand Human Health Risk Assessment

Executive Summary: Overall, the draft StRAPs for the EPA's Chemical Safety for Sustainability (CSS) and
Human Health Risk Assessment (HHRA) research programs are scientifically robust and well aligned to
the overarching EPA Strategic Plan. In fact, the programs were considered to be on a path to
revolutionize chemical safety assessment and viewed as leading the field.

4.3.1. Charge question 2a. How well will the research directions in each Early Draft StRAP (2016-2019)
support EPA in achieving the relevant Agency objectives and cross-cutting strategies, as described in
the EPA Strategic Plan (2014-2018)?

Key Recommendation: Communicate more effectively the priorities within the programs and the
approach to priority setting. At a high level the research directions appear to align with the highest
priority issues, but the programs are extremely ambitious in their scope with plans that include enough
research to last well beyond 2020 yet with not enough resources to accomplish everything within the
2016-2019 time frame. Subsequent versions of the StRAP should clearly indicate those projects which
are considered the highest priority and can be realistically accomplished with available resources. In
setting priorities, the SAB and BOSC recommend that the landscape of relevant global research activities
be considered in order to avoid duplicative efforts and ensure that the EPA is working in areas which
best play to its strengths. Also it will be important to resist the temptation to de-prioritize the long-term
programs in order to overcome budgetary constraints on more urgent deliverables.

EPA Response: The CSS and HHRA programs agree with the SAB and BOSC recommendations and will
make priorities and timelines for milestones more explicit in the revised StRAPs. We are striving to
maintain an appropriate balance of near-term and long-term priorities. In CSS, the construct of the
research strategy is designed to drive the longer science vision, for example in Adverse Outcome
Pathway Discovery and Development (AOPDD). But within each project, specific case studies are being
developed in conjunction with program and regional partners to reflect the near term priorities of the
Agency. This approach to implementation resulted in a near-term focus on endocrine disrupting
compounds (EDCs) as individual chemicals and mixtures in the AOPDD project, wherein relevant adverse
outcome pathways will be developed. CSS will work closely with its partners to evaluate the application
and relevance of these pathways to prioritization, screening, testing, and regulation of EDCs. In HHRA,
higher priority will be placed on resolution of critical methodological issues that challenge risk analysis
approaches in general or that can inform assessment priorities in the FY16-19 time frame. For example¦,
advancing decision analytic and probabilistic approaches to more fully characterize dose-response
functions and uncertainty will better inform benefit-cost analyses, a priority area for our program and
regional partners.

Key Recommendation: Clarify the intended uses of new tools. Both the CSS and HHRA StRAPs speak in
high-level terms about generating new tools which should enable the Agency to make "better informed,
more timely decisions about chemicals." While the phrase "fit for purpose" was frequently used
throughout both documents, limited information was provided on what those purposes were.

Therefore, it is recommended that more information on intended applications on new methods be
provided to ensure that the research products delivered by the program are actually fit for purpose in
the eyes of end users. Since National Program Directors for both programs were able to provide "fit for
purpose" projects that they had prioritized, the SAB and BOSC recommend that these example details
be added to the plans. Some suggested applications of new tools might include:

-------
• Setting priorities among the research programs and plans in the StRAP

• Using ToxCast + ExpoCast data to increase the throughput of Provisional Peer-Reviewed Toxicity
Values (PPRTVs) and to increase the certainty of the ones already generated; and

• Creating new integrated approaches to testing and assessment (lATAs) and/or new Organisation
for Economic Co-operation and Development (OECD) testing guidelines.

EPA Response: The CSS and HHRA programs agree with the SAB and BOSC recommendations for
improved communication about the intended uses of new data and tools, and the purposes for which
new types of information may be aligned. As noted by the SAB/BOSC, the range of decisions that new
types of data may inform is quite broad, for example, from prioritization of chemicals for screening and
testing, to evaluation in monitoring programs, to use in support of risk assessments. As described above,
in the CSS program, projects are implemented through specific case studies developed in collaboration
with program offices and regions. This case study approach ensures that the purpose and the application
of the tools is clearly defined upfront in collaboration with the partners and that the product developed is
fit for the intended purpose. In addition, in CSS, the Translation and Knowledge Delivery Topic
incorporates both partner-driven short term projects and projects through which the applicability of the
emerging science will be demonstrated and evaluated (or conceptually 'test-driven') early on before
going too far down a research path. In the HHRA program, application of emerging data streams and
approaches will provide application characterization to critical assessment products. For example,
hazard identification for HHRA assessments will be informed by application of high-throughput platform
case studies, being conducted via collaboration between HHRA and CSS.

The final StRAPsfor both programs will provide additional clarity regarding the approach for 'fit for
purpose' development and application of new data and tools.

4.3.2. Charge Question 2b. What are the SAB/BOSC perspectives on the proposed research directions
in the StRAP providing research to address environmental issues of 2020 and beyond?

Recommendation: Build confidence in new approaches for assessing safety. The CSS program in
particular conveys an overall tone of exuberant enthusiasm, which give the impression that the new
tools are ready for use today. In reality, some new tools are very close to being ready for Agency
application, but many others will require much more work to prove their validity and utility. As the
program contemplates its strategic direction beyond 2020, it would be extremely useful to internal and
external stakeholders to have an approximate timeline describing targets for transferring research
products into actual practice within the Agency. Such a time line would foster a methodological, step-
wise transition from conventional to modern "21st century" methods and approaches, and would
ultimately build stakeholder confidence. Toward this end, an iterative approach to tool creation,
evaluation, and application is strongly recommended in order to maintain confidence during this period
of transition. This approach should involve partnerships between researchers, end users and key
stakeholders.

A number of other activities for building confidence in new approaches also are recommended,
including:

• Quality control to verify the accuracy of high throughput/high content data already collected;

• Demonstration projects in collaboration with program offices to show how new methods can be
put into practice. It is recommended that highly experience risk assessors within the Agency be
involved in such projects;

-------
• Evaluation projects to qualify new methods for specific assessments;

• Leveraging of human data to evaluate the ability to predict human toxicity.

• Contingency plans to account for the possibility that predictive models may not always be
predictive (especially for highly complex end points, susceptible life stages, etc.); and

• Bridging of newer molecular and cellular level toxicity end points with conventional toxicology
end points through the use of systems- and Adverse Outcome Pathway-based approaches.

EPA Response: Development, implementation, and confidence building in tools developed in CSS are
being done through case studies and in direct collaboration with program and regional partners, in
addition to collaboration with the HHRA program. For example, tools being developed for prioritization
and screening of the Endocrine Disruptor Screening Program (EDSP) are developed in collaboration with
the Office of Chemical Safety and Pollution Prevention's (OCSPP) Office of Science Coordination and
Policy (OSCP). Working with the partners in developing the tools through their and ORD's stakeholder
engagement processes and scientific review activities ensures that the learning and confidence building
occurs along the way as the tools are being developed to fit the intended purpose. Similarly,
computational and modeling tools being developed to assess the impact of pesticide exposures to
endangered aquatic species is being developed in collaboration with the Office of Pesticide Programs
(OPP), as well as partners in other federal agencies, regions and states. Partners are not asked to adopt a
final product or tool. Rather, they are engaged and involved in the design and development of the tools,
their early adoption for application to case studies, and the exercise of confidence building. In each fiscal
year, we are focusing on about three of these tools (in varying stages of development) so as not to
exhaust the capacity of our program or our expert partners.

Similarly, in the HHRA program, development of methods and tools for risk assessment is being done in
collaboration with our program and regional partners. The HHRA program is designed to advance risk
assessment methodology to be responsive to NRC vision recommendations, including Tox21 and
Exposure21. For example, HHRA researchers are evaluating the exposure-dose-response continuum both
from a "bottom-up" perspective represented by understanding molecular initiating events (MIE) for
chemicals with different physicochemical properties and potential adverse outcome pathways (AOP); but
also by devoting a project research area with a "top-down" perspective aimed at developing approaches
for disease-based data integration. The HHRA program intends to foster stronger collaboration with the
CSS program to support further application of mechanistic insights and understanding of AOP in
assessment products.

4.3.3. Charge Question 2c. Did the presentations and plans indicate that ORD is designing for
integration, where appropriate, on topics that are relevant to other research programs?

Recommendation: Continue integrating in a targeted and purposeful manner. Integration is clearly
and deliberately emphasized in both StRAPs. In fact, both programs have already come a long way in
terms of integrating and collaborating with other programs inside the EPA and with Federal partners, in
no doubt due to a robust process in place to drive the integration. CSS/HHRA are expanding integration
by working with, or planning to work with, the World Health Organization, the European Commission
(European Chemicals Agency, the Joint Research Centre), OECD, and China. Program Directors for CSS
and HHRA discussed their revised approaches to work with these groups using a targeted approach that
addresses the EPA's needs. Given the time consuming and labor-intensive nature of integration, ORD's
targeted and purposeful approach to integration is applauded. As mentioned previously, it is important
to know what other organizations are doing in this space so that the efforts to integrate are synergistic
rather than duplicative. One important area of integration which appears to have been overlooked is

-------
the need to pay close attention to large-scale chemical assessment programs in other regions of the
work (e.g., European Union's REACH, Korea and China) that continue to generate animal toxicity data on
hundreds, if not thousands of chemicals. Can the EPA partner with these organizations on the
development of alternative, non-animal approaches? Can some of the subchronic animal data
generated in REACH be used to predict longer-term toxicity endpoint (notwithstanding issues around
access to proprietary data)?

EPA Response: As reflected in the draft StRAPs and the discussion with the SAB/BOSC, with the
successful'stand up' of the various ORD national research programs ORD has moved to enhance
integration across the national programs, and with other organizations. As noted, successful integration
is time consuming and labor-intensive, and the HHRA program has targeted efforts through a
cooperative agreement with WHO to specifically collaborate on evaluation of priority chemicals. Further,
the National Center for Environmental Assessment has the lead role for EPA in the WHO Chemical Risk
Assessment Network, a cross-organizational coordinating group including the International Program on
Chemical Safety (www.who.int/IPCS).

The CSS program's partner and stakeholder community includes the international community. A few
examples include:

• In collaboration with regulatory colleagues in OCSPP, holds regular meetings and with the
European Chemical Agency on coordinating and exchanging data and tools, such as read-across.
Most of the REACH data thus far have been proprietary and not open to public access, even on a
government to government level. Nevertheless, the continual engagement is leading to collateral
coordination opportunities.

• The European Commission's Joint Research Center (JRC) is a partner with CSS and the
ToxCast/Expocast research efforts within.

• CSS is working closely with Health and Environment Canada, to collaboratively and share as
much 'new' data including computational data, as possible. This collaboration extends to the
application of these data to risk-based prioritization.

• Many of the methods and data developed in CSS that are approaching adoption by the program
offices are evaluated and harmonized through the various OECD working groups. This includes
tools for existing chemicals as well as emerging materials, such as nanomaterials.

• CSS has also been building strategic bridges with countries in Asia, including China and
Singapore, primarily for the purpose of coordinating research activities and data and knowledge-
sharing.

• CSS has been participating in WHO International Agency on Cancer Research (IARC) most
recently as experts for evaluation of computational and high throughput screening data in IARC
assessments.

• Through the National Nanotechnology Initiative (NNI) CSS is engaged in international
partnerships for funding and conducting research on the environmental health and safety
implications of nanomaterials.

-------
4.3.4. Charge Question 6a. Please comment on approaches the HHRA research program might target
to better tailor its exposure and response assessment approaches to address fit-for-purpose
characterizations (e.g., risk prioritization, risk screening, risk assessment).

Key Recommendation: Advance exposure-based approaches. There was strong support for the use of

risk-based approaches which combine hazard identification, dose-response assessment and exposure, as
these should enable the Agency to allocate resources to the problems that will have the greatest impact
on public health. In fact, the effort to obtain exposure data on a large universe of chemicals in different
products at a variety of life stages is truly game-changing. Incorporation of exposure into early stage
screening (e.g., endocrine screening) will make such screening more effective and meaningful.

EPA Response: The HHRA program has a long history of advancing the area of exposure science and
applications. Its web-based ExpoBox provides guidance on how to conduct exposure assessment for
various scenarios; and the compiled information is continually updated with exposure factors that have
been vetted by the scientific community in support of its Exposures Factors Handbook. A near-term
project in the FY16-19 portfolio will extend exposure factor guidance to ecological receptors in support of
HHRA cumulative risk assessment efforts. Further, computational and modeling tools being developed in
CSS are being used to assess the impact of pesticide exposures to endangered aquatic species in
collaboration with the Office of Pesticide Programs (OPP), as well as partners in other federal agencies,
regions and states. The HHRA program also recognizes that new exposure measurements via sensors are
rapidly emerging and is now partnered with the NIOSH Center for Direct Reading and Sensor
Technologies to characterize analytical considerations and develop interpretation guidance for the use of
these data to inform risk assessments. This program builds on an integration area already established
with the Air, Climate and Energy (ACE) research program.

4.3.5. Charge Question 6b. Please comment on approaches proposed by CSS and HHRA research
programs to identify and integrate novel data streams to develop innovative fit-for-purpose
assessment products.

Recommendation: Take a methodical, step-wise approach to incorporation of novel data streams. As

a general concept, the SAB and BOSC support the increased incorporation of novel data streams into
EPA assessments. With respect to high throughput and/or high content data, there are questions about
what these new data are telling us and how they will be used in different types of assessments. As
mentioned previously, the StRAPs would benefit from more clarity and detail around the intended uses
of these new data for various agency programs as the approach to using new data streams will vary
between different tiers of assessments (e.g., screening vs. quantitative risk assessment). As the field
develops approaches for using modified experimental techniques including but not limited to pathway
based approaches, new in vitro models and modified in vivo models, the CSS and HHRA programs should
use a transparent and stepwise approach to incorporate these novel data streams. CSS and HHRA
research should leverage human disease literature and human epidemiology data from sources such as,
but not limited to, the National Institutes of Health, Food and Drug Administration and Centers for
Disease Control. Use of the National Health and Nutrition Examination Survey (NHANES) would also be
useful. Lastly, it is also recommended that 'omics' technologies be incorporated into the array of
bioprofiling tools in the ToxCast program. Such methods enable extremely broad coverage of biological
responses to chemical exposure.

-------
EPA Response: The HHRA program agrees and is developing the step-wise approach recommended by
the SAB and BOSC. The HHRA program will begin by evaluating new applications using the approaches
described above under the Recommendation above, "Build confidence in new approaches for assessing
safety." As described above, through the tailored case studies and projects in the Translation topic, the
CSS program will continue to work on integrating a broader spectrum of data into its risk-based
evaluations. For example, in the CSS Demonstration & Evaluation project 'omics technologies are already
being evaluated for incorporation into the bioprofiling tools in CSS comptox approaches.

CSS will continue to engage its EPA and external partners and stakeholders in the process to explore and
evaluate opportunities to integrate and use data, to build confidence in the approaches being explored,
and to ensure complete transparency in these approaches. For example, EPA and NIEHS have partnered
together to hold a workshop in order to prioritize research areas that will improve our scientific
understanding of how to reduce uncertainty and improve human health risk assessments. EPA has also
engaged the National Research Council (NRC) to provide recommendations on integrating new scientific
approaches and data streams into human-health risk assessment. Specifically, this new committee will
consider the scientific advances that have occurred following the publication of two NRC reports, Toxicity
Testing in the 21st Century: A Vision and a Strategy, and Exposure Science in the 21st Century: A Vision
and a Strategy, and provide recommendations on how best to integrate and use the emerging results in
evaluating chemical risk and identify how traditional human-health risk assessment will need to change
to incorporate the new science.

Recommendation: Continue to emphasize communication, education and outreach. The CSS program
in particular has realized that the tools it is developing are transformative and may be unfamiliar to staff
in program offices and regions that are distant from the research and have long-standing ways of doing
their work. As such, the initiatives dealing with education, training, and outreach are strongly
encouraged and are considered essential for stakeholder understanding and acceptance of new
approaches.

EPA Response: The HHRA program was restructured in 2014 to emphasize stakeholder engagement and
cross-program integration in order to inform program and regional partners as well as external
stakeholders about advancements in the assessment program, and to raise critical scientific issues in the
area of human health risk assessment. For example, scientific workshops are now being conducted
annually to engage stakeholders on critical issues that represent current challenges to assessment
approaches in general or to specific chemical evaluations. Also included in HHRA outreach is its risk
assessment training experience (RATE) program comprised of over 30 specific modules covering hazard
identification, exposure assessment, dose-response assessment, benchmark dose modeling, PBPK
modeling, mixtures guidance and cumulative risk assessment. The RATE program is an area where
advancements in risk assessment can be shared and discussed with our end-users to further their
understanding of these advancements.

Every project in the CSS research topic areas has a significant education, outreach, and engagement
component first in building and executing the case studies, and also more broadly through targeted
webinars and panel listening and discussion sessions set up monthly with program and regional partners.
In a continued effort to integrate across other research programs, these sessions are intended to inform
within and across ORD as well. This engagement culminates in a face to face 'Connectome' meeting, held
approximately annually across CSS and with partners designed to allow direct interaction among
partners and CSS project investigators. In addition, the projects within Translation and Knowledge
Delivery are specifically intended to carve out components of the science that are ready for application,

24

-------
and to engage stakeholders in education and outreach opportunities around these components. CSS has
sponsored or co-sponsored a number of workshops in collaboration with other federal partners (such as
FDA and NIEHS) aimed at this outreach. For example, 'genius bars' were set up at these workshops to
allow for hands-on database and tools training for partners and stakeholders. These activities are carried
out throughout the year in a variety of venues to allow for maximum interaction.

4.3.6. Charge question 6c. Are there other areas of fit-for-purpose characterizations (e.g., risk
prioritization, risk screening, risk assessment) that are ripe for such collaboration/integration?

Recommendation: Develop novel approaches to address cumulative risk in a holistic manner. Some
of the new tools for both toxicity testing and exposure assessment are expected to lead to novel
approaches for assessing cumulative risk that were not possible before. In addition, there is strong
support for holistic approaches which consider both chemical and non-chemical stressors. In the future,
cumulative risk could become the next cross-cutting roadmap area.

EPA Response: As in the July 2014 draft, the revised HHRA StRAP has a project area devoted to
advancing cumulative risk assessment methods. The project has been placed under Topic 3 that is
targeting community and place-based issues, consistent with the problem formulation for cumulative
risk assessment recommended by the NRC (2008) report on Science and Decisions. Specific tasks have
been carefully selected to take a step-wise approach to evaluating how to incorporate additional
stressors in key areas where HHRA has already advanced the science including: integration of human and
ecological effects based on mode of action and characterizing considerations of susceptibility in
cumulative risk assessment (CRA). Additional research will be devoted to apportioning multimedia
exposure and risk across receptors and developing an understanding of how to consider epigenetic
markers, which reflect both chemical and non-chemical stressors, in the context of cumulative risk
assessment. The HHRA program is co-sponsoring a scientific workshop on the use of epigenetics as
biomarkers in FY16 and lessons learned from that workshop will inform the on-going research activities
in the HHRA program for the FY16-19 period. Scientists in HHRA will also continue to support the
Agency's Risk Assessment Forum activities on the EPA mixtures guidance and its CRA 2003 framework.
As mentioned above in response to recommendation 7, HHRA scientists will also continue to provide
training and advice on how to construct and implement both the mixtures guidance and the CRA
framework.

CSS will also invest in and explore the application of novel data streams minimally for evaluation of
cumulative risk of exposures to co-occurring chemicals. These will be developed as part of the partner-
driven case studies related to endocrine disruptors. Further collaboration between CSS and HHRA on
developing and incorporation AOPs in human health risk assessment is anticipated to aid in
understanding the potential effect of chemical interaction in the biological response to environmental
chemicals.

Finally, future work in the area of Health Impact Assessment by the SHC program may highlight possible
areas for use of novel tools and approaches in understanding the impact of cumulative exposures.

-------
4.4 Sustainable and Healthy Communities

Executive Summary: ORD's Sustainable and Healthy Communities (SHC) StRAP provides a thoughtful
applied roadmap for advancing high-priority agency research. SHC focuses on conducting basic research
on community-oriented environmental and health issues. It also focuses on providing information to
communities and the agency's regional offices concerning the development and application of
sustainable practices relating to environment, society and economy.

Recommendations: Listed below are recommendations for addressing the complex challenges
associated with developing sound approaches to SHC research. In identifying these challenges, the SAB
and BOSC recognize that the SHC StRAP is intended to serve as a high-level guidance document, not as a
detailed accounting of its research agenda. The SAB and BOSC strongly recommend that ORD revise the
StRAP to acknowledge these challenges both up-front and throughout the document.

4.4.1. Charge Question 2a. How well will the research directions in each Early Draft StRAP (2016-2019)
support EPA in achieving the relevant Agency objectives and cross-cutting strategies, as described in
the EPA Strategic Plan (2014 -2018)?

Key Recommendation: Address the challenges of developing good community-based decision support
tools and maintaining and sustaining partnerships. The SHC StRAP does a good job of addressing the
need for a multi-criteria approach to decision-making, and supporting the establishment of partnerships
and community decision-making. However, it may underestimate the difficulty of developing good
decision support tools general enough for broad application and flexible enough to address decision
making in a regional context.

A key element of the SHC StRAP is to establish a research agenda focused on collecting information, and
developing methods and tools for supporting agency and community decision-making. The desire for
improved decision-making runs the gamut from agency to community concerns (e.g., air, water,
contaminated sites) and involves differing temporal and spatial scales. The SAB and BOSC strongly
endorse ORD's proposed work in this area. Advancements in how to improve the quality and
defensibility of agency decision-making at the community level is needed to help the EPA achieve its
sustainability goals operating as a high-performing organization responding to citizens' needs and
expectations. However, the StRAP does not adequately address many of the challenges likely to be
encountered in a research agenda focused on decision-making, especially at the EPA Region and
community levels.

EPA Response: We appreciate the strong endorsement for our research on improved decision making on
EPA and community concerns to better achieve sustainability goals. We agree that we face considerable
challenges implementing this research agenda and that these should be acknowledged more specifically
in the StRAP.

The issue of developing support tools general enough for broad application and flexible enough to
address regional decision making is a very important priority for SHC. We are looking into whether
community-level decisions can, in some way, be categorized so that the knowledge can be more readily
transferred to a group of communities with similar characteristics, recognizing the importance of the
decision context. We are working with EPA partner offices to develop representative typologies of US
communities based on social, environmental and social characteristics such as population density, GDP

-------
per capita, open and built space. We anticipate that an improved understanding of community
typologies can help EPA to develop better and more customizable decision tools in the future.

We are aware of the need to initiate and maintain relationships with our partners (EPA's program
offices and Regions) as well as our stakeholders in communities. The Administrator's cross-Agency Goal
to "Make a Visible Difference in Communities" will facilitate these relationships through an inter-Agency
workgroup that includes all of the Regions and the Program Offices.

We agree with the SAB and the BOSC that SHC's research may not address all of the decisions likely to be
made at the community level. However, we intend to learn from our successes as well as failures and
believe that our current approach of studying a number of representative community-based decisions,
evaluating these decisions and their contexts, and building tools around these decisions is an appropriate
way to begin, given available resources. As recommended in the SAB and BOSC report, we have revised
the StRAP to better emphasize the challenges ahead in implementing this research agenda. These points
are discussed further below in response to more specific recommendations.

• Recognition that the development and application of effective decision-making approaches
(and specific decision-making tools) will depend upon building and—importantly-
maintaining effective partnerships and engagements with communities. Effective engagement
with communities is especially important when considering the unique needs of marginalized
stakeholders and communities. ORD's crosscutting research devoted to environmental justice
needs to be more tightly integrated into the SHC StRAP. The SHC StRAP also needs to clearly
distinguish between stakeholders and partners and clearly articulate the relationships between
the two.

EPA Response: We agree on the importance of partnerships and engagement with communities and do
not underestimate the amount of work involved in doing so. A number of EPA's STAR Centers of
Excellence for Children's Environmental Health are successful because they have long-standing
relationships with communities in which they are engaged in community-based participatory research;
SHC has worked to maintain many of these cohorts and long-standing relationships.

An additional example is SHC's phased approach to developing a research partnership with the National
Institute of Minority Health and Health Disparities (NIMHD) to address environmental health disparities.
SHC started with a 2 year pilot to link to established research centers that, in turn, have long-term
relationships with communities, Historically Black Colleges and Universities, Hispanic serving institutions,
and Tribal colleges. This was a 2-year pilot program that used inter-agency agreements to jointly fund
10 established centers. The success of this approach led to the issuing of a 2014 Request for Applications
(RFAs) for Centers of Excellence on Environmental Health Disparities, jointly issued with NIMHD and
NIEHS. These Centers are designed to address scientific issues related to at-risk communities with 5 year
grants. The RFA, along with one addressing Healthy Schools, includes strong criteria for incorporating
community engaged research strategies.

We are working to ensure integration between the SHC program and the EJ Roadmap. SHC also now has
the lead for completing the Environmental Justice (EJ) roadmap. This will facilitate enhanced integration
between the two efforts through the implementation of the research program. We note that the SHC
research portfolio contains a significant body of research oriented directly toward at-risk groups
including children, tribes, and overburdened communities, and includes cumulative assessment. SHC
research comprises much of ORD's EJ Roadmap this will be conveyed more clearly in the revised StRAP.

-------
SHC is using the ORD convention for defining partners and stakeholders, and in updating our StRAP, we
refer to our EPA colleagues in the program offices and regions as our partners. We also use the term
partners to include other researchers we collaborate with (e.g., from other agencies). The term
stakeholders generally refers to those outside of EPA who have an interest in our research and may use it
for a variety of applications.

• Acknowledgement of the complex, multifaceted nature of decision-making processes.

Effective decision-making processes are highly sensitive to context, with different decision
problems (and decision opportunities) requiring varying levels of technical sophistication.
Although ORD currently has ongoing research to identify a typology of communities and the
contextual basis for decision-making, the SAB and BOSC question the ability of ORD research, as
currently articulated and configured in the StRAP, to provide effective decision support across all
contexts.

EPA Response: We certainly understand your concerns about whether SHC is currently configured to
provide effective research across the complex, multi-faceted nature of decision-making. We have revised
the SHC StRAP to more explicitly focus on decision support, identifying critical science needs and how
that science can be used.

We are relying on our typology study to categorize communities and decision contexts to help us design
our research. We also expect to gain insights in a wide variety of contexts by working with communities
through a number of ORD and Agency venues, including SHC's Regional Sustainability Environmental
Science (RESES) program, that provides funds to EPA's Regions to build direct collaborations with SHC;
SHC's Technical Support centers that provide direct assistance to Regions working in communities with
contaminated sites; place-based research on ecosystems goods and services and the application of
health impact assessments linking ecoservices and human health in communities; and collaborations
identified through the cross-Agency goal of making a visible difference in communities and other
evolving opportunities such as those presented by the Rockefeller 100 cities initiative. EPA is a technical
partner to the Rockefeller 100 Resilient Cities Challenge that shares sustainability science to help
communities build resilience and adapt to climate change. EPA is able to share tools and best practices
with communities while learning about new tools and research from Rockefeller's other technical
partners, including the World Bank, Sandia National Labs and the Nature Conservancy.

• Acknowledgement that setting and implementing a research agenda focused on decision-
making will not be easy. In addition to the challenges outlined above, the development of
decision-making approaches and tools will require careful evaluation. Evaluation will require
that ORD establish systems for communication across different communities to maximize
opportunities for cross-community and cross-approach learning. Community needs and
priorities will change over time, so decision tools require flexibility and adaptability.

• Key Challenge: Recognizing that a research agenda focused on sustainability and decision-
making may be a "tough-sell" in some communities. Communities dealing with certain classes
of problems (e.g., contaminated sites) may simply want straight answers and not better
processes. They may want to identify how to fix problems and not just think about or analyze
problems. Thus, the SHC program should include research on more fundamental concerns
around environmental health and "sustainability" as inputs to decision-making processes.
Addressing this challenge will require bilateral communication channels across many very

-------
different communities, as well as with researchers working in different aspects of environmental
systems.

EPA Response: We agree with the SAB and the BOSC regarding how "tough" it might be to implement
decision support tools in communities that are overwhelmed with a legacy of pollution. We also
recognize that in communities with contaminated sites (e.g., Superfund), the decision to remediate a
legacy of contamination is likely the overriding decision rendering a decision-support tool unnecessary.
SHC continues its commitment to fundamental environmental health, ecological and sustainability
research that remain a very substantial part of the program. This information, together with research
from other EPA and agencies' programs are indeed what we draw from as inputs to decision-making.

SHC employs its Sustainability Assessment and Management Cycle (SAM) both as a framework for
building an actual toolbox and as a heuristic for identifying critical data, data bases, models, and
processes that will be necessary to develop flexible decision support tools. In SHC's schematic SAM cycle,
the decision about whether a full-blown sustainability assessment /structured decision process is needed
comes at the start. Where the issue and objectives are clearly demarcated as at many contaminated
sites, the full process may not be necessary. This dovetails with the recognition captured in the NRC
Green Book that sustainability will require a suite of tools, from simple to complex. Processes to address
issues in contaminated communities have been developed by the Agency and include community
engagement as well as application of technology for remediation. SHC provides research to support
community engagement and technology. SHC's remediation technology research and development
includes assistance through SHC's Technical Support Centers, and fundamental research on topics such
as permeable reactive barriers, pore water, ground water movement and fate and transport of
contaminants, and site characterization. We do believe that an important aspect of using decision-
support tools, even in those communities where the immediate decision is obvious, is the transparency of
the process leading to subsequent decisions.

Key Recommendation: Communicate that environmental health is a critical driver of overall human
well-being and sustainability. The SHC focus on "sustainability" should include an increased focus on
environmental health. Some members found that the current emphasis on community sustainability did
not adequately account for the environmental risks and ecological concerns identified in the agency
Strategic Plan. In other terms, the planned SHC research on sustainability indicators and sustainability
objectives related to decision making focus too much on social and economic metrics that may be at
odds with ecological constraints and may result in the EPA (and the communities the agency serves)
inadvertently failing to account for the critical importance of environmental health as a driver of overall
human well-being.

EPA Response: We agree that SHC's emphasis on sustainability should have more of an environmental
health focus and this is reflected in revisions to the StRAP. Additional resources received in our FY 15
appropriation are going towards understanding how ecosystem goods and services affect environmental
health. For example, SHC research includes the application of ecosystem service production functions to
understanding the impacts of climate change and other stressors on ecosystem health and the provision
of ecosystem services. SHC's EnviroAtlas provides a mapping tool and analytics for visualizing ecosystem
health using nationally consistent measures of ecosystem services, scalable down to the community
level. Also, SHC includes research on the impacts of nitrogen pollution on environmental receptors and
ecosystem services. We are increasing attention to the development of indicators of human health, well-
being, and environment quality, along with indices that incorporate them all. Our prevailing view is that
the economic and social elements of sustainability depend on a healthy, functioning environment and

-------
that environmental quality is the principal driver of human well-being. Part ofSHC's research strategy is
to conduct research to evaluate this view.

4.4.2. Charge 2b. What are the SAB/BOSC perspectives on the proposed research directions in the
StRAP providing research to address environmental issues of 2020 and beyond?

Recommendation: Plan for robust and flexible decision tools. The state-of-the-art in the different
disciplines that comprise the science of decision-making continue to evolve, and the data and
information used to inform decision making is always in flux. Therefore, the approaches and tools being
developed by ORD should be sufficiently robust and flexible to address both present and emerging
environmental issues and take advantage of new information. From the information provided, it was
unclear to reviewers whether the planned SHC research considered expected changes such as aging
populations, immigration, flooding of coastal communities with sea-level rise, or the changes occurring
in legacy cities (e.g., Detroit) or whether planned tools and decision support systems were adaptive
enough to respond effectively and efficiently to changing conditions.

Recommendation: Assess emerging environmental issues with special attention to changing
demographics, cumulative impacts, and energy-related environmental impacts. The SAB and BOSC
recommend that ORD devote some of its resources to tracking and assessing changing environmental
conditions and concerns at the community level, which will be sensitive to changing demographics.
These kinds of changes, which may initially appear relatively unimportant or insignificant, have the
potential to develop into those that pose the greatest challenges to the EPA in 2020 and beyond. In
addition, the committees recommend that ORD closely monitor issues for which cumulative effects
might likely be problematic. For example, ORD should monitor concerns that arise on the chain of
energy development, from site-specific environmental concerns to energy transport (e.g., interstate
and—in many cases—international risks from pipelines, rail, and ship transport) to energy use (e.g.,
carbon pollution and air quality concerns), which, clearly, will involve integration and cooperation with
ACE and other programs.

EPA Response: \Ne do recognize in our StRAP that community demographics are changing (e.g.,
migration to urban areas; environmental literacy, economic status, and value-based socio-demographic
drivers), that we must consider and understand cumulative impacts, not just from chemicals but from
non-chemical stressors as well, and that sustainability has an energy-environmental component. We
acknowledge and have planned for research to better assess the environmental concerns associated with
energy transport and have increased our resources in understanding inland oil and fuel spills.
Additionally, ORD has begun to use ideation techniques to anticipate the issues of2020 and beyond by
engaging our recent post-doctoral scientists to identify those issues of the future in the context of
sustainability.

SHC has included issues associated with changing conditions due to climate change and other dynamic
stressors throughout its program. These include indicators and indices of community resilience,
community and national scale coverages indicating climate change vulnerability in the EnviroAtlas, and
spatial and temporal considerations for contaminated sites including changes in hydrodynamics that
might occur under drought or other conditions associated with climate change. The recommendation to
track and assess changing environmental conditions and concerns at the community level is very
important to ensuring the relevance of our program and we will take that under consideration.

30

-------
4.4.3. Charge Question 2c. Did the presentations and plans indicate that ORD is designing for
integration, where appropriate, on topics that are relevant to other research programs?

By and large, the answer is yes. Integration is very evident at the conceptual level of the StRAP. And,
where specific discussions of integration across the other five national research programs—as well as
the different crosscutting roadmaps—were absent, reviewers found it easy to see the connections
between those activities and planned SHC research activities.

Recommendation: Make integration of research planning a priority for management and capacity
building. Although integration is evident at the conceptual level of the SHC StRAP, it was not clear that
the reality of implementing research described in the EPA StRAPs and roadmaps will achieve needed
integration. Without management attention, the six national program areas could devolve into six
isolated silos. The SAB and BOSC observe that, while specific recognition of collaboration among the six
national program areas was evident in the presentation made by ORD at the SAB-BOSC meeting of July
24-25, 2014, such detailed information was not discussed in the StRAP. The SAB and BOSC recommends
that ORD senior leadership ensure that effective communication and integration across the six national
program areas takes place. Both leadership on, and resources devoted to, building transdisciplinary
capacity within the EPA will be required to effect and maintain integration.

EPA Response: We agree that achieving real integration of research will require effort from all levels of
management and staff and leadership by senior management will be key. ORD's senior leadership is
firmly committed to integrative, transdisciplinary research, and continues to seek opportunities to
promote integrated work. As one example, previous management information systems made it difficult
to enter and track projects involving staff from more than one laboratory. The project management
system for 2016 and beyond is constructed with the expectation of cross-laboratory research. An
emerging senior leadership emphasis on integrating human health and environmental research will
further convey the importance of truly integrative approaches.

Additionally, all of ORD's National Research Programs are keenly aware of the need to integrate and
collaborate. The StRAP's reviewed by the SAB and the BOSC in July, 2014 will be finalized in 2015 and
each StRAP will emphasize integration across the six national programs and, in SHC's case, better
integration within SHC's 11 projects. The four Roadmaps are another technique for integration and
collaboration. Integration also occurs within a research laboratory as a principal investigator working
on, for example, green infrastructure, is likely to conduct work that contributes to both SSWR and SHC.

4.4.4. Charge Question 4a. Does the research program contain the elements necessary to integrate
two critical elements (ecological and human health) of EPA's mission?

Broadly speaking, the answer to this question is, yes. However, greater clarity is required in the StRAP
with respect to the extent to which human health and ecological health are being addressed as parts of
integrated systems or being developed separately from each other following a traditional stressor-
response paradigm.

Recommendation: Address ecological and human health as parts of integrated systems. SHC research
should treat ecological and human health as components of coupled natural-human systems. The SAB
and BOSC identified weaknesses in the StRAP's discussion of systems-based approaches and topics
requiring transdisciplinary research. For example, some of the individual research examples outlined in
the StRAP seemed to focus on individual elements of systems and not the coupling of different human

-------
and natural dimensions within systems. The SAB and BOSC recommend that specific and direct lines of
collaboration be developed between SHC and the Human Health Risk Assessment programs. The SHC
StRAP should include a focus on life-cycle analysis and recognize explicitly that sustainability is largely a
function of the tradeoffs and synergies among environmental, social, and economic priorities.

EPA Response: We agree that human and ecological health are intricately connected. SHC is designed to
address ecological and human health as parts of an integrated system. The revised StRAP acknowledges
this in a number of places. For example, the Program Design section states:

• Improve community and individual well-being by providing tools, approaches, and guidance that
acknowledge the interconnectedness of the human-ecological system, and the importance of a
healthy environment to promote human health, economic resilience and social connectivity.

Several projects directly address this ecological-human health connection:

• Final Ecosystem Goods and Services (FEGS) Classification, Metrics and Production -includes
quantifying the linkages between the supply of ecosystem goods and services to changes in
human health and other measures of human well-being. There is a direct linkage here between
SHC, ACE, and HHRA as it relates to ecosystem effects of nitrogen in the context of the Secondary
National Ambient Air Quality Standards for oxides of nitrogen and sulfur.

• Environmental drivers of community health and well-being - focuses on understanding of the
associations and causal relationships between community health and well-being, ecosystem
goods and services and community environmental (including non-chemical) stressors and
conditions. This work is also coordinated with HHRA as both programs address aspects of
cumulative risk assessment. The research in SHC will help inform the broader methodological
questions about cumulative risk assessment that HHRA advances.

• Interpreting environmental conditions in terms of ecological relevance, public health outcomes,
and well-being endpoints - Utilizes holistic approaches for assessing human health and well-
being in the interpretation of changes in environmental conditions for evaluating the utility of
full suites of indicators in the SHC decision support tools, integrated approaches and ROE. The
indicators and indices developed under the scope of this project must address the issue of
concern and be technically sound, easily understood and accepted by stakeholders. The
construct validity of indicators and indices will be evaluated to make sure that they are robust
and meaningful measures of specific issues of concern (i.e., ecological, environmental, human
health).

• Life Cycle Management of Materials - develops a framework to support decision making within
the nexus of life-cycle assessment (LCA) and sustainable materials management (SMM) by
integrating LCA methods being developed throughout ORD's national research programs with
approaches for the design of sustainable alternatives. The intended outcome of the framework
will be the identification of optimal SMM strategies given the numerous options for impact
reduction within a material life cycle.

-------
4.4.5. Charge Question 4b. Is increased well-being the appropriate outcome to aim for, rather than
amelioration of specific health conditions? If so, does the SAB/BOSC have recommendations for
shaping the Community Public Health research project more toward broader well-being impacts?

The SAB-BOSC answer to the charge question is a definite yes. However, the committee recommends
that the StRAP state more explicitly that there is more to "well-being" than environmental and human
health. For example, well-being can be connected to economic security, as well as various social and
psychological considerations that are likely to be evident at both the community and individual levels.
Moreover, well-being—much like the concept of sustainability—will have an evolving definition based
on the decision context for which it is being considered. Further, the definition of well-being may differ
within and between communities.

Recommendation: Continue and expand research focused on defining and measuring well-being,
building on existing resources on this topic. The SAB and BOSC recommend that ORD continue and
expand its research focused on measures of human well-being and its relationship to environmental
health. This report strongly recommends that ORD review the wealth of existing research, across a
broad range of disciplines, focused on defining and measuring well-being (McGillivray and Clarke, Eds.
2007; McGillivray 2008; Helliwell et al 2013; ecoAmerica 2014). This research can provide a valuable
basis for assessing and determining well-being across different communities. Moreover, under its
research efforts devoted to decision-making, ORD should attempt to identify the varied objectives of
stakeholders that will comprise "well-being" and work toward developing measures and measures that
characterize these objectives to assess well-being in community-specific contexts (Diener 2009, United
Nations 2012).

EPA Response: We agree with your comment that in measuring "well-being" there is more to it than
considering just the environment and human health and will take these recommendations into account
as we continue research in this area. SHC recently released a report on the Human Weil-Being Index
(HWBI: http://www.epa.aov/aed/wellbeing.pdf) that includes these factors: connection to nature,
cultural fulfillment, education, health, leisure time, living standards, safety and security, and social
cohesion. The scale of this HWBI is at the US county level but we recognize that well-being is a function
of the factors noted above operating within a community structure. The development of the HWBI
included a review of other well-being indices (Smith, et al. Ecological Indicators 28 (2013) 79-90.
Continued development of this index includes adaptation of HWBI approaches to selected vulnerable
populations with emphasis on health.

The SHC team is in full agreement on the importance of looking across research findings from a range of
disciplines as we work to improve indicators of health and well-being. Engaging stakeholders is needed
to achieve our goal of developing technically sound, easily understood and useful measures of well-
being.

4.4.6. Charge Question 4c. SHC is interested in thoughts and suggestions from the SAB/BOSC on ways
to conduct research on the science of sustainability.

The SHC's emphasis on research in decision-making processes and the foundational data to support
those processes is precisely where ORD should be focusing its efforts. However, to inform decisions
regarding ways to conduct research on the science of sustainability, the SAB and BOSC provide the
recommendations below.

-------
Recommendation: Conduct a thorough review of ongoing research in the science of sustainability.

There are an extensive number of publications focused on research on the science of sustainability that
can inform ORD's thinking about ways to conduct its own research. Sustainability science is increasingly
well-defined and research agendas have been published; presumably these are known to ORD and can
provide additional perspective (see, for example, NRC 1999a; NRC 1999b; Kates et al. 2001; Clark and
Dickson 2003; Swart et al. 2004; Kates, 2011; NRC 2012a; NRC 2012b).

EPA Response: We agree that understanding the current scientific literature on the science of
sustainability is crucial to the development of the SHC StRAP. We appreciate the references provided in
the recommendation, and we are continuing to review the literature as we plan our research. The work
in the first research topic in the StRAP is designed to build on the existing body of knowledge on
sustainability. While straightforward and intuitive conceptually, sustainability in practice is much harder
to understand. Research under SHC aims to advance the science of sustainability and disseminate tools
and methods for integrating sustainability approaches into EPA and community decision making.
Decisions that promote sustainable outcomes and minimize unintended consequences require access to
relevant information, structured analytic approaches, tools for assessing and optimizing outcomes,
examining trade-offs, and tracking progress. Also, SHC is developing science pertinent to sustainability
with ORD's other research programs as described in the cross-cutting roadmaps and with additional
projects, e.g., green infrastructure, NetZero, and sustainable water projects with SSWR; environmental
public health projects with ACE; cumulative assessment approaches with HHRA.

Recommendation: Look outside the agency for ideas about research methods. The generation of ideas
from outside ORD would add significant value for the agency. Calls for research proposals from scientists
outside the EPA through EPA's Science to Achieve Results (STAR) program, as well as through
partnerships with other government agencies and foundations (e.g., the National Science Foundation),
would almost certainly lead to the development of new research approaches and practices that would
be applicable to the SHC's mission.

EPA Response: We agree and have continued to reach out to outside groups. ORD, along with the Mac
Arthur Foundation, has commissioned a study by the National Academy of Sciences — Pathways to Urban
Sustainability: Challenges and Opportunities. We are in discussions with the European Environmental
Agency to learn how they have incorporated sustainability into their research agenda and the European
version of the ROE. Further, SHC will consider an RFA through our STAR Grants program soliciting
proposals that address the science and application of sustainability to communities. As mentioned
previously, EPA's participation in the Rockefeller 100 Resilient Cities Challenge is an excellent opportunity
to share research methods and tools with the other technical partners in this effort. The collaboration
offers opportunities to learn from the other organizations as well as the community partners who are
applying the different methods and tools.

Recommendation: Build transdisciplinary and applied social science research capacity within ORD.

There is a need to build capacity in the social and behavioral sciences and to effectively integrate these
sciences with ORD's traditional strengths in the natural sciences. The SAB and BOSC recommend that
ORD, when hiring opportunities occur, explore adding personnel with training in transdisciplinary
approaches to addressing issues and opportunities at the environment-human nexus, in addition to
additional personnel with expertise and skills in the applied social and behavioral sciences. ORD should
also train existing EPA staff in these areas in collaboration with appropriate academic institutions or
professional associations.

-------
EPA Response: We agree that ORD needs to build capacity in the social sciences and to integrate the
social sciences with environmental science. We do employ some social scientists, especially in the areas
of environmental economics and decision support, and some of our recent post-doc employees have
good backgrounds in the social sciences. Social science is a field that we will recruit for as hiring
opportunities become available. We are persuaded that achieving a sustainable future is both a
scientific endeavor as well as a need to understand the behavioral motives of people, both individually
and collectively. (See also the responses to the overarching recommendations).

Recommendation: Clarify the "three-legged stool" approach to sustainability. "Sustainability" in this
StRAP relies on the concept of the "three-legged stool" (i.e., environment, society, economy), although
this is not the definition offered by any other StRAP or the Strategic Plan. The more recent sustainability
literature critiques the pillars (which lend themselves to silos) in favor of more holistic representations in
recognition of the hard constraints imposed by environmental limitations (Dawe and Ryan 2003; Raskin
et al. 2010; Diaz et al. 2011; Costanza et al. 2012). This latter view of sustainability seemed to be the
focus of the information presented in the SHC overview slides at the SAB-BOSC meeting, but not in the
StRAP itself.

Systems-based approaches, emphasized as necessary across all the StRAPs, will by definition require
integrated approaches to achieve agency objectives. ORD should revise the StRAP to acknowledge the
reliance of societal and economic metrics on environmental constraints (Rockstrom et al. 2009,

Baronsky et al. 2012; Costanza et al. 2012), particularly as communities may not be aware of the
tradeoffs and synergies among economic, societal, and environmental goals.

EPA Response: We agree, and SHC has revised the StRAP to provide its perspective on sustainability that
acknowledges the reliance of societal and economic wellbeing on environmental constraints. The revised
StRAP states that "The nested relationships of a resilient economy existing within a healthy society
dependent on an intact, functional environment illustrates the holistic definition of sustainability that
recognizes the hard constraints imposed by environmental limitations. SHC's goal is to expand
community stakeholders' capabilities to consider impacts of decision alternatives to optimize these
outcomes through flexible sustainability assessment approaches that consider the dynamic nature of the
integrated environmental/social/ economic system."

-------
4.5. Homeland Security

Executive Summary: The Homeland Security research program (HSRP) has a primary mandate of
performing research related to the EPA goals of protecting water supplies and providing post-disaster
clean-up in both indoor and outdoor environments. ORD has recently broadened the mission in the draft
StRAP to include both terrorist and natural disaster threats to water supplies and post-disaster clean-up.

4.5.1. Charge Question 2a. How well will the research directions in each Early Draft StRAP (2016-2019)
support EPA in achieving the relevant Agency objectives and cross-cutting strategies, as described in
the EPA Strategic Plan (2014 -2018)?

The HSRP's historical focus and new "all threats" mission supports many agency objectives including,
protecting America's waters, preventing pollution, clean-up from long term pollution releases as well as
immediate disasters, enhancing community resilience, and reducing community vulnerabilities.

Key Recommendation: Explore a systems approach and develop partnerships to stretch limited
resources to help meet the HSRP historical focus and new "all threats" mission. The main question of
concern is how the HSRP can widen its focus to include all threats, given that HSRP resources are not
being expanded. While the review of the StRAP does not provide an opportunity to provide detailed
advice, the SAB and BOSC suggest that the EPA utilize systems approaches to meeting the agency's
overall mandates, exploiting commonalities between threats, cleanup-measures etc. Section 4.5.5.
addresses the systems approach topic further. In addition, the HSRP should seek creative partnerships
with other agencies and entities conducting research on topics relevant to the EPA HSRP mission.

EPA Response: The revised Homeland Security StRAP for FY16-19 outlines how the program uses a
systems approach to EPA's emergency response as the basis of its program design. HSRP will continue
and expand its work with other agencies to plan research and develop useful products. For example,
HSRP participates in a memorandum of understanding, and associated steering workgroups, with the
Departments of Homeland Security and Defense to improve coordination across the organizations'
homeland security research programs to foster collaborative efforts that leverage funding and skills.
Partnerships with the U.S. Department of Agriculture's Research, Education and Economics Agency are
also being explored to increase EPA's capabilities to respond to food and agriculture disease outbreaks.

Additionally, HSRP is coordinating work on community resilience across the EPA research programs.
Community resilience can be affected by a number of factors including the changing climate, decreasing
ecosystem goods and services, and disasters. HSRP, in consultation with the other research programs, is
developing community environmental resilience indicators to capture how disasters can impact
environmental and ecological services. These indicators will allow communities to understand how
disasters may affect these services and how they can make decisions that mitigate the impacts of
disasters and facilitate better adaptation post-disaster.

-------
4.5.2. Charge Question 2b. What are the SAB/BOSC perspectives on the proposed research directions
in the StRAP providing research to address environmental issues of 2020 and beyond?

Overall, the general strategy of providing tools, technology, and data for threat prevention,
decontamination or cleanup, is relevant to threats likely to be faced in 2020 and beyond. Some of the
threats to water supplies and cleanup requirements to be faced are potentially predictable on the basis
of today's knowledge as infrastructure ages and as ad hoc containment systems fail, although
assembling and maintaining databases and providing new tools to identify the most vulnerable aging
infrastructure is a daunting task.

On the other hand, there are inherent uncertainties in other future threats. For example, identifying the
locations at primary risk of future flooding due to global warming and the resulting flood-related
chemical and biological contaminations and cleanup needs are dependent upon uncertain speeds of
ocean rise. Specific terroristic threats are also uncertain. Based on today's knowledge, it is difficult to
know whether future failures in cyber security could allow terrorists at a distance to perpetrate major
threats to water supplies through contamination releases from highly automated modern factories or
storage locations. More generally all terrorist threats depend upon the unknown levels of specific
expertise and access, technological sophistication, and the motivations of the future actors in future
events.

Recommendation: Plan for tools relevant to multiple related hazards and invest in innovation
programs. The HSRP tends to be driven by many short-term immediate demands in highly applied
research topics, and indeed flexibility is very important for the response to disaster needs. It is,
however, also very important to make efforts to build tools that will be relevant to multiple related
hazards when responding to specific disasters or prevention requirements.

Because the HSRP is producing outputs on a continuous basis and getting continuous input, the HSRP
research products will be coming out in a phased manner, so these needs will be better identifiable and
addressable as 2020 approaches.

The SAB and BOSC recommend that the HSRP participate in ORD's innovation programs to allow
interesting potentially high-reward research ideas and projects to be considered.

EPA Response: We agree that, whenever possible, our scientific products should be applicable to
multiple environmental challenges. Within HSRP's StRAP, tools are/will be developed that are relevant
to multiple hazards. Some specific examples include a real time water systems modelling tool for
response to contamination of any type, a water systems modelling framework that assesses the impact
of operational and design decisions on overall system resilience to all hazards, and community
environmental resilience indicators focused on understanding the impacts of disasters on environmental
and ecological services.

HSRP strives to foster innovation to our research in several ways. We recruit permanent staff and term
research fellows who are leaders at the cutting edge of their disciplines, bringing innovative thinking to
our core research program. As recommended above, we encourage our staff to participate in ORD's
formal innovative research programs that often provide seed funding for potentially high risk/high
reward ideas; and, as funds allow, HSRP provides seed funding for internal, inventive ideas that help us
fulfill our mission. HSRP is coordinating with ORD's Innovation Team on new sensor technologies to be
used for measurements in water.

-------
4.5.3. Charge Question 2c. Did the presentations and plans indicate that ORD is designing for
integration, where appropriate, on topics that are relevant to other research programs?

Key Recommendation: Maintain the current high level of integration with ORD research programs and
terrorism-related research of other agencies and build partnerships with non-terrorism, disaster-
related agencies as well as EPA regional offices. The HSRP is best designed to deal with a higher level of
uncertainty about its future research directions than most other ORD programs. Nevertheless, there are
many tasks of the HSRP that are clearly joint projects with other portions of ORD and other agencies.
These include but are not limited to: further developing knowledge about the transport and long term
fate of specific contaminants; contaminant detection; exposure assessment; and risk assessment.

Overall the draft StRAP describes collaboration within ORD very well. It is clear also that the HSRP is very
well integrated with the terrorism-related areas of other agencies. The HSRP is probably less well
integrated with the non-terrorism, disaster-related agencies. Such integration will need to be deepened
as HSRP fully embraces its "all threats" mandate. The SAB and BOSC specifically encourage the HSRP to
work through the EPA regional offices in an effort to expand its understanding of potential threats to be
faced as well as to reach all relevant stakeholders.

EPA Response: We agree with the SAB/BOSC recommendation that developing additional partnerships
with other disaster-related federal agencies and EPA's regional offices is important to the future of the
HSRP. The HSRP plans to increase our engagement with portions of federal agencies that address all
hazards and we plan to participate in interagency workgroups that address all disasters. For example,
the HSRP will engage more broadly within the DHS including the Resilient Systems Division and at the
Federal Emergency Management Agency. We also plan to continue and grow our involvement in the
Subcommittee on Disaster Reduction (SDR) under the U.S. National Science and Technology Council's
Committee on Environment, Natural Resources and Sustainability. Under SDR, HSRP recently began
active participation on the National Preparedness Science and Technology Task Force (NPST). This task
force aims to integrate S&T related to all hazards national preparedness so HSRP's participation will
allow information sharing and may potentially result in collaborative research efforts with other disaster-
related agencies. Finally, the HSRP will work with the Regions to expand the utility of our tools and other
research from Chemical, Biological, Radiological and Nuclear to all threats. One example of this
expansion is the involvement of EPA Regional response personnel in an upcoming field-scale assessment
of drinking water infrastructure decontamination methods where the contamination may be due to any
one of many disasters.

4.5.4. Charge Question 7a. What advice (e.g., strategic, tactical, structural) can the SAB/BOSC give to
further guide the program toward this broader role?

Key Recommendation: Plan to integrate terrorism-related and non-terrorism disaster response and
prevention research. An overriding issue is how the HSRP can broaden its mission without weakening its
role in security research, especially given budgetary constraints. Whenever possible the HSRP should
aim to exploit the tools, lessons, and science developed for terrorism-related research to all hazards
rather than starting from scratch if the problems addressed are similar enough. Employing systems
thinking will also help identify specific research projects which address multiple hazards. The program
should avoid developing two silos of programs, one related to terrorism response and prevention and
the other related to non-terrorism disaster response and prevention.

-------
The HSRP should continue to develop strategic and synergistic partnerships with other agencies and
stakeholders. The HSRP already is well integrated with terrorism-related agencies and research
programs. Generalizing these sorts of synergies and integrations to the all-threats component of the
program will be extremely helpful in meeting the expansion of the HSRP mission.

EPA Response: The SAB/BOSC recommends that HSRP integrate its terrorism and non-terrorism disaster
and prevention research into its program and expand tools relevant to multiple hazards. In response, the
HSRP will work with its traditional terrorism-focused Agency partners and Agency partners attentive to
non-terrorism related disasters to expand our tools and other research to address multiple hazards. We
will work closely with the ORD research programs to ensure we complement efforts. This expansion will
also be done at the community level through ORD's involvement in the Rockefeller Foundation's 100
Resilient Cities effort (http://www. 1 OOresilientcities.org/#/- /). Specifically, the program will provide its
tools for cities to use in developing their resilience strategies.

4.5.5. Charge Question 7b. How could the research program better incorporate this systems thinking
and engage its partners in this systems thinking from a strategic and tactical standpoint?

Recommendation: Clarify in the StRAP what is meant by the HSRP systems approach and how it will
be used in research planning. The HSRP StRAP should include a clearer explanation of its systems
approach, clarifying which models are being applied, the level of complexity of the models, and
describing how they will be implemented. It is important to clarify how much investment is being made
in formalizing the systems approaches to be used, e.g., by use of specific systems engineering software
or specific tools for graphical representation of the life cycle of projects. More generally, the StRAP
should clarify how a systems approach is to be differentiated from approaches that do not take a
systems approach in actual practical application and how a systems approach is to be used in the
prioritization of HSRD research.

Expanding on comments above, the HSRP should inform multiple federal partners about this research
(e.g., Department of Health and Human Services, Federal Emergency Management Agency), as well as
the EPA regional offices, since having such partners informed and involved will help HSRP achieve goals
related to the systems research being planned. Clear explanation of how ORD plans to use system
approaches in planning and prioritization will help partners understand ORD research and how to
complement it.

The SAB and BOSC note that HSRP is in a position to be a lead federal government research program
addressing environmental fate and transport issues that are currently associated with Homeland
Security threat agents but involve environmental pollutants and particles. Outside of DOE, there may
not be another focused federal agency research effort on this topic. ORD should continue to build a
leadership role in this valued area.

EPA Response: The HSRP StRAP will explain that our "systems approach" is an articulation of the
complex process to prepare for and respond to disasters acknowledging the interdependent nature of the
decisions that must be made in carrying out this process by EPA, other federal agencies, states, and local
decision makers. The HSRP uses this systems approach to understand the nature of the products needed,
how they might be used, and to prioritize funding. For example, although understanding the impacts of
decontamination decisions on the subsequent waste stream quality and volume was not a top priority for
most of HSRP's partners in the past, HSRP funded the development of a tool that reflected the system-
nature of cleaning up after a disaster enabling decision makers to conduct such assessments. This tool,

-------
developed for a radiological incident, is now frequently used in intergovernmental National Level
Exercises, including those led by EPA, for estimating wastes from many different hazard types (including
natural disasters) and the impact of cleanup approaches on the waste stream.

40

-------
Recommendations for Integration

5.1 Childrens Environmental Health Roadmap

Executive Summary: The Children's Environmental Health (CEH) cross-cutting roadmap preliminary draft
is superbly developed and represents a great start to integrating research on CEH across the six
programs. The EPA's ORD has a unique niche and important leadership role in selecting CEH as a cross-
cutting area.

Research in Children's Environmental Health (CEH) spans and develops links from basic science, models,
and mechanisms to the health and well-being indicators of the next generation and their
neighborhoods. Consequently, it is an excellent choice for crosscutting research and integration across
ORD program areas. The CEH preliminary draft is well developed and provides a sound framework for
integrating research on CEH across the six ORD Program areas. This type of integrative research is not
being done in any other research program or organization.

5.1.1. Problem statement and key research topics

The research directions outlined in the CEH cross-cutting roadmap preliminary draft contribute to a
holistic science base in support of children's environmental health with specific calls for applied research
under the general goals, articulated in EPA's Strategic Plan, of cleaning up communities and advancing
sustainable development, and ensuring safety of chemicals and preventing pollution.

The importance of research that incorporates cumulative impacts of chemical and non-chemical
stressors across life stages is noted and of key importance. The key research topics (four research
priority areas) are well presented in the context of their "drivers". The Introduction effectively
summarizes the recent actions in children's environmental health and explains the current drivers that
define the need and focus of the CEH. The translational framework for CEH (Figure 2 in the roadmap) is
excellent in its clarity and comprehensiveness, and the summary of key governmental and international
actions (Table 1 in the roadmap) is excellent and helpful, however, it is unclear as to why some
international programs were not included (e.g., Canada 2010 National Strategic Framework on
Children's Environmental Health [http://www.hc-

sc.gc.ca/ewhsemt/pubs/contaminants/framework_childrencadre_enfants/index- eng.php#a0; European
Union Helix Project (http://www.proiecthelix.eu/i1.

EPA Response: We have added the Canadian strategic framework. The EU Helix project was already
included. We have updated activities of the President' Task Force on Environmental Health Risks and
Safety Risks to Children.

Key Recommendation: Develop a more comprehensive translation research strategy to enhance the
links from basic and observational science to intervention/implementation science to community
action/policy toward the goal of improving children's health. There seems to be much greater
emphasis on the ends of the translational research spectrum (metrics, tools and policies) and less
emphasis on the "middle" of this spectrum (key perturbations, targets and exposures; individual and
community risks). The emphasis appears to be on both foundational research (animal models, toxicology
studies and tools, observational cohorts) and then, at the other end, community action. The pieces that
appear to be quite limited are: (1) the development of interventions; (2) the science of implementing

-------
proven interventions; (3) the science of assessing the impact of implementing the intervention on
environment and health outcomes; and (4) the science of dissemination.

EPA Response: We have enhanced our discussion of the ORD focus and contributions towards
comprehensive translation including how we work with other EPA Program Offices and Regions to
demonstrate, evaluate and disseminate ORD research products. We also clarify that this research in the
middle is required to identify adverse outcome pathways, associated exposures, and risks. (See revised
key research questions under Research area 2.)

Key Recommendation: Clarify and support research on communities' roles and involvement. In a

community context, there is emphasis on the development of tools that can be used by communities to
assess risk and plan, but it is not clear that there is a plan to support research aimed at understanding
how often the various tools are used, by whom, whether they lead to any change in the community or
action by the community, and whether that change has beneficial effects.

EPA Response: We agree that this is an important area for ORD's SHC research to address. Current SHC
research extends beyond what is specific to CEH and it is not discussed in detail in the roadmap. SHC's
emphasis on structured decision approaches is designed to both identify where it is most appropriate to
use science and science tools to support the decision making process. This includes the identification of
indicators or indices of health that can be used to establish baseline conditions and to evaluate beneficial
change after regulatory or non-regulatory intervention.

Child-specific research in this area is being addressed in the EPA/NIEHS Centers of Excellence for
Children's Environmental Health program. As summarized in the Roadmap appendices, all of the centers
engage in community based participatory research that identifies interactions between pollutant
exposures, features of the built environment, and behaviors that put families at risk. These centers have
been highly successful at identifying hazards and then proposing and evaluating solutions that have been
at the center of environmental public health advisories and policy changes to protect children and
families.

5.1.2. Relevant research in each of ORD's six programs

Key Recommendation: More clearly describe how research themes in the StRAPs will be integrated to
support the issues described in the CEH roadmap. Direct links are made to strategic goals on
communities and chemical safety, but links to other goals are deep into the report and not fully
elaborated. For example, highlighting the role of water and air as pathways for toxicity toward children
as a vulnerable population seems highly relevant (CEH Roadmap, p. 18). Further, the extensive risk
assessment and decision-support tools from HHRA could be better integrated into the CEH Roadmap.
The list of activities in Appendix B is helpful, but attention to these links could be better integrated into
the text.

EPA Response: We have provided additional discussion in the section on Research Alignment and
Coordination to address this recommendation.

We did focus on the EPA strategic goals that explicitly call for research in CEH. We have now added a
discussion describing the relationship of other relevant EPA strategic goals including those focused on
clean air and water.

-------
In describing current research activities, we make direct connections in the research inventory by
identifying the research program that is engaged in the identified research activities. However, because
so much ofORD research is foundational to addressing risk assessment and decision support across
vulnerable groups as well as the more general population, we worked to tightly focus the roadmap on an
inventory of research that is specific to CEH so that CEH specific gaps could be clearly identified.

5.1.3. Important scientific gaps

Recommendation: Recognize and optimize the role of leveraging partnerships and prior investments.

Environmental health research now encompasses human epidemiologic and clinical trial studies that
require very large sample sizes. Specific examples of the kinds of studies that are needed include those
that examine gene-environment interactions, Epigenome-Wide Association Studies (EWAS), and
exposome and mixture studies, along with chronic exposure studies. These require extremely large
sample sizes and longitudinal designs. Maintaining the investments made in successful, established large
studies and enhancing partnerships across studies are ways to make progress with limited resources.
Leveraging U.S. population-based surveys such as the NHANES and National Health Interview Survey
(NHIS) offer additional opportunities for maximizing the impact of ORD funds. In particular, data on
environmental contaminant uptake in children using biomarkers in the age group under six years is
needed to complement the current national CDC biomonitoring program, which does not routinely
include this life stage.

EPA Response: We have updated Table 5 to reflect EPA participation in CEH related interagency
activities. We will continue to aggressively pursue all opportunities for leveraging to maximize public
investment to support CEH and to amplify the impact ofORD research.

Recommendation: Research gaps are noted but more specificity and synthesis is needed. Important
research gaps are identified but in broad, unspecific terms. Examples of specific gaps that are not
highlighted that should be considered include expanding the priority health outcomes to include
pediatric cancer [in addition to birth outcomes, neurodevelopment/neurobehavior, metabolic (obesity),
and asthma/airway function] and elucidating the human microbiome within children's environmental
health research.

EPA Response: The reviewed draft did not adequately capture the gap analysis and prioritization that
was conducted to identify key topics areas for ORD cross-cutting CEH research. We have now thoroughly
documented the gap analysis and synthesis to support the research areas identified as a priority for ORD
leadership.

5.1.4. Implementation

Recommendation: Describe how research priorities will be set. This roadmap comprises ambitious and
broad crosscutting activities. The list of research objectives and future projects is reasonable for
significant progress to be achieved by 2019. However, given uncertainties of resource allocation, the
roadmap would benefit from a description of how prioritization might be achieved.

-------
EPA Response: The cross-cutting research areas and priority research needs in the CEH Roadmap were
identified based on three criteria: (1) Information is required by the Agency to support decisions and
actions; (2) Child-specific information is required to protect vulnerable lifestages and groups; and (3)
EPA/ORD leadership is required to fill the scientific gaps. ORD research in CEH is designed and
implemented through case examples that allow for demonstration and evaluation of research products.
ORD works with EPA program partners and regions through our six National Research Programs: to
identify and prioritize useful case examples; to develop and demonstrate the research products fit-for-
purpose; and to evaluate the value added of ORD information and tools to both inform decisions and to
support measurement of impact resulting from those decisions. As such, prioritization of specific
research activities will be conducted through the ORD research planning process and will be guided by
the StRAPS. This has been clarified in the roadmap section discussing how identified research areas will
be implemented by ORD.

Recommendation: Identify mechanisms for motivating and evaluating the responsiveness of ORD's
research programs to the CEH roadmap. The roadmap lacks a description of the stewardship needed to
foster its implementation and success. Experience suggests that achieving and sustaining the needed
level of commitment often depends upon how research budgets are determined and distributed and on
how well individual scientists' professional goals are supported and their work is recognized and
rewarded in the context of this and the other cross-cutting research enterprises.

EPA Response: These important issues are common to all of the ORD research roadmaps as well as to
the six Strategic Research Action Plans. The transition to an environment where collaborative trans-
disciplinary research is required to support the Agency's strategic goals requires a commensurate shift in
how we manage research activities. ORD is evaluating and updating many of the processes used to plan
budgets, distribute resources and support scientists' professional development to foster successful
implementation of ORD research.

44

-------
5.2 Nitrogen and Co-pollutants Roadmap

Executive Summary: Research on the biogeochemical cycling of nitrogen and co-pollutants spans
multiple environmental media and requires integration of basic science, models and mechanisms across
multiple EPA program areas. Consequently it is an excellent choice for cross-cutting research and
integration across ORD program areas. The Nitrogen and co-pollutants roadmap is well written and well
organized. It is highly responsive to previous SAB input and recommendations (U.S. EPA SAB 2011b).

Charge Question 8a. How effective is each Draft Roadmap in presenting a problem statement,
elucidating key research topics, capturing relevant research in each of the six programs, and
identifying any important scientific gaps?

EPA Response: We appreciate the SAB/BOSC's thorough review of the Nitrogen and Co-pollutant
Research Roadmap. These comments have helped provide clarity to the document, and we found them
quite useful overall. The Executive Summary has been expanded as suggested. The Roadmap has been
revised to address issues of scale (Introduction and Purpose), benefits of nitrogen to society (Science
Challenge 3), linkages to the research programs (Section IV, Science Challenge 3 and Appendix E),
nutrient resource recovery (Science Challenge 4), agricultural issues (Science Challenges 3 and 6),
Chesapeake Bay (Science Challenge 5), cross-Agency collaborations (Executive Summary), and
implementation (Executive Summary and Introduction). Detailed responses are explained below.

5.2.1. Problem statement and key research topics

The six over-arching science questions and challenges in the Nitrogen and co-pollutants roadmap are
well-formulated and provide an effective framework. The overarching goals, outcomes, and output of
the roadmap are explicitly articulated, and well-oriented to informing policy and decisions, however, the
summary in the roadmap is too short.

EPA Response: Thank you. The Executive Summary has been expanded to include a general overview of
the Roadmap framework including the science challenges, sub-outcomes, sub-outputs and research path.

Recommendation: Include a discussion of the scale of important components of environmental
problems associated with nitrogen and co-pollutant discharges. In presenting the science challenges,
the issue of scale is not called out and discussed explicitly. The concept of scale as a key variable and
important challenge should be discussed. The large-scale aspect of important components of the
problem of nitrogen and co-pollutant discharges will require flexibility in responses, and this should be
recognized in the roadmap.

EPA Response: We agree that there are a variety of scales operating in terms of policies, sources,
impacts and decisions. ORD programs related to reactive nitrogen are operating at a variety of scales
from local, to state to regional to national, and this is now clearly stated in the introduction and purpose
sections of the Roadmap. The issue of scale is also addressed by the program offices. For example,
National Ambient Air Quality Standards are by definition national and require both national and case
study level analyses for the five year reviews. Non-attainment areas are local, but still require regional
transport analyses. The ecosystem benefits from air regulations can be seen at regional to local levels,
e.g., whole forests and watersheds. Improvements in one ecosystem also provide benefits to

-------
downstream ecosystems. Hence in the nitrogen cascade, there are multiple scales for air, land and water
policy interventions.

Recommendation: Provide a more balanced discussion of the benefits nitrogen provides and the
impacts of treatment and control of nitrogen and co-pollutants. The roadmap should discuss more in
its Introduction the benefits that nitrogen provides to society, and the cost-benefit analysis involved in
deciding how much control to undertake. Treatment and control of nitrogen and co-pollutants has
associated environmental impacts, e.g., the energy and chemicals required to remove nitrogen from
municipal wastewater. In the interest of providing balanced R&D to inform decisions, benefits of
nitrogen and impacts of control measures should be recognized in the roadmap.

EPA Response: We appreciate the comment and have added addition text under Science Challenge 3.

5.2.2. Relevant research in each of ORD's six programs

The roadmap makes connections to the StRAPs for the six ORD research programs, especially SSWR, but
this could be strengthened in the document. Conceptual mappings to the six ORD program areas (e.g.,
Table 1) should be expanded.

EPA Response: Additional mapping is underway in the development and integration of the four
research program Strategic Research Action Plans (StRAPs) directly related to nitrogen and co-pollutants
(ACE, HHRA, SHC, SSWR). The StRAPs will be completed by October 2015. Currently, we are discussing
the research recommendations specific to each research program and identifying research than can be
added or continued in the FY16-19 StRAPs. As an example, in SSWR, and in SHC's work on integrated
nitrogen management, there are several projects directly responsive to the Nitrogen and Co-Pollutant
Roadmap Science Challenges. We will have expanded the discussion in section IV of the document to
designate how and where additional conceptual mapping will take place.

Recommendation: Strengthen discussion of nutrient resource recovery and related technologies. The

roadmap should make a stronger connection to the priority area in the SSWR StRAP on transforming the
concept of "waste" to "resource" in management of "wastewater," especially the capture of nutrients.
The roadmap is weak in the area of development and demonstration of nutrient resource recovery
technology. More generally, the roadmap can be more explicit about the relationship between
prioritized research needs as presented in the StRAPs and research gaps.

There has been great progress made on the ability to remove nitrogen from municipal wastewater. The
roadmap should note this, and discuss how this technological advancement can be leveraged and
incorporated in ORD programs to help address the challenge of controlling nitrogen and co-pollutants.

EPA Response: We have revised Science Challenge 4 of the Roadmap to address this. EPA is crafting
more explicit statements on the research it will conduct in the area of nutrient resource recovery under
Topic 4 of the SSWR FY16-19 StRAP. Linkages between projects under Topic 4 (Water Systems) and Topic
2 (Nutrients) are being discussed in ORD's planning activities.

Recommendation: Expand discussion of linkages with the SHC and ACE programs. The roadmap
mentions linkages with the SHC research program, but there are more overlaps with SHC that could be
discussed. A number of the science challenges presented in the roadmap involve value judgments, and
relate to ongoing research in the SHC program.

-------
EPA Response: Thank you for your feedback, and we have incorporated additional examples highlighting
research in the SHC research program that is relevant to the Roadmap, including nitrogen research in
decision support, the EnviroAtlas, damage costs, and ecosystem services. We agree that it is important
to capture the linkages to the behavioral and decision sciences research ORD is doing.

There are also linkages with the ACE research program that should be discussed. The roadmap has little
discussion of releases of nitrogen to the atmosphere and atmospheric inputs of nitrogen to land and
water. What research is needed to enable adequate reduction of nitrogen emissions to the atmosphere?

EPA Response: ORD is continuing to improve integration efforts across the six research programs.
Nitrogen emissions to the atmosphere and deposition to water (or water via land) is a critical component
of the Nitrogen Cascade presented in the Roadmap. Several aspects of the ACE StRAP will be noted in
the SSWR StRAP to round out how EPA will address nitrogen emissions, including integrated modeling
and Secondary NAAQS development. Linkages with the ACE program are delineated in Appendix E as
supporting research. New research needs are noted as part of new innovative approaches with specific
reference to unregulated air sources. Identification of further research on unregulated sources has been
identified as a research gap in the Roadmap (see Appendix E).

An example of a nitrogen topic that integrates contributions from ACE, SHC and SSWR Research
Programs is eutrophication stemming from nitrogen loading delivered from the Mississippi River Basin to
the Gulf of Mexico (see figure below and Roadmap Figure 3). In this example, the one-environment
model, nitrogen deposition and model linkages are developed under ACE, nitrogen and phosphorus
loadings to the edge of field, aggregation to the Mississippi River Basin (cross-scale) and response to
climate change are developed under SSWR, and linkages to ecosystem health and services, and broad
nitrogen budget research are developed under SHC tasks.

Also, considering the uncertainty in the recently updated national climate assessment about whether
nitrogen emissions will induce cooling, it would be useful to have more discussion about nitrogen
impacts on climate.

EPA Response: The ACE program has addressed the role of nitrogen in the gas and aerosol species
comprising the short-lived climate forcers. The direct role of nitrogen itself is relatively minor. The

-------
climate impact of nitrogen is addressed in ACE has been added to Science Challenge 3. We feel that an
extensive discussion of nitrogen and climate in the high-level Nitrogen Roadmap is not warranted here.

5.2.3. Important scientific gaps

Key Recommendation: Evaluate effectiveness of approaches for management of nitrogen in
agriculture.

The importance of agriculture and related EPA research needed should be addressed more in the
document. One member of the SAB stated: "It is striking how little the word 'agriculture' is used in this
narrative document given the preponderance of evidence that [agriculture] is a key driver of nutrient
loading." It is recommended that the roadmap should include study of the effectiveness of voluntary
approaches for control of agricultural runoff of nitrogen and co-pollutants from non-point sources, and
should make clear the need for substantial engagement with and leveraging of USDA programs and
resources. In the context of USDA engagement, the USDA Mississippi River Basin Healthy Watersheds
Initiative is a research and demonstration program of national importance in which ORD should be
engaged.

EPA Response: We absolutely agree that agriculture is an important driver of nutrient loading, and are
surprised that it came across as underemphasized. We mention partnerships with USDA and organized a
workshop in June 2014 with EPA, USDA and USGS to talk about reactive nitrogen issues. A recent
publication from SHC connects N release to the environment and impacts on human health and public
welfare; this work indicated that agricultural N contributes substantially to these costs (Sobota et al.
2015 Environmental Research Letters). USDA Under Secretaries and USDA scientific staff were briefed
on this work prior to publication. Several of the research projects in SHC and SSWR rely upon
collaboration with USDA, and also make use of information generated by USGS; and we emphasize this
in the revision under Science Challenges 3 and 6 (page 13 and 15).

The roadmap also makes only brief mention of the Chesapeake Bay program, the national, full-scale
experiment in how to control nutrients on land and discharge to water. There should be more discussion
about ORD engagement with the CBP and the importance with respect to the roadmap.

EPA Response: We agree and have added language under Science Challenge 5.

Recommendation: Provide a more explicit discussion of the process for cross-agency, industrial, and
international cooperation. There is discussion in the roadmap about the importance of research
coordination, and about an envisioned cross-agency team to identify research that informs the
development of effective policies for implementation of an integrated and sustainable reactive nitrogen
and co-pollutant management program, but no specific recommendations are made about the
envisioned team.

EPA Response: The Roadmap team has representatives from OW, OAR, and the Regions and has
proposed to OW and OAR that such a cross-agency team is essential to leverage the variety of statutes,
regulations, and voluntary programs to implement programs that will achieve nutrient loading and
concentration reductions. These implementation policy activities are being picked up by the program
offices and is now noted in the Executive Summary, the plans for those activities are under development,
and will be captured in their planning documents. In addition, EPA and USGS co-chair the National Water
Quality Monitoring Council, which helps facilitate information exchange among key Federal Agencies
(USGS, EPA, NOAA). EPA OW also provides some support for coordination with USGS -NAWQA in work

-------
with OW/OSTproviding some computation tools related to nutrients and co-pollutants that are made
available to States.

In June 2014, an initial Joint USDA-EPA-USGS meeting brought together 79 researchers and managers
from EPA, USDA, USGS, NPS, NOAA, DOE, land grant universities, NGOs and private industry groups. The
group discussed coordination of research about reactive nitrogen and co-pollutants across agencies. In
2015, the group is publishing a report from this meeting, and identifying leads for continuation of this
coordination effort. Next steps include 1) Creation of a Joint Science and Management Action Plan to
support EPA-USDA-USGS N and co-pollutant management; 2) Continue to inventory and identify
connections and common ground for the research across agencies, expanding to involve key regional
entities, state agencies and land grant universities; 3) Annual meetings on nutrient-related topics of
complementary interest (water quality monitoring, food security, source measurements, etc.) and 4)
Identify mechanisms and build support for regional nutrient centers. USDA leads are currently planning a
second workshop in the near future that will bring in a larger stakeholder audience (expected fall 2015,
date TBD). During the workshop, EPA, USGS and USDA leads agreed that better capture and
coordination of nutrient research within our respective agencies was needed and because of this we
recently obtained funding to support a shared post-doc with ORD and USDA-NIFA staff to focus on
connecting research databases across agencies and mapping the research.

Currently we have several international efforts that scientists across EPA are actively involved in,
including:

•	The Reactive Nitrogen Research Coordination Network funded by the National Science
Foundation and others.

•	EPA participates and co-leads an inter-agency Critical Loads of Atmospheric Deposition Science
Committee under the National Atmospheric Deposition Program, working on critical loads for
nitrogen and sulfur deposition with other federal agencies, states and private entities.

•	EPA also participated in the Bioeconomy Federal Strategy Workshop, and it engaged in an
Interagency Working Group on Harmful Algal Bloom and Hypoxia Research and Control -hypoxia
subgroup

•	EPA scientists have ongoing joint international and interagency nitrogen research resulting in
published papers with several institutions, including Environment Canada, Tsinghua Univ.,
Beijing, French National Institute For Agricultural Research (INRA), Center for Ecology and
Hydrology (CEH, Edinburgh, UK, Department of Defense, Department of the Interior, and the
National Park Service

•	There is a nitrogen related inter Agency agreement with the Department of Energy on
Understanding the Evolving Interdependencies between Energy Development and the
Environment

•	EPA scientists are participants in the new UNEP project towards the International Nitrogen
Management System ('Towards INMS') that involves synthesizing national and global data on
nitrogen inputs, management and scenarios of future N use.

The discussion of cross agency, industrial, and international cooperation is insufficient. The Gulf of
Mexico and Great Lakes are obvious candidates for international cooperation. There is also great
opportunity for coordinating with and making use of research on control of nutrients and co-pollutants
being conducted in other countries. There should be more discussion of cooperation and collaboration
opportunities in the roadmap.

49

-------
EPA Response: We agree that there is a lot of nutrient research being conducted internationally, and
that the Gulf of Mexico and Great Lakes are candidates for international cooperation. The Gulf of
Mexico Program Office (GMPO) supports international cooperation by implementing binational (U.S. and
Mexico) early-warning detection systems for coastal community management of harmful algal blooms.
GMPO is further engaged in supporting EPA efforts with the Gulf Hypoxia Task Force; in supporting State,
Region and OW efforts to address water quality nutrient management; and in supporting the coastal
component of the OW National Aquatic Resource Surveys, which includes nutrients among other stressor
assessments. ORD interacts with GMPO and Region 4 on a current project to develop genomic-based
indicators of aquatic life impacts due to nutrient pollution in agricultural watersheds of the northern Gulf
of Mexico.

EPA serves on the Great Lakes Water Quality Agreement (GLWQA) Annex 4 - Nutrients: Objectives and
Loading Task Team for Lake Erie. Annex 4 is international and co-chaired by Region V with Great Lakes
National Program Office staff. This group has been meeting for 2 years and is presently setting
Phosphorus Loading targets for Lake Erie with respect to Microcystis blooms and Hypoxia. In addition
EPA researchers provide technical guidance to GLNPO regarding nutrient loadings and modeling in Lake
Ontario in preparation for GLWQA Annex 4 - nutrients discussions. EPA/ORD is conducting field studies
and mathematical modeling of nutrients in Lake Michigan under the Great Lakes Restoration Initiative
and in preparation for GLWQA Annex 4 - nutrients discussions. Th to provide research on survey designs
and indicators for the Great Lakes and to provide understanding of coastal and lake-wide responses to
watershed stressors that include nutrient loading.

EPA and Environment Canada have implemented and amended the Great Lakes Water Quality
Agreement that commits the United States and Canada to cooperate and coordinate efforts on such
issues as updating phosphorus targets for open waters and nearshore areas of each Great Lake and
taking actions to reduce phosphorus levels that contribute to harmful algal blooms. Development of
further collaborations and cooperation opportunities will be pursued as appropriate for given research
objectives. The Nitrogen and Co-pollutant Roadmap lists some 17 areas where collaboration among
agencies might be expanded to enhance the research enterprise. These listed collaborations include
EPA's STAR (Science to Achieve Results) program funding of four nutrient research centers, including one
through the industry group, the Water Environmental Research Foundation (WERF) for the National
Research Center for Resource Recovery and Nutrient Management. Additional collaborative
opportunities have included: emphasis on tool development using open source or community software;
support of U.S. industry and international client communities comprised of tool distribution, model
development, training and support; sharing of field measurements with industry and international
colleagues; model intercomparison studies with industry and international colleagues; taking a
leadership role in EU sponsored and global conferences; and ongoing scientist exchanges with non-U.S.
community members. The Roadmap brings EPAs research to a point where it can be used to explore
further opportunities to expand research integration at an international scale, which will be explored in
future iterations of the Nitrogen and Co-pollutant Roadmap.

Key Recommendation: Include an extended discussion of uncertainties associated with modeling and
assessment of impacts of proposed management actions. There is little mention of scientific
uncertainty; this is a serious omission. Modeling and assessments of impacts of proposed management
actions should be accompanied by a defensible quantitative statement of uncertainty. If stakeholders
and/or decision makers are considering management actions based in part on modeling/assessment,
they must be provided with some measure of the confidence (uncertainty) in the science. For too long
and too often, the EPA has failed to insist on this requirement for predictive models. As a consequence,

-------
ORD has tended to stress large elaborate models, such as that described in the section on integrated
multimedia modeling, that appear to be motivated by the assumption that models must be sufficiently
detailed so the modelers can "get the processes right." The result of stressing the development of
elaborate models is that these models can become over-parameterized.

Among experienced modelers, it is well-recognized that many sets' of parameter values will fit large
simulation models about equally well; similar predictions can be obtained by simultaneously
manipulating several parameter values in concert. This is expected because all models are
approximations of actual ecosystem processes, and because all parameters represent aggregate
processes (spatially and temporally averaged at some implicit scale) and are unlikely to be represented
by a fixed constant across scales. In addition, many mathematical structures impart extreme correlation
among model parameters, even when the model is over-determined. It is recommended that ORD
routinely require uncertainty analysis of model forecasts.

EPA Response: EPA and ORD are keenly aware of and routinely conduct sensitivity and uncertainty
analyses using various methods on a wide range of models and have added this to Science Challenge 3.
There are EPA, the Environmental Modeling Community of Practice (E-CoP), and National Academy of
Science guidances on conducting such analyses. For example, in development of the coastal ocean
models to examine Gulf of Mexico hypoxia, the research plan includes a quantitative uncertainty analysis
in FY15. Improving estimates of model uncertainty is the principle reason for including two different
ecosystem models (GEM and GoMDOM) in the research effort, and is why ORD participates in the NOAA
COMT (coastal ocean modeling testbed) model intercomparison study with academic modelers.

Uncertainly in parameterization of model coefficients can be mitigated by calibrating the model to a
good observation dataset. ORD oceanographic data collection has resulted in an unprecedented 5+ year
peer-reviewed dataset to calibrate against. In GEM the subjectivity of calibration is removed by using an
evolutionary optimization algorithm that searches for the best parameter set (within specified bounds)
to fit the observations. In GoMDOM, a more traditional approach is used of iteratively changing a
parameter and examining model i/s observation. The degree of fit between the model and observations is
then quantified using standard error metrics such as bias and root mean square error and calculate
model prediction skill. To put error bars on the model results, parameter uncertainties must be
propagated through the model. Efforts in FY15 will include error propagation work using a Markov Chain
Monte Carlo approach through the EPA Computer Center, EMVL. Other uncertainty issues that will be
specifically addressed in the research include the issue of non-unique parameter sets that may provide
the same or very similar results, uncertainty in the model structure, and uncertainty in the boundary
conditions.

For the evaluation of wetland and nitrogen sediment-flux models being developed under SSWR,
approaches have included the use of Generalized Sensitivity Analysis (GSA) and Global Sensitivity Analysis
(GloSA) as parameter sensitivity methods. Both sensitivity methods are sampling based and require
Monte Carlo Simulations (MC). For uncertainty analysis, researchers have used the Generalized
Likelihood Uncertainty Estimation (GLUE) method and a formal Bayesian methodology (Bayesian Monte
Carlo and Maximum Likelihood (BMCML) method'), recently developed for model calibration and water
quality management. Other methods for sensitivity analysis that have been used in the past include first-
order and second order analysis (FOA) using condition numbers, and One-factor-At-a-Time (OAT)
sensitivity analysis.

-------
The air quality models undergo near continuous challenges through systematic evaluation and sensitivity
analyses. These range from multi-model inter-comparisons, to evaluations against new, special field
campaigns, to dynamic evaluations against an observed change signal to challenge the model
parameterizations and test a model's ability to estimate change, a key use of the models. Ensemble
estimates of futures are being examined as an approach to address model uncertainty in process-based
models, an approach being developed in coordination with clients and stakeholders.

Bayesian methods for evaluating model uncertainties and model likelihoods, given new data are being
used in empirical models for use in eutrophication in lakes, reservoirs, and some estuaries. The approach
is similar to the above approaches and includes the use of Bayesian and MCMC sampling algorithms,
prior probabilities for model parameters of simple process-based empirical models, and parameter
updating based on new observations (e.g., for lakes from EPA's National Aquatic Resource Survey; or
moored instrumentation in estuaries). The relative strengths of different models, given the same set of
observations can be compared. When available, likelihood-based methods that simultaneously minimize
both bias and variance are used to improve model parsimony and reduce

overparameterization. EPA/ORD has published on the explicit treatment of uncertainty and propagating
uncertainty in nutrient budgets (Lehrter and Cebrian 2010; Uncertainty propagation in an ecosystem
nutrient budget. Ecological Applications 20(2): 508-524.)

In response to this comment and related comments in Appendix B, we agree that there is a need to
quantify connections between water quality criteria and designated uses. Doing so effectively and
quantitatively ought to be the cornerstone of our scientific foundation for nutrient management. Being
clear and direct will make the need for nutrient management that much more compelling to
stakeholders. The research that we will be conducting in an SSWR project on nutrient thresholds was
conceived from this conviction, starting with useful ways to quantify aquatic life use attainment and
linking water quality and aquatic life responses. We agree, as this SAB/BOSC comment alludes to, that
there are serious questions about the quantitative uncertainty associated with nutrients and aquatic life
"endpoints" and that there is an ongoing need to build on the data and analyses that quantify those
linkages.

We are looking forward to future interactions with the BOSC to articulate our approach to uncertainty
and to communicate the results of the analyses.

Key Recommendation: Clarify how research priorities are set based on gap analyses and consider a
"value of information assessment" approach. It is good to see frequent referral to gap analysis
however, it is unclear how research priorities will be established based on the gap analysis. This should
be clarified in the document "Adequate" is a word that is frequently used in the draft roadmap to
characterize expectations for new research (e.g., "to determine if an approach is adequate"). However,
the document does not address how "adequacy" of a research program is to be rigorously assessed.

EPA Response: The Roadmap is intended as a tool to inform the research planning process led by each of
the six National Research Programs. Using the extensive gap analysis and priority needs identified in the
Roadmap, the Roadmap team has worked with the National Program Directors to ensure alignment with
the StRAPs at the strategic level, and will continue to work with the NPDs as well as the Labs and Centers
through our annual planning processes to identify what research ORD will actually be able to accomplish,
given expertise and resources. Ongoing conversations with the users of this research (e.g., Program and
Regional Offices) will help EPA refine what is meant by "adequate" in the context of specific research
projects. Although we did not use the Value of Information Assessment approach in the gap analysis, it

-------
is an interesting concept for further planning. We will have to investigate further to see if the technique
can be used in our construct to compare research, such as air quality model improvements to ecosystem
services evaluations. We appreciate the recommendation.

Recommendation: Address how the roadmap will be implemented. There is no implementation plan in
the roadmap. Who will be responsible for overseeing, monitoring, and coordinating implementation of
the roadmap? This is a very important concern for the SAB and BOSC.

EPA Response: We've clarified the purpose of the Roadmap within the Executive Summary and
Introduction. We hope it is clear that the goal of the Nitrogen and Co-pollutant Research Roadmap is to
develop a common understanding of the Agency nitrogen and co-pollutant management goals, the
research program portfolios developed by the ORD National Program Directors (NPDs) in relation to the
OW and OAR program offices' priority research needs, and to identify major focus areas, opportunities
for integration across the Agency, research gaps and future research directions. As such, the Roadmap is
not a research program or implementation plan in and of itself. Rather, the results of the Roadmap's
analyses will be used by Program offices and ORD Research Programs to inform the design of integrated
research portfolios and policy mechanisms. The research itself is then implemented through the ORD
Labs and Centers. We have also proposed several suggestions related to implementation in Section V of
the Roadmap, under "Informing 2016-2019 Research Planning".

-------
5.3. Climate Change Roadmap

Executive Summary: Creating the EPA climate change cross-cutting roadmap is a challenge and the
current draft roadmap on climate change is somewhat disappointing. The EPA's resources devoted to
climate change, a critical environmental issue, are a small percentage of the overall federal climate
change budget. ORD can best approach this budget challenge by focusing resources on "actionable
science" that informs ways the EPA can help address how climate influences air and water quality.

Charge question 8a. How effective is each Draft Roadmap in presenting a problem statement,
elucidating key research topics, capturing relevant research in each of the six programs, and
identifying any important scientific gaps?

Although climate is explicitly a part of ORD's ACE program, the science and impacts of climate change
span all ORD Program areas, and therefore it is an excellent choice for cross-cutting research and
integration across ORD. Creating the EPA climate change cross-cutting roadmap is a challenge and the
current draft roadmap on climate change is somewhat disappointing. The EPA's resources devoted to
climate change, a critical, crosscutting environmental issue, are a small percentage of the overall federal
climate change budget. ORD can best approach this budget challenge by focusing resources on
"actionable science" that informs ways the EPA can help address how climate influences air and water
quality.

EPA Response: The Climate Change Research Roadmap is continuing to undergo revision and will be
reviewed again by the BOSC executive committee later this year. The revised draft clarifies that the
Roadmap is intentionally and explicitly focused on ORD's research that addresses the research needs of
the Agency. Specifically, it centers on research that enables EPA to achieve its mission to protect human
health and the environment, most immediately for those areas in which EPA has a legislated
requirement, such as air quality, water quality, waste management, and ecosystem health. It is within
these areas that research is most clearly and consistently requested by our EPA partner offices to inform
their need to address the climate-related decisions they currently face, and the Climate Roadmap was
developed to address those needs in a way that ensures the science conducted by ORD is "actionable,"
meaning that ORD and our partners expect the results to be of immediate utility to decision makers. Such
work includes the "20 Watersheds" study that provides OW with understanding of the effects of climate
change on water quality, the climate model downscaling work that enables evaluation of potential
future impacts of climate change on local and regional air quality, and application of robust decision
making methods to climate adaptation decisions in the Chesapeake Bay.

5.3.1. Problem statement and key research topics

Key Recommendation: Describe how planned research will inform future EPA decision making. The

roadmap should address what science can be brought to bear on the consequences of some high profile
decisions [e.g., Corporate Average Fuel Economy (CAFE) standards, power plant emissions, renewable
fuel standards]. Moving forward, the EPA should focus on what the life cycle consequences of
renewables—solar, wind, tidal, and biofuels—might be. For biofuels and biochar, in particular,
knowledge of the impacts on water availability, crop production and disease needs improvement. Many
agricultural activities have substantial impacts on air pollution and human health that need
investigation. It is important for the StRAP to focus on actionable science related to the EPA's regulatory,
voluntary, and information-based programs, as well as technologies and outreach to other countries.

-------
EPA Response: ORD works closely with our program office partners to ensure that the Agency's high
profile decisions are based upon the best available science, and that the Agency provides consistent
information about the consequences of those decisions. A substantial portion of the analyses conducted
in direct support of those decisions, including their consequences, is conducted by the Office of Air and
Radiation or the Office of Water. While ORD provides scientific and technical input and review of those
analyses, our research more often focuses on methods, models, data, and assessments that are more
foundational to the development and analysis of specific EPA actions. This approach is analogous to that
followed in the development of the Integrated Science Assessments that inform decisions on the National
Ambient Air Quality Standards. In the area of climate change, this approach was taken with the
Synthesis and Assessment Products developed for the 2009 National Climate Assessment as well as the
upcoming Assessment of the Impacts of Climate Change on Human Health. Both of these Assessments
were, or will be, interagency products, but were developed to inform EPA policy decisions. Similarly, the
Assessment of Climate Change on Water Quality will be an ORD product developed as a foundational
document to inform EPA decisions related to water quality, while analyses of specific EPA regulatory
actions will be conducted by the Office of Water.

Several of the topics that the SAB/BOSC recommends evaluating are addressed through interactions with
other federal agencies. ORD and EPA work closely with these agencies to ensure that the required
scientific information is developed and communicated. Interagency bodies such as the Committee on
Environment, Natural Resources, andSustainability (CENRS) and the Council on Environmental Quality
(CEQ) provide fora for agency-level discussions, with interagency working groups under each body to
foster program- or project-level communication, coordination, and interaction to ensure that needed
data and understanding are developed and transmitted. Through these and other interactions, EPA is
able to influence the relevant research of other agencies, which are generally quite interested in
providing information of use to EPA's regulatory and voluntary programs. Examples of such interactions
include the development of water availability data by the Department of the Interior, which is then used
by EPA to address how water availability, including availability of the various types of water, affect water
and drinking water quality. Similarly, the issue of crop production and disease is addressed by the
Department of Agriculture, even as these data are of importance to EPA's biofuels regulatory programs.

ORD's climate research is designed to provide the actionable science to address the priority needs of
EPA's regulatory, voluntary, and information-based programs, and therefore focuses on scientific topics
that directly impact air quality, water quality, and waste management, and subsequently human health
and the environment. ORD's efforts to develop new research to support the interagency Climate Change
Health Assessment, initial steps to develop a climate-water quality assessment with OW, and research to
evaluate remote sensing methods for methane are examples of actionable science in support of EPA's
programs. The Climate Change Research Roadmap has added text to more explicitly communicate this
point.

Key Recommendation: Describe more clearly the ORD climate change research niche ("actionable
science" and its plan to work with other international and federal partners to ensure EPA's science
needs are met).

The roadmap should describe integration with key domestic and international research efforts on
mitigation/adaptation. Although some of this information was communicated verbally in the Climate
Change Roadmap presentation at the July 24-25, 2014 SAB-BOSC meeting, the roadmap should include
this basic information. The roadmap should address how ORD accounts for research needs not currently
addressed by its own research programs. An example is the water-climate question, i.e., why ORD seems

55

-------
only to analyze a one-way relationship, considering only the climate impact on water and not how water
affects climate. If other agencies are focusing on the water-climate question, how is that incorporated?
A clear guideline for such partnerships should be included in the roadmap. The roadmap should also
communicate more clearly the unique role that ORD has in climate change research relative to other
federal partners.

EPA Response: Integration with domestic research efforts on mitigation and adaptation occurs through
a variety of venues. As with any research organization, connections between ORD researchers and
researchers in other domestic and international organizations is critical to maintaining scientific
relevance. More formally, ORD's strategic research directions are coordinated with those of other
federal agencies through the U.S. Global Change Research Program (USGCRP), which develops annual
research budget priorities at the federal level, and coordinates research activities and fosters interagency
collaborations through 12 interagency working groups. The Roadmap text has been revised to more fully
explain these activities.

EPA is one of 13 Federal agencies that perform climate change research coordinated at the federal level
by USGCRP. USGCRP agencies conduct research that addresses changes to atmospheric and oceanic
circulation, weather (including extreme events), fish and wildlife habitats, water supply risks,
transportation, energy production and demands, and agriculture, among others. The focus of EPA
research is to understand how climate change may affect human health and the environment to enable
EPA to meet its legislated responsibilities. A table has been included in the Roadmap to provide
information about the climate-related data from, and research conducted by, other agencies that is
relevant to EPA.

Many Interagency research efforts are coordinated using Memoranda of Understanding (MOUs) that
define research activities and resource investments within each participating agency and how
information will be shared across agencies. ORD's MOUs with other federal agencies define
commitments of the agencies toward achieving a common goal without transfer of resources. Some
MOUs related to climate research include those with the Army to evaluate "Net Zero" technologies, with
NASA to promote collaboration on environmental and earth sciences, with NIEHS on children's health
(including effects of climate), and with USGS to develop collaborative research on aquatic species habitat
modeling. Further informal interagency collaborations occur more frequently based on recognition of
complementary expertise and resources, mutually beneficial outcomes, and availability of resources.
ORD and EPA more broadly are in frequent conversations with staff from other agencies, promoting
good awareness of capabilities and needs across agency boundaries. These interactions are quite diverse
in their constitutions and goals, EPA benefits substantially from strong interest on the part of other
agencies to interact and conduct research that will be applied to addressing EPA needs. The Roadmap
text has been revised to better explain these activities.

The Subcommittee correctly observes that most of EPA's climate-water research is primarily related to
how water is affected by climate. With regard to the question regarding water's impacts on climate, ORD
conducts limited work related to "blue carbon," that is, the role of aquatic ecosystems as a carbon sink.
The primary focus of ORD's efforts related to aquatic ecosystems are related to ecosystem health, with
evaluation of the carbon sink potential as a secondary component. ORD also conducts work designed to
reduce greenhouse gas emissions from water treatment systems within the SSWR program. The major
impacts of the role of water on climate, specifically the role oceans play in global flows of heat and
carbon, are addressed by research conducted or supported by NOAA, NASA, DOE, and NSF. ORD works

-------
closely with these agencies to ensure there is clear communication of research needs and results across
the agencies, through efforts noted above.

Key Recommendation: Improve the presentation and flow of the climate change roadmap. The

roadmap would be improved by the addition of diagrams, model schematics, and other organizing
approaches to help convey the systematic approach that ORD is taking in this important cross-cutting
area.

EPA Response: A figure has been included to illustrate the role of the roadmap in guiding the planning
and implementation of climate research across ORD's research programs. The Roadmap text has also
been changed in several places as described in this response to improve clarity and flow of the Roadmap
narrative.

5.3.2. Key research topics and relevant research in each of ORD's six programs

Recommendation: Identify research priorities associated with the problem statement The climate
change topic is broad and the EPA research role is not well defined in the roadmap. Outlining a few basic
components of research to inform future EPA decision making would be useful. The roadmap should
more clearly identify its relationship to the EPA's Climate Change Adaptation Plan. The roadmap should
also more clearly communicate the rationales for the EPA program-identified "Research Needs" listed in
Appendix C.

EPA Response: Regarding EPA's research role, see the responses to Recommendation 5.3.1 above.
Section IV of the Roadmap describes current research efforts designed to inform EPA decision making.
Specific examples of the descriptions in Section IV include: (1) guidance to incorporate changing
temperatures into regulatory programs for water quality and hydrologic and biogeochemical sensitivity
to climate change; (2) development of downscaling methods to enable chemical transport air quality
models to project changes in air quality due to climate change and investigations into the effects of
climate change on particulate matter concentrations; and (3) health impacts of increased allergens and
changes in waterborne and infectious diseases associated with a changing climate. These efforts are
examples of how ORD's research is responding to the needs identified by our partners to inform their
decision making.

The majority of research needed by our partners focuses on adaptation, as reflected in the needs
identified through a variety of on-going interactions between ORD and our EPA partners. The Roadmap
has been revised to include a summary of EPA's major programmatic drivers, how they are connected to
climate change, the research needs that result, and the major sources of scientific and technical
information to meet those needs.

Executive Order 13514 required EPA to develop an Agency-wide Climate Change Adaptation Plan to
ensure that EPA is able to achieve its mission in the face of a changing climate. The EPA Adaptation Plan
and the 17 Office-specific Climate Adaptation Implementation Plans provide additional insights into the
research needed to understand and respond to climate change. The research priorities developed by
ORD's partners in EPA were developed in parallel to the vulnerabilities identified in the Adaptation
Implementation Plans, and were informed by an awareness of those vulnerabilities. Under the Climate
Change Adaptation Plan, EPA is developing cross-EPA priority science needs. Although this effort is in
progress, discussions to date indicate the cross-EPA priority science needs have been captured by the
program-specific research planning effort and do not reflect significant gaps in the ORD research

-------
portfolio. Similar text concerning the Adaptation Plan and the Office-specific Implementation Plans has
been added.

The Research Needs presented in Appendix C are the result of focused discussions between EPA Program
and Regional Offices to determine their most important needs for scientific information. The needs
identified by the Program and Regional Offices reflect the issues those Offices are currently facing, or
expect to be facing in the near future, and for which additional scientific information is needed to inform
decisions on those issues. This text has been added to Section V.

5.3.3. Important scientific gaps

Key Recommendation: Expand the roadmap discussion of "Research gaps and priority research needs"
(social sciences, uncertainties, decision-relevant scale, and synthesis). The SAB and BOSC recommend
that the roadmap describe more clearly the research intended to address these gaps. In regard to the
social sciences, there is an opportunity in the climate change roadmap to clearly identify what is meant
by social science research supporting the EPA's mission and how such research would be used. It is
important to "unpack" the heterogeneity of social sciences and to learn what is most needed for the
agency. The social sciences may be especially important for ORD to consider because EPA's climate
change science must be communicated clearly to and with the public. The SAB and BOSC also are
interested in how ORD plans to help inform decision makers of the timing, magnitude and uncertainties
of climate change. Communication to decision makers regarding these topics is an additional area of
important research.

EPA Response: We agree that the Roadmap can be clarified to describe research to address gaps. The
Roadmap text in Section V will be expanded to incorporate the recommendation to more clearly describe
ORD's plans to conduct research needed to address priority needs.

We agree with the point that social science is heterogeneous (similar to physical science). The scope of
relevant social sciences expertise related to climate change is substantial, including economics, decision
science, communications, community and institutional dynamics, planning, policy formulation, and
cultural heritage. Text has been added to the Roadmap to provide a better indication of what is meant
by social science research. Discussions regarding the incorporation of social sciences into climate
research are on-going, with guidance from National Research Council reports and a white paper
developed by USGCRP.

We agree that communication of EPA's climate change science can benefit from an understanding of the
social sciences related to communication and community involvement, among other topics. The growing
involvement of the Sustainable and Healthy Communities program in developing community resilience
has brought increased social science capabilities into ORD's climate research efforts, as noted in Section
IV of the Roadmap. Communicating EPA's climate science has benefitted from the growing citizen
science and community resilience efforts, both of which involve increased understanding of social
sciences. Text has been added to the Roadmap to convey this point.

We also agree with the point about informing decision makers about the timing, magnitude, and
uncertainties of the impacts of climate change, and ORD is actively working on this topic. One of the
findings of the effort to develop cross-EPA climate research priorities is the consistent message from
across the Agency that closer interactions are needed between ORD and EPA's Program and Regional
Offices to ensure clear and consistent information about the applicability of research on climate change

-------
and impacts, including timing, magnitude, and uncertainties. This is described in added text in the
Roadmap.

5.3.4. Implementation and intended use of the roadmap

Recommendation: Provide a discussion of how the roadmap will be used to guide research. Many of
the suggestions above indicate a need for further development of the roadmap. A revised roadmap
should include discussion of implementation factors (e.g. dependency on personnel, resources), and the
intended use of the roadmap. The roadmap should indicate how it will evolve and whether there will be
indicators or milestones for evaluating the program. Successful integration depends on how
participating programs and laboratories commit to collaborations proposed. The roadmap should
discuss how budget and personnel resources are committed to the roadmap activities and how planning
across ORD to meet the climate change research needs will occur.

EPA Response: The purpose of the Climate Change Research Roadmap is to develop a common
understanding of EPA's climate changes research goals, the research program portfolios developed by
the ORD National Program Directors (NPDs) related to climate change research, the EPA program and
regional offices' priority research needs, research gaps and future research directions. The Climate
Change Research Roadmap is not a research program or implementation plan in and of itself. Rather,
the Roadmap will be used to guide priority-setting in the National Research Programs to inform the
design of integrated research portfolios. The research is then implemented through the ORD Labs and
Centers, which are responsible for personnel planning and management.

-------
5.4 Environmental Justice Roadmap

Research in Environmental Justice, as was the case in CEH, spans and develops links from basic science,
models, and mechanisms to the health and well-being indicators of communities and their
neighborhoods. Consequently, it is an excellent choice for cross-cutting research and integration across
ORD program areas. The Environmental Justice roadmap provides a good framework for a research
path in environmental justice.

5.4.1 Problem statement and key research topics

Although the problem statement is well described, specific goals and objectives are not. Without
anticipated achievements, it is difficult to know which steps should be taken in a research roadmap to
lead to effective results.

EPA Response: The EPA appreciates the support of the two committees for the development of the
Environmental Justice (EJ) Cross-cutting Research Roadmap, as well as for their endorsement of the
research framework that the Roadmap provides. Members of the SAB and BOSC noted that since the
document was in a very early stage of development, it did not include specific research topics or
proposed methods to address EJ issues. These missing pieces are currently being added to the draft
document, along with a stronger articulation of the specific goals and objectives of the research. As the
revised roadmap undergoes continued development and revision it will be reviewed again by the BOSC
executive committee later this year.

Key Recommendation: Incorporate input from communities to identify problems associated with
environmental, biological, behavioral, social, economic and spatial stressors, and how they
interrelate. The roadmap mentions the involvement of stakeholders but doesn't specifically describe
their expertise or experience, or what demographics they represent. ORD should partner with
individuals from a variety of communities that represent various situations and circumstances.
International partnerships are also lacking. The roadmap does include non-chemical stressors, which
provides the broad approach to problem identification and evaluation. However, conducting
conversations with communities is critical in identifying these problems and possible desired solutions,
as well as strategizing effective research approaches. This type of expertise that can only be obtained
from community partners is lacking in the draft roadmap. Incorporating this input would allow for the
identification of problems related to environmental, biological, behavioral, social, economic and spatial
stressors, and how they interrelate. Community involvement also fosters a "practice" approach to
problem-solving, as well as a research-based regimen.

The draft roadmap does not include specific research topics or proposed methods to address
environmental justice issues. As such, research priorities are not identified, along with the trade-offs of
their inclusion. Three science challenges are listed, with community engagement included as a separate
item. It is suggested to integrate community participation throughout each science challenge and have
community individuals inform the research process. Community involvement will also help identify
research priorities. Section B.4. in Appendix B provides suggestions for research topics/approaches to
explore. Involving individuals within communities as research partners at the onset of developing
environmental justice research strategies and throughout the research process ensures that the
research is "highly relevant and responsive" (Environmental Justice Roadmap goal).

-------
EPA Response: ORD agrees that integrating perspectives of a wide range of community stakeholders
throughout the research process — during the development of the research agenda, the conduct of the
research, and the application of the new data, tools and technologies in support of community decision
making - is essential. This recommendation was recently reinforced by the National Environmental
Justice Advisory Council1 and other stakeholders consulted by ORD. The Environmental Justice (EJ)
Research Roadmap will highlight where community engagement is occurring in research activities that
are part of both ORD's extramural grants program (e.g., the EPA-NIH Centers of Excellence for
Environmental Health Disparities Research) and the intramural research program (e.g., the development
and testing of the Community-Focused Exposure and Risk Screening Tool, or C-FERST). Emphasis will also
be placed on seeking opportunities to involve community stakeholders in future EJ-related research
activities. Finally, as recommended by the SAB and BOSC, community participation (currently described in
the EJ Roadmap as Science Challenge #3) will be integrated into the other two Science Challenges to the
extent possible.

5.4.2. Relevant research in each of ORD's six programs

Recommendation: Add text to Appendix A that explains specific connections to each program area,
using the CEH roadmap as a model format. The draft roadmap demonstrates the need for
environmental justice across ORD's six research programs showing some synergies, particularly with the
SHC program area. More detail, however, is needed. Table 1 and Appendices A and B begin to show how
environmental justice can provide a research foundation for the research program areas, but there are
not as many examples of an environmental justice presence within some research programs as might be
expected. It is suggested to add text to Appendix A that explains specific connections to each program
area. Although sets of projects are listed that show how environmental justice can be integrated, the
current format is disjointed rather than an informative, "stand-alone" document. It is suggested to use
the CEH roadmap as a model for communicating such information.

EPA Response: ORD agrees with the recommendation to better articulate the role of environmental
justice throughout all six research program areas. As noted, this Roadmap was in the early stages of
development and as we expand and revise it, we will work to better synthesize the information from all
six national programs. This should aid in creating an integrated, "stand-alone" document as
recommended.

Key Recommendation: Consider including examples to illustrate relationships to ORD's six research
areas and employing Community-Based Participatory Research (CBPR) to promote research relevance.

The inclusion of non-chemical stressors in the Environmental Justice draft roadmap helps address the
cross-cutting issues related to the six research program areas, but including specific examples would
better illustrate these relationships. To promote research relevance, CBPR could be employed in each
research program area. This could lead to the incorporation of environmental justice issues and help
develop relevant Requests for Proposals (RFPs). This roadmap could provide the needed intersections
for each of the six research programs, which would provide an overall framework for ORD's research
strategy.

1 httpi//www.epa.gov/environmentaliustice/resources/publications/neiac/neiac-research-recommendations-

2014.pdf

-------
EPA Response: EPA agrees that the use of specific examples would be beneficial in demonstrating the
importance of non-chemical stressors in environmental justice research. The use ofCBPR is an area of
increasing growth for EPA, and will continue to be expanded as we move forward in our research
implementation. For example, ORD and the EPA Regions work together to engage communities in the
development of health impact analyses. The communities convey their most significant concerns and
work with EPA on the development of health impact analyses that help inform local decisions. As noted
by both committees, one of the current gaps in EPA's research program is the lack of social science
expertise across all six programs. In addition to our responses elsewhere in this document about
workforce planning and building in house social science expertise, the STAR program helps fill some of
these gaps through extramural research grants. As we revise our current environmental justice roadmap,
we will look to this program for increased CBPR efforts to promote the relevance of the research in the
EPA. This will allow us to show the translation of our research efforts to benefit communities as well as
fill needed gaps in expertise. Further, our work with communities will serve an excellent method for
furthering increased integration across the six research programs.

5.4.3. Important scientific gaps

Recommendation: Identify environmental justice scientific gaps emerging in major ORD research
programs. The draft roadmap does not identify scientific gaps. As details regarding key research topics
and how they relate to the six research program areas emerge, scientific gaps should become apparent.

5.4.4. Implementation and intended use of the roadmap

Although the Environmental Justice cross-cutting roadmap well documents the need for cross-cutting
research in this area, it provides a relatively abstract discussion of the science. The issues are discussed
in general, but the specific science proposed to address needed knowledge gaps is either omitted or
discussed in general terms. Because of the preliminary nature of the document, sections on research
gaps and research needs, examples of ORD integration, and opportunities for additional integration are
not included. While the underlying science challenges are well-described in a general sense, the cross-
cutting roadmap should include discussions of these topics so that ORD research in this area is
coordinated as a well-organized whole.

EPA Response: EPA agrees with the need for identification of scientific gaps in the area of environmental
justice research. Because of the preliminary nature of the document, sections on research gaps and
research needs, examples of ORD integration, and opportunities for additional integration were not
included. As we revise and complete this roadmap, information will be added related to the key research
gaps and data needs in environmental justice, as well as proposed examples of ORD integration to aid in
addressing these data gaps. This area will be informed by ORD's interactions with our EPA partners in
the Office of Environmental Justice, stakeholders like NEJAC, as well as communities in which we are
currently engaged.

-------
5.5. Integration across ORD Programs

Charge Question 9a. Do ORD's plans, taken collectively, indicate that integration, where appropriate,
will develop the needed scientific knowledge and produce results that advance EPA's ability to
address complex problems?

Integrated research is critical because of the EPA's resource-limited environment and the
interdisciplinary, cross-program application of ORD's data, tools, knowledge and products. Integration
must occur internally within the EPA, external to the EPA within the United States with the agency's
partners, and internationally. The ORD's cross-cutting roadmaps represent a very important step
forward for the EPA in providing a framework for research integration on large-scale, complex
environmental challenges. These roadmaps provide frameworks for integrating research across ORD
programs and offices, and with other agencies, and for keeping ORD research forward looking.

The SAB and BOSC commend ORD's progress in undertaking this integrated planning and offer
recommendations for strengthening the roadmaps and making them more consistent. Integration must
be a key concern as ORD moves from research planning to research execution, and as it defines a
process for providing research to decision makers that incorporates institutional learning about that
process. Until progress is made or more information is available regarding those points, the SAB and
BOSC cannot determine whether the integrated research described in the StRAPs will produce the
results the EPA needs to advance solutions to the complex problems it faces.

Recommendation: Strengthen the roadmaps and make them more consistent with each other and
with the StRAPs. Sections 5.1 through 5.4 of this report provide recommendations for revising and
strengthening individual roadmaps. ORD has acknowledged that the draft roadmaps were at different
stages of development and completion. In revising the roadmaps and StRAPs, the SAB and BOSC
recommend that additional attention in each StRAP and roadmap be given to: (1) communicating a clear
vision that lays out the key science needs and ORD's research niche vis a vis its research partners; (2)
describing the current state of knowledge to provide baseline data in the research topic areas and
where EPA's current projects fit; (3) describing other actors in the environmental protection scene and
what they contribute to the specific goals and objectives in the plans (there are frequent references to
"partners," but no definition of the term); (4) clarifying and making more consistent the reciprocal
relationships between the sources of research and the targets (in many cases, one program says they
are producing products for another, yet the target program is silent about those inputs); and (5)
including in each StRAP a section along the lines of "Relationships to other StRAPs."

EPA Response: EPA agrees with the importance of greater consistency among the Roadmap documents,
and the specific recommendations above are quite helpful. In particular, the StRAPs and Roadmaps need
to describe a clear vision of ORD's niche in relation to other ongoing research. The final StRAPs and
Roadmaps will include information on how the proposed research plans benefit from integration and
collaboration across each of the National Research Programs, as well as with other Federal partners.

63

-------
Key Recommendation: Identify and communicate ORD research priorities in the roadmaps and
commit ORD resources to them. Plans alone cannot guarantee that needed future integration across
ORD program areas will occur. Given the planning necessary to conduct integrated research and the
heavy demand on research resources to conduct the work, a data-driven prioritization of the questions
ORD chooses to address is necessary. The roadmap mechanism appears appropriate for fostering
integration across programs, but it must be clear who is the steward of priority topics in the roadmap
and who implements and makes sure the research happens.

EPA Response: The intent of the Roadmaps is to look across EPA research to understand ongoing work
on the topic, and to identify scientific gaps. This gap analysis is then conveyed to the National Program
Directors so that important gaps can be addressed through the StRAPs. The StRAPS are the strategic
planning documents for the six research programs. They serve as the basis for resource planning in ORD.
Research that is incorporated into the StRAPs will be implemented managed in the same way as all other
ORD research. We agree that it will be important to track and coordinate Roadmap-related research, so
that progress in addressing Roadmap gaps can be assessed. Our experience with this first round of
Roadmaps will help us to define the ongoing role of the Roadmap leads and to improve the process for
coordinating cross-cutting research.

Key Recommendation: Acknowledge and plan for actual integration, which requires active
collaboration from the onset. Attempts at cross-program integration are evident within the StRAPs and
ORD briefing materials provided for the July 24-25, 2014 SAB-BOSC meeting. Whether these efforts
develop the needed scientific knowledge to advance EPA's ability to address complex problems depends
on the implementation of these proposed linkages. Often, linkages and relationships proposed in
strategic planning documents occur at only a superficial level - for example sharing final results only at
the end of research projects. In other cases, one research program may independently produce research
projects that are reported to be "relevant" to other research programs. To the extent that these linkages
are of this superficial nature, they may not provide the needed scientific knowledge. However, if the
proposed linkages are implemented in a meaningful, in-depth and ongoing manner, they should help
provide the integrated scientific knowledge required by the agency.

To be most useful, cross-program integration should involve active collaborations from the outset of
research projects - so that each project benefits from the combined expertise of multiple programs. The
extent to which linkages reported in the StRAPs will promote truly integrated work is unclear. The
StRAPs and briefing materials characterize cross-program linkages only in abstract terms, for example
reporting broad research topics within each program that are relevant to other programs, or over which
some type of otherwise unspecified interaction is planned. For example, page 8 in the SSWR StRAP
states, "Watershed Sustainability has clear linkages to SHC through the EnviroAtlas and Report on the
Environment, and to ACE, particularly in the realms of climate change and prediction and management
of materials and waste." It is unclear to what extent these "clear linkages" will include active cross-
program collaborations, and how areas such as this will benefit from this integration.

-------
EPA Response: We agree with the SAB-BOSC that achieving meaningful integration requires much more
than identifying potential linkages and relationships at the strategic planning level. However, this is an
important first step. It signals to the laboratory management and the research project leaders that
cooperation and collaboration is expected across the research programs. Opportunities to begin
collaboration occur throughout the research planning process. For example, each research program
holds an annual spring meeting, with ORD scientists and EPA program and regional office staff and
representatives from the other ORD research programs are also invited. These meetings help to shape
priorities for the upcoming year, and future years, and can help identify areas of mutual interest.

As more specific plans for project implementation take shape, integration is built into the proposed
research. Currently, each research program is developing Project Charters for the major research
projects; these Charters describe the planned research, including scope, structure, rationale, team
members, stakeholders, expected products and impacts, measures of success, and
assumptions/constraints. This provides the structure to identify integration needs/opportunities, such as
expertise needed from multiple research disciplines, inputs that will be required from other research
projects, and outputs that will be useful to other research programs and stakeholders. Although this level
of integration is very staff and time intensive, we believe it will lead to greater success, particularly as we
learn and improve the process.

Recommendation: Plan for the human resource and information needs required to carry out
integrated research programs. The SAB and BOSC recommend that the EPA develop a long-term hiring
and training plan (versus short-term hiring of postdocs or supporting external research) to better
integrate behavioral science into their long term research. ORD should institute a formal means for
broad communication and interaction among ORD technical staff to support its integrated research
program. ORD would benefit from widespread use of the communities of practice approach undertaken
by the computational toxicology program for sharing information and collaborating on environmental
research.

Because transparency is identified as a core value in the EPA Strategic Plan and accessibility to
intramural and extramural data and information is a key challenge, ORD would benefit from developing
state-of-the-art information management to provide the optimal interface(s) for the interdisciplinary
scientists engaged in EPA research and the science products held or used by the agency programs.
Development of overall strategic plan goals in the arena of information management science that serves
and integrates the national research programs and cross cutting areas may help assure resources and
commitment to this need.

EPA Response: ORD has established a lab and center management workgroup to identify workforce
gaps and develop ways to enhance the current workforce. So far, ORD used several analyses and studies
to identify a variety of workforce gaps including in the social, behavioral, and decision sciences. To date,
these efforts have been used to focus hiring in selected areas using the Post-Doc authorities. ORD used
these workforce gaps to identify the elements of both a short-term and long-term workforce strategy to
address these gaps and plan for the workforce it will need in the future. The next step is to identify
opportunities to enhance its current workforce to meet these longer-term needs. ORD's senior managers
are currently charged to address the multitude of factors and challenges to develop a sustainable
methodology for supporting both a short-term and long-term workforce strategy.

We appreciate the SAB/BOSC recognition of the communities-of-practice approach used in the
computational toxicology research program as an effective means to foster communication and

-------
interaction among our scientists. These communities of practice host outside scientists, share
information on emerging science, and may share data and methods - and provide a good way to
facilitate integration between and within research programs. For example, the Adverse Outcome
Pathways Community ofPractoce is working to integrate across CSS-SSWR on a specific task, minimizing
overlap and optimizing interactions. Other Communities of Practice are ongoing for statistics, analytical
chemistry, modeling, and scientific data management. EPA is also co-leading an inter-agency
Community of Practice for citizen science.

We agree with the SAB/BOSC on the importance of state-of-the-art information management. Issues of
concern include: meeting new federal open access requirements; keeping pace with the size and rate of
scientific data growth; eliminating duplication of efforts; reducing costs of storing unnecessary data;
providing accessibility to data; and promoting productive collaboration across ORD. Several initiatives
are already underway. ORD drafted a scientific data management policy to address the needs over the
full life cycle of scientific data. After the policy is approved, ORD will implement it in multiple phases with
a gradual introduction of requirements and services. A Guidance is in development to assist ORD
scientists and others in preparing scientific data management plans. New tools are also planned; for
example, ORD is currently developing ScienceHub, a new web application that will enable researchers to
track information, research plans, and manage and share data related to their research efforts.

Recommendation: Identify the criteria or a process for evaluating research "results that advance EPA's
ability to address complex problems." There is a need for identification of criteria or a process for
determining whether a product or research activity has been successful and advanced the EPA's ability
to address complex problems. Regulatory and research activities within the agency must, to be effective,
continually absorb, synthesize, adapt, and use information from within and outside the agency.
"Absorptive capacity" is known to vary greatly among organizations. ORD hears the needs identified by
Program Offices and Regions, does research, and feeds information back to the offices and regions. The
Program Offices and Regions translate the information into regulations and procedures and work with
local authorities to protect human health and the environment. There is some recognition in research
plans of the need for information loops rather than one-way transmission, but no consistent or
systematic attention to this challenge.

In addition, ORD should more clearly define the success of research that meets community needs. In
many places throughout the planning documents, tools to support communities are emphasized, but
"community" is rarely defined with any precision, and "community engagement" is rarely
operationalized into a coherent process.

EPA Response: We agree with the SAB/BOSC on the importance of feedback loops to assess whether the
results of research have proven successful. We have significantly improved our interactions with EPA
partners over recent years. While this initially has focused on research planning, we have been working
with partners to get feedback on the products they have received and used. As a further step, we want
to similarly expand interaction with external partners, to gather their input on the effectiveness of
research outputs.

In general, ORD uses the term "community" to refer to research affecting a sub-regional, sub-state scale,
such as a city or a neighborhood. Research that meets "community needs" may be done for or with a
community directly, or for an EPA program or regional office to use in their work with communities.

Much of the research that meets community needs is in the form of decision support tools. One indicator
of success is the number of downloads or website visits. Beyond that, for tools such as EnviroAtlas or C-

-------
FERST, we are gathering case studies of how communities use tools. Ultimately, we hope to find that
more-informed decisions with positive outcomes are made by communities that use the tools.

ORD, particularly the SHC program, has been working very closely with EPA's Office of Environmental
Justice, the Office of Solid Waste and Emergency Response, and many of our Regional offices, who have
a long history in community engagement. We will continue to work with our EPA partners to better
define the success of research that meets community needs.

Key Recommendation: Implement a process for identifying ORD cross-cutting research topics and
managing their life cycle. The four cross-cutting research projects reviewed by the SAB and BOSC are all
appropriate choices. At the July 24-25, 2014 meeting, ORD explained that the four current cross-cutting
topics were developed in the following way. ORD convened its Executive Council whose members
brainstormed topics and developed a "heat map" showing their relationship to ORD's national research
programs. ORD looked for research topics related to agency priorities where ORD could make a
difference. The number of topics was capped at four to keep the planning and coordination efforts
manageable. The SAB and BOSC note that the individual draft roadmaps did not include a description of
this selection process.

Moving forward, it is not clear how current efforts will mature and be brought to closure or how future
topics will be initiated. If there is no plan to initiate new cross-cutting projects, the four existing efforts,
while highly meritorious, will not fully represent the diversity of integration challenges faced by the
agency. If there is no plan for bringing the efforts to conclusion, then there is the danger of proliferation
of the efforts, diluting their effectiveness.

As ORD considers future roadmaps, the SAB and BOSC recommend that it consider adding SHC as a
cross-cutting topic. This topic, like Climate Change, is associated with its own research program, and is a
topic central to the EPA's protection of public health and welfare, worthy of a cross-cutting roadmap.
Such an SHC roadmap would meet important needs to plan for involving communities and evaluating
community engagement throughout ORD's research programs.

EPA Response: We appreciate the SAB/BOSC committees' enthusiasm for the cross cutting roadmaps
and support for the four initial topics ORD chose. As the committees have recognizeddeveloping these
cross-cutting roadmaps require coordination and collaboration across the Agency, involving significant
staff time, from all six programs and EPA partners over two years. As Dr. Kavlock emphasized, these
roadmaps are being used to improve our ability to integrate research efforts across the existing six
programs and with other research organizations. They are not intended to create additional ORD
research programs.

We agree that SHC is a broad topic central to EPA's mission. SHC is working to make connections across
research programs on projects that involve communities. At a minimum, SHC can share lessons learned
with other researchers, and we can build this capacity more broadly across ORD.

This is the first time we are using the draft Roadmaps in our StRAP development and in research
planning. The roadmaps are not all final at this point in time. We are still in the early stages of learning
the extent to which the development and application of a roadmap can facilitate effective integration
across the research programs. We think it prudent to have some experience working with these
roadmaps and assessing their value before proposing a process for bringing them to closure and
initiating new topics. Regular progress reports will discuss lessons learned and factors affecting most

-------
successful research coordination, collaboration and integration so that we can judge the effectiveness
and status of the roadmaps. At the same time, we will be considering other topics that reach this level of
need for formal program integration. With roadmaps as a living dynamic area of activity, we intend to
consult with the BOSC throughout this learning process and expect that with their guidance we will
consider those next steps. In addition, there is a lot of cross-cutting research that occurs across ORD
that may not require an actual roadmap to be successful. ORD needs to do more to document these
efforts that lead to high quality, integrated research.

68

-------