\ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
8 WASHINGTON, D C. 20460
July 17.2009
Office of
Cooprative Environmental
Karen Sheffer Management
ASRC' Management Services
EPA Headquarters Repository (Library)
MC 3404T
Washington, DC 20460
Dear Ms Sheffer
Enclosed are copies of reports prepared by or for the National Advisory Council for
Environmental Policy and Technology (NACEPT) under the Federal Advisory Committee Act
(FACA) NACEPT is managed by EPA's Office of Cooperative Environmental Management
I. NACEPT Advice Letter NACEPT's Role (April 2009)
2 NACEPT Report Hncoiu aging Regional Solutions to Sustaining Water Scctoi Unlilu">
(March 2009)
3 NACEPT Report- Outlook for the EPA (March 2009)
4 NACEPT Advice Letter' EPA's Draft 2009-2014 Strategic Plan Change Document
(December 2008)
5 NACEPT Advice Letter to the Administrator on Biofucls (December 2008)
6 NACEPT Review of EPA's Strategy for Improving Access 'I o Environmental
Information (November 2008)
7 NACEPT Advice Letter Integrated Modeling (September 2008)
11'you have any questions, please contact me at 202-564-0243 or alticn soma(«}cpa gov
Sincerely,
onia AlTTeri"
Designated Federal Officer
NACEPT
Internet Address (URL) • rinp //www epa gov
Recycled/Recyclable • Pnnied with Vegetable Oil Based Inks on 100% Postconsumer, Process Chlorine Free Recycled Paper
-------
N ACE PT
Shaping the Nation's Environmental Policy
National Advisory Council for Environmental Policy and Technology
September 22, 2008
The Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Re: White Paper on Integrated Modeling for Integrated Environmental Decision
Making
Dear Administrator Johnson:
The U.S. Environmental Protection Agency (EPA) has requested the National Advisory
Council for Environmental Policy and Technology's (NACEPT's) critical review of the
White Paper on "Integrated Modeling for Integrated Environmental Decision Making"
(hereinafter White Paper). In response to the charge, NACEPT formed a work group
consisting of eight NACEPT members and five external members, who were selected for
their experience in the field of environmental modeling (see attached list of workgroup
members and their affiliations). The workgroup conducted its review over a 3 month
period, from late May to August 2008. This advice letter represents the consensus view
and recommendations of the work group and of the full NACEPT council.
NACEPT would like to commend EPA staff for preparing a well-written and
comprehensive White Paper on such a broad and complex topic as integrated modeling.
Our investigation resulted in general concurrence with the White Paper, and confirmation
that computational models are useful tools that play an increasingly important role in the
decision-making process. We agree with the White Paper that integrated modeling
encompasses a broad range of approaches and configurations of models, data and
assessment methods to describe and analyze complex environmental problems, often in a
multimedia and multidisciplinary manner. Furthermore, as environmental issues that
require decision-making become more complex, the panel believes that integrated
modeling will become increasingly important to EPA.
NACEPT believes that the White Paper deserves serious consideration and follow-up by
senior level EPA managers. EPA should give high priority to the systematic
development and deployment of integrated models to inform EPA decision-making. To a
large extent, the science and technology are already available within and outside EPA;
what is needed is strategically directed funding and Agency-wide, organized, and
systematic implementation of the action plan outlined in the White Paper.
-------
NACEPT offers the following three findings and recommendations, which, generally,
support and reiterate points made in the White Paper:
Finding:
1. Integrated modeling is a significant cross-cutting science and technology tool that is
worthy of broad-based strategic review and support within EPA. EPA's Science
Policy Council may be the most appropriate internal coordination body to vet the role
of integrated modeling and how best to ensure its most efficient and effective use.
Recommendations:
• The EPA Administrator should direct the Science Policy Council or other appropriate
body within EPA to analyze and offer its recommendations on the following:
(1) Estimating the initial investment required to move toward the increased and
coordinated use of integrated modeling;
(2) Determining if the development and application of integrated models is suited to
most regulatory timeframes;
(3) Identifying a discrete number of projects that demand integrated modeling
approaches and would lend themselves as potential case studies for development of
an integrated modeling organizational structure;
(4) Evaluating the potential effect on staffing, budgeting, and environmental
protection from using integrated modeling; and
(5) Evaluating the importance of the coordinating function among the research teams
developing integrated models and the program and regional office staffs using those
models.
• The Science Policy Council, or other appropriate body, should provide its findings
and recommendations to the EPA Science Advisor. The Science Advisor should
collaborate with the EPA Deputy Administrator on priority setting and organizational
and budgeting issues relating to additional efforts to develop and use integrated
modeling (IM).
Finding:
2. The White Paper demonstrates the need for a community of practice (CP), which can
serve as a knowledge base, develop and disseminate standard operating
procedures/protocols for integrated model development and application, and provide
means to link the modeling activities of different offices. NACEPT wholeheartedly
supports the community of practice concept. A community of practice will reduce
redundancy in model development, help with model dissemination, and guard against
misapplication of models. A community of practice will not, however, address all
organizational challenges including priority setting, budget allocation, cross-program
collaboration, and stakeholder involvement.
-------
Reco mmendat ions:
• NACEPT believes that addressing organizational challenges will be an iterative
process and recommends that EPA build its IM organizational infrastructure
organically rather than superimpose an artificial structure that may render IM
dissemination more cumbersome. Consequently, NACEPT recommends that EPA
initiate a pilot integrated modeling project, such as biofuels full life cycle assessment,
using the Science Policy Council as the forum in which to discuss model
development and application, and for formulation of organizational/institutional
recommendations for supporting EM within EPA.
• NACEPT recommends that EPA exercise due diligence to guard against misuse and
misapplication of all simple as well as integrated models. While we agree that EPA's
modeling work should be made more broadly available, a community of practice
should help distinguish site- and case-specific models from more generic/evaluative
models. This will ensure that models are effectively matched to the problems they are
attempting to solve. EPA can employ the CP as a means to advance the establishment
of IM both within and outside of the Agency. The CP should be broadly available to
all users, inside and outside EPA.
• NACEPT recommends that EPA support a research program aimed at developing a
quantitative understanding for the processes that require integrated multi-media and
multi-disciplinary modeling.
Finding:
3. Data flow, or more generally "information flow", within integrated modeling systems
is critically important.
Recommendations:
• EPA should automate and standardize the process for accessing, retrieving, and
processing data from nationally available sources (databases) for use in integrated
models. The exchange of information among modeling components and the
processing of model outputs should be standardized. EPA should look for ways to
streamline the model-building process by adopting more uniform formats for
handling and moving information between databases and models and adopting a
common set of methods and software tools that will service a broad range of models
with only minor adjustments. Such tools (or "integrated model utilities") will support
data acquisition, input and output, visualization, pre- and post-processing, evaluation,
decision support, and sensitivity and uncertainty analyses. Close coordination
between EPA's Office of Research and Development (ORD) and Office of
Environmental Information (OEI) will be important in addressing this issue.
In conclusion, NACEPT is generally supportive of the White Paper and hopes this review
provides useful feedback to the Administrator in order to take the next steps towards
-------
implementing its recommendations. Attached are the specific answers to the questions
posed in our charge. We wish to express our sincere appreciation for the valuable
contribution of our five ex-officio work group members, as well as the EPA staff that
provided valuable support to this effort: Noha Gaber, Environmental Engineer, US EPA;
Lara Autry, Senior Advisor for Measurement, Monitoring, and Laboratory Science, US
EPA; and Gerry Laniak, Environmental Engineer, US EPA. We appreciate the
opportunity to work on this important topic and offer any additional advice that you may
require in the future.
Sincerely,
/Signed/
John L. Howard, Jr.
NACEPT Chair
Attachment
cc: Arleen O'Donnell, Workgroup Chair
Marcus Peacock, Deputy Administrator
Charles Ingebretson, Chief of Staff
Ray Spears, Deputy Chief of Staff
George Gray, Assistant Administrator for Research and Development
Pai-Yei Whung, Chief Scientist
Rafael Deleon, Director, Office of Cooperative Environmental Management
Sonia Altieri, NACEPT Designated Federal Officer
-------
RESPONSE TO CHARGE QUESTIONS
In developing responses to the charge questions, four key themes resonated throughout
the White Paper and helped focus NACEPT's discussions: (1) identifying multiple
perspectives on integrated modeling; (2) automating the process for transporting data
from sources to models and between models; (3) managing uncertainty; and (4) building
a modeling community of practice. It is important to review these themes here to provide
context for NACEPT's response to the questions posed.
Identifying Multiple Perspectives on Integrated Modeling
Environmental systems are inherently linked through an intricate web of processes and
feedbacks. Integrated modeling can be defined as a continuum of increasing complexity:
conceptual integration of existing models (without feedbacks), full coupling of existing
models (with feedbacks), and integrated system models (developing fully-coupled
discrete models). Institutional integration also is important to facilitate communication
and coordination among the disciplines.
Automating the process for transporting data from sources to models and among
models
A key source of delay in the use of integrated modeling (IM) is the need to move large
amounts of data from diverse, often incompatible databases into a large environmental
modeling system. When an integrated modeling system has been developed by linking
models, there also is a need to pass output data from one model to another model as input
in a compatible form. Much of the delay and opportunities for error can be reduced
substantially by harmonizing and automating the process by which data are collected and
transferred. The water quality exchange (WQX) effort for EPA Office of Water's
STORET database is a good example of how data collection and exchange can be
facilitated and improved.
Managing uncertainty
Multiple sources of uncertainty arise in modeling systems and can be a problem for
managing IM systems. Sources of uncertainty include variation in measured data,
incompatibility of scales (spatial, temporal, or functional) among model components,
disagreement between alternate sources of information, under-determined model
parameterization (i.e., more unknown coefficients than stated variables), natural
heterogeneity, the selection of one model over another, simplifications of model
structure, extrapolation (prediction) errors, and value judgments. Uncertainty in IM can
compound that of simple models. When one links two models together, the uncertainty in
the integrated model can exceed that in either model alone. Choosing the appropriate
model complexity to suit the problem definition and the available data resources is
crucial. If the model is too simple, it will not answer the management questions and
therefore will have low utility. If the model is too complex, it likely will have insufficient
-------
data for calibration and confirmation, leading to a level of uncertainty that is not
acceptable, thus having low utility for decision support.
Building a community of practice (CP)
A community of practice is an organizational concept in which practitioners share
knowledge, leverage experiences, and improve individual and collective capacity to
contribute to the success of the organizations with which they are affiliated. A
community of practice may help to address such issues as managing uncertainty, how to
apply IM through single and linked systems models, and how to make science and
technology transparent to decision makers. Building a community of practice will help
institutionalize skill sets and knowledge bases, including advice on how to avoid
misapplying models that might work in one situation but not another.
1. Does NACEPT agree with the White Paper's focus on the importance of
integrated modeling to inform EPA decision making?
NACEPT agrees that integrated modeling is an important component of EPA decision
making. However, follow up work is needed to make decisions on how it can be better
utilized within EPA to solve environmental problems. The importance of IM could be
better understood if the range of IM was articulated with examples of appropriate
applications to ensure a common understanding of the forms of integrated modeling and
their use. A concern shared by NACEPT and the authors of the White Paper is that some
readers will assume that linking together a set of established and widely evaluated models
should provide a reliable IM system. However, integrated modeling is only useful if the
model is appropriately matched to the problem it is intended to solve.
Another issue in need of further elaboration is the difference between scientific (research)
and regulatory models. Research modeling seeks to advance the state of the science by
identifying knowledge or data gaps, by posing and testing hypotheses, and by generally
investigating how processes function and interact in environmental systems. Regulatory
models use the best practical science, requiring some compromises or simplifications to
be made for the models to be useful for regulatory purposes. When there is incomplete
information, regulators often must make decisions based on conservative assumptions,
default options, or worst-case scenarios. In other words, regulatory models should be
using the "best practical science".
2. Does the White Paper make a compelling case for the need to implement an
organization-level solution to promote consistency and repeatability of
integrated modeling analyses?
NACEPT believes that the White Paper makes a compelling case for implementing an
organization-level solution to promote consistency and repeatability of IM. A community
of practice will go a long way towards providing consistency. The most effective and
efficient organizational EPA structure within which IM can thrive is less clear.
Organizational options could be explored by piloting a few complex IM applications and
-------
letting "form follow function" (and, if a more formal process is desired, the Kaizen
Method could be employed). Good pilot candidates would be cross-media issues
involving a systems approach to problem solving such as:
• Biofuels: Examine full life cycle costs based on impacts to air quality, climate
change, water resources, and energy.
• Gulf of Mexico hypoxia: Examine atmospheric and land-based nutrient inputs
and their fate and transport characteristics, and develop hypoxia reduction
scenarios based on the optimization of costs and benefits of various control
strategies.
• Community health/comprehensive environmental assessment: Assess multiple
stressors on community health (EPA/CDC joint project) or an ecosystem such as
the Great Lakes, and model various outcomes based on different control
strategies.
There is general agreement that a CP is needed to formulate guidelines to share,
disseminate, and ensure proper use of models that EPA provides or has developed. For
example, a portfolio of archetypal models (a form of "generic" models) might be
considered with respect to a CP. For EPA's modeling work to be made more broadly
available, community practice guidelines such as precautionary notes regarding
adjustments that are needed for accurate application would be instructive to the user.
No group of experts can offer one set of rules about the correct modeling approach. The
knowledge needed to determine the best approach in each case must be built through a
CP, which is tied into the science as well as the modeling process. Much of the
infrastructure needed for the CP is forming: CREM, EPA's internal modeling program,
is building a community and culture. This is important because the group can provide
continuity and technical knowledge and become a repository for case studies to help
create guidelines for understanding capabilities.
Most environmental organizations are still stove-piped based on media or management
focus (e.g., toxics modelers versus eutrophication modelers). The community of practice
concept could be effective in helping modelers come together to solve what are
increasingly cross media problems. EPA should define a framework that establishes a
community of modelers or interdisciplinary science teams and builds a community that
serves as a reservoir of knowledge and skills on how to approach model design. The
Agency needs an institutional memory, and to develop one EPA needs to think about
building a cohesive group of people with problem solving skill sets. Examples follow:
• The CREM has been building a Models Knowledge Base on the web that could be
continued and enhanced and integrated with the CP concept.
• The Science Policy Council (SPC), under the OSA, consists of senior-level
officials from the different EPA program and regional offices, and 'examines
cross-cutting issues in science and technology. This group makes
-------
recommendations to the Science Advisor, and could be more involved in the
discussions on IM. The SPC is in a unique position because its membership
includes senior EPA staff members, many of whom have moved among different
program offices and up the chain of command. These people may be in the best
position to understand how useful IM could be in the various aspects of priority
setting and problem solving. The EPA Administrator could refer this advice letter
to the SPC, and that the SPC could report back to the Administrator with
recommendations. It would be useful to apply a pilot project such as bio fuels IM
using the SPC as a forum in which to discuss the model development and
application, and as a source of recommendations for organizational options to
support IM within EPA.
• At the National Aeronautics and Space Administration (NASA), new science is
enabled through satellite missions; forecasting models are created and handed off
to the National Weather Service (NWS). These are adapted to various regions.
There is always a transition when attempting to add the new state-of-the-science
developments that will improve the forecast into complicated forecast models that
have developed skill in their particular regions. This is done in major releases
rather than sharing a continuous model, and there is a transfer of science and
technology between developers and users. At EPA, this approach can be applied
by having the research team develop an integrated model and at the point when
there is confidence in its ability to predict, it would be handed off and tuned per
region as groups become familiar with the model and adapt it for their localities.
When the science team creates another major release, they need to work with the
applied community to determine when they have an improvement, and how to
understand the nature of the improvement's impact on regulations. The
developers also will ensure that the models are applied to the problem
appropnately at the regional level, and incorporate feedback from the regions into
model refinement. Localized knowledge can be added to each model. EPA should
examine the NASA/NWS modeling process, and perhaps the SPC could be the
coordinating body if the NASA/NWS approach is adopted at EPA.
3. Is the concept of integrated modeling clearly captured in terms of its meaning
and its emerging role in EPA and greater scientific community?
In general, the White Paper adequately captures the meaning and emerging role of IM,
and is quite forward thinking in its approach; however, the White Paper could benefit
from more in-depth analysis of how IM could be effectively implemented. For example,
most problems require site-specific model adjustments determined on a case-by-case
basis to address specific management issues, differences in environmental system
properties, time constraints, and other factors. A set of pre-packaged models cannot be
pulled off the shelf and linked to address any question. NACEPT shares this concern with
the authors of the White Paper. The Agency, through the CREM, has already drafted a
guidance document for regulatory model development and application; this document
also applies to integrated modeling.
-------
Another important issue is the assessment of whether and how to include feedback in
integrated models. Feedback between models and within models, where appropriate
based on the ecosystem being modeled, is an ideal, desirable property of an integrated
modeling system. In practice, however, it is not always feasible to implement complete
feedback. The Chesapeake Bay Program (CBP) is an example of an integrated modeling
application that features various degrees of feedback at multiple levels of model
integration. At the topmost, whole-system level, CBP links the airshed, watershed, and
estuary water quality models in a linear (sequential) pattern with little or no feedbacks.
Each of these component models is fully integrated in itself, however, with complete
feedbacks built in. For example, in the estuary water quality model, the water column
production submodel provides organic matter deposition to the sediment submodel,
which in turn exerts sediment oxygen demand on, and provides nutrient fluxes back to,
the water column. Even the water column submodel includes feedbacks representing
nutrient-food chain interactions. Thus, at the component model and submodel levels, the
CBP modeling system exhibits appropriate feedback. The panel recommends that careful
consideration be given to the need for feedback processes and to the formulation of those
processes in the development of integrated modeling systems.
4. Are the challenges associated with assimilating and applying integrated
modeling methods and technologies complete and accurate?
Although we recognize that integrated modeling is an investment in better science, it is
not without challenges. The White Paper presents three broad categories of challenges
associated with IM: science, technology, and institutional. Science challenges include
design of the integrated analytical framework, development of a better, quantitative
understanding of processes in environmental systems that integrate heretofore separate
environmental problem domains, model evaluation and uncertainty analysis, education,
and knowledge management. Technology challenges include achieving interoperability;
automated data access, retrieval, and processing; and the development of decision support
interfaces. Institutional challenges include organizational stove-piping and an imminent
shortfall in human capacity.
As noted in the forward to the White Paper, EPA's challenge is difficult: to produce
scientifically compelling and pragmatic linkages between fate and transport, human
exposure, risk, ecological effects, and/or economic models in which few such
connections currently exist. This requires an understanding of the way all of the
components fit and how to optimize system performance and characterize resultant
uncertainty. This is a tremendous challenge worthy of our concerted effort.
The Executive Summary of the White Paper notes that "the major challenge facing
environmental protection today is to eschew the single-chemical, single-medium, single-
pathway view that existing environmental statues have fostered, in favor of a view of
organisms and the environment as they really are: an integrated whole. Yet the
realization of the benefits of this approach is hampered by a number of challenges,
including the lack of a community-wide interchange and consistency and harmonization
across these various efforts."
-------
The White Paper also notes "the disconnect between the manner in which EPA programs
are organized, and the fluid interconnectedness among pollutants and their environmental
consequences across all media. This dichotomy presents a significant challenge for
scientists and decision makers to develop effective and efficient environmental protection
strategies and to understand and communicate environmental outcomes". Our
recommendations may help to address some of these challenges.
5. Are the recommendations for advancing awareness, appreciation, and
application of integrated modeling effective? Will they be sufficient to overcome
the challenges? Are there other recommendations you would suggest?
NACEPT believes that the White Paper's recommendations are thoughtful and likely to
be effective, but they will not be sufficient to overcome the challenges without adequate
funding, higher visibility within EPA, and a concerted effort to sort out the next level of
inquiry on how IM is most effectively implemented. A very important and early science
challenge - developing a quantitative understanding for the processes that require
integrated multi-media and multi-disciplinary modeling - has not been adequately
addressed in the White Paper's recommendations. NACEPT believes that EPA should
consider supporting a research program to address this critical science challenge to ensure
successful IM implementation. By itself, a community of practice will not be able to meet
this challenge.
In addition, NACEPT firmly believes that the following actions are key components to
the success of integrated modeling within EPA:
• An evaluation should be made of the up-front investment needed to implement the
White Paper's recommendations;
• Within a short period of time, a community of practice should be established and
all modelers within EPA should be expected to follow those practices;
• Integrated modeling should be showcased, and organizational issues
simultaneously addressed, by applying integrated modeling to a high profile,
complex, cross media issue such as bio fuels or the Gulf of Mexico hypoxia. One
or two such "case studies" can be the focal point for establishing protocols,
examining organizational relationships and identifying key stumbling blocks
along the way. Scientists and modelers with cross-media perspectives and
problem solving skills from across offices within EPA should be pulled together
work on this, and they should work with the Science Policy Council to evaluate
the process in "real time."
10
-------
NACEPT Integrated Modeling (IM) Workgroup Members
NA CEPT Members
Mr. Joel Bolstein
Environmental Law Practice Group
Fox Rothschild LLP
Mr. Robert Gruenig
National Tribal Environmental Council
Mr. Stan Laskowski
Masters of Environmental Studies
Program
University of Pennsylvania
Mr. Erik Meyers
Sustainable Programs
The Conservation Fund
Ms. Arleen O'Donnell
IM Workgroup Chair
Massachusetts Environmental Trust
Ms. Jennifer Nash
Product Stewardship Institute, Inc.
Dr. Bradley Smith
Huxley College of the Environment
Western Washington University
Dr. Dan Watts
York Center for Environmental
Engineering and Science
New Jersey Institute of Technology
Supplemental Members
Dr. Joseph C. Coughlan
Intelligent Systems Division
NASA/ARC
Dr. Joseph V. DePinto
LimnoTech
Dr. Jon Goodall
Civil & Environmental Engineering
University of South Carolina
Dr. Thomas E. McKone
Indoor Environment Department
Lawrence Berkeley National Laboratory
Dr. Olufemi Osidele
Geosciences & Engineering Division
Southwest Research Institute
NACEPT Chair
Mr. John Howard
Vinson and Elkins, LLP
EPA
Ms. Sonia Altieri
NACEPT Designated Federal Officer
Office of Cooperative Environmental
Management
Ms. Lara P. Autry
IM Workgroup Designated Federal
Officer
Office of the Science Advisor
Dr. Noha Gaber
Workgroup Liaison
Office of the Science Advisor
11
------- |