\       UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
         8                     WASHINGTON, D C. 20460
July 17.2009
                                                                             Office of
                                                                        Cooprative Environmental
Karen Sheffer                                                                Management
ASRC' Management Services
EPA Headquarters Repository (Library)
MC 3404T
Washington, DC 20460

Dear Ms  Sheffer

Enclosed are copies of reports prepared by or for the National Advisory Council for
Environmental Policy and Technology (NACEPT) under the Federal Advisory Committee Act
(FACA)  NACEPT is managed by EPA's Office of Cooperative Environmental Management

    I. NACEPT Advice Letter NACEPT's Role (April 2009)
    2  NACEPT Report Hncoiu aging Regional Solutions to Sustaining Water Scctoi  Unlilu">
      (March 2009)
    3  NACEPT Report- Outlook for the EPA (March 2009)
    4  NACEPT Advice Letter'  EPA's Draft 2009-2014 Strategic Plan Change Document
      (December 2008)
    5  NACEPT Advice Letter to the Administrator on Biofucls (December 2008)
    6  NACEPT Review of EPA's Strategy for Improving Access 'I o Environmental
      Information (November 2008)
    7  NACEPT Advice Letter Integrated  Modeling (September 2008)

11'you have any questions, please contact me at 202-564-0243 or alticn soma(«}cpa gov

                                        Sincerely,
                                         onia AlTTeri"
                                        Designated Federal Officer
                                        NACEPT
                                Internet Address (URL) • rinp //www epa gov
         Recycled/Recyclable • Pnnied with Vegetable Oil Based Inks on 100% Postconsumer, Process Chlorine Free Recycled Paper

-------
                       N ACE PT
                       Shaping  the  Nation's Environmental Policy
       National Advisory Council for Environmental Policy and Technology
September 22, 2008

The Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460

Re: White Paper on  Integrated Modeling for Integrated Environmental  Decision
Making

Dear Administrator Johnson:

The U.S. Environmental Protection Agency (EPA) has requested the National  Advisory
Council for Environmental Policy and Technology's (NACEPT's) critical review of the
White Paper on "Integrated Modeling for Integrated Environmental Decision  Making"
(hereinafter White Paper).  In response to the charge, NACEPT formed a work group
consisting of eight NACEPT members and five external members, who were selected for
their experience in the field of environmental modeling (see attached list of workgroup
members and their affiliations). The workgroup conducted  its review over a  3 month
period, from late May to August 2008. This advice letter represents the consensus view
and recommendations of the work group and of the full NACEPT council.

NACEPT would  like to commend EPA  staff  for preparing  a  well-written  and
comprehensive White Paper on such a broad and complex topic as integrated modeling.
Our investigation resulted in general concurrence with the White Paper, and confirmation
that computational models are useful tools that play  an increasingly important role in the
decision-making process.   We  agree with  the  White Paper that  integrated  modeling
encompasses a broad range of approaches and configurations of models,  data and
assessment methods to describe and analyze complex environmental problems, often in a
multimedia and multidisciplinary  manner.  Furthermore,  as environmental issues that
require decision-making  become  more  complex,  the panel believes that integrated
modeling will become increasingly important to EPA.

NACEPT believes that the White Paper deserves serious consideration and follow-up by
senior  level EPA  managers.   EPA should  give high  priority to  the  systematic
development and deployment of integrated models to inform EPA decision-making. To a
large extent, the science and technology  are already available within and outside EPA;
what is  needed is strategically directed funding  and Agency-wide,  organized,  and
systematic implementation of the action plan outlined in the White Paper.

-------
NACEPT offers the following three findings and recommendations, which, generally,
support and reiterate points made in the White Paper:

Finding:

1.  Integrated modeling is a significant cross-cutting science and technology tool that is
    worthy of broad-based strategic review and support within  EPA.  EPA's  Science
    Policy Council may be the most appropriate internal coordination body to vet the role
    of integrated modeling and how best to ensure its most efficient and effective use.

Recommendations:

•   The EPA Administrator should direct the Science Policy Council or other appropriate
    body within EPA to analyze and offer its recommendations on the following:
    (1) Estimating the initial investment required to move toward the increased and
    coordinated use of integrated modeling;
    (2) Determining if the development and application of integrated models is suited to
    most regulatory timeframes;
    (3) Identifying a discrete number of  projects  that  demand  integrated modeling
    approaches and would lend themselves as potential case studies for  development of
    an integrated modeling organizational structure;
    (4) Evaluating the potential  effect  on  staffing,  budgeting,  and  environmental
    protection from using integrated modeling; and
    (5) Evaluating the importance of the coordinating function among the research teams
    developing integrated models and the  program and regional office staffs using those
    models.

•   The Science  Policy Council,  or other appropriate body, should provide its findings
    and recommendations  to the  EPA  Science Advisor. The Science  Advisor should
    collaborate with the EPA Deputy Administrator on priority setting and organizational
    and budgeting issues relating to additional efforts to develop  and use integrated
    modeling (IM).

Finding:

2.  The White Paper demonstrates the need for a community of practice (CP), which can
    serve  as  a   knowledge  base,  develop  and  disseminate   standard  operating
    procedures/protocols for integrated model  development and application, and provide
    means to link the modeling activities  of different offices. NACEPT wholeheartedly
    supports the  community of practice concept. A community of practice will reduce
    redundancy in model development, help  with model dissemination, and guard against
    misapplication of models. A community of practice will not,  however, address all
    organizational challenges including priority setting, budget allocation, cross-program
    collaboration, and stakeholder involvement.

-------
Reco mmendat ions:

•  NACEPT believes that addressing  organizational  challenges will be an  iterative
   process  and recommends  that  EPA  build  its IM  organizational  infrastructure
   organically  rather than superimpose  an artificial structure that may render IM
   dissemination more cumbersome. Consequently, NACEPT recommends that EPA
   initiate a pilot integrated modeling project, such as biofuels full life cycle assessment,
   using  the  Science Policy Council  as the  forum  in  which  to discuss  model
   development and  application, and  for formulation  of organizational/institutional
   recommendations for supporting EM within EPA.

•  NACEPT recommends that EPA exercise due diligence to guard against misuse and
   misapplication of all simple as well as integrated models. While we agree that EPA's
   modeling work should be made more  broadly available, a community of practice
   should help distinguish site- and case-specific models from more generic/evaluative
   models. This will ensure that models are effectively matched to the problems they are
   attempting to solve. EPA can employ the CP as a means to advance the establishment
   of IM both within and outside of the Agency. The CP should be broadly available to
   all users, inside and outside EPA.

•  NACEPT recommends that EPA support a research program aimed at developing a
   quantitative understanding for the processes that require integrated multi-media and
   multi-disciplinary modeling.

Finding:

3. Data flow, or more generally "information flow", within integrated modeling systems
   is critically important.

Recommendations:

•  EPA  should automate and standardize the process for accessing, retrieving, and
   processing data from  nationally  available sources (databases) for use in integrated
   models.  The exchange  of information  among  modeling  components  and the
   processing of model outputs should be standardized.  EPA should look for ways to
   streamline  the  model-building process by  adopting more uniform formats for
   handling and moving information between databases and models and adopting a
   common set of methods and software tools that will service a broad range of models
   with only minor adjustments. Such tools (or "integrated model utilities") will support
   data acquisition, input and output, visualization, pre- and post-processing, evaluation,
   decision support,  and  sensitivity and uncertainty analyses.  Close  coordination
   between EPA's  Office  of Research  and  Development (ORD)  and  Office of
   Environmental Information (OEI) will be important in addressing this issue.

In conclusion, NACEPT is generally supportive of the White Paper and hopes this review
provides useful feedback  to the Administrator in order to take the next  steps towards

-------
implementing its recommendations. Attached are the specific answers to the questions
posed in our charge.  We  wish  to express our sincere appreciation for the valuable
contribution of our five ex-officio work group members, as well as the EPA staff that
provided valuable support to this effort: Noha Gaber, Environmental Engineer, US EPA;
Lara Autry, Senior Advisor for Measurement, Monitoring, and Laboratory Science, US
EPA; and  Gerry  Laniak,  Environmental Engineer, US  EPA.  We  appreciate  the
opportunity to work on this important topic and offer any additional advice that you may
require in the future.
                                       Sincerely,

                                       /Signed/
                                       John L. Howard, Jr.
                                       NACEPT Chair
Attachment

cc:    Arleen O'Donnell, Workgroup Chair
       Marcus Peacock, Deputy Administrator
       Charles Ingebretson, Chief of Staff
       Ray Spears, Deputy Chief of Staff
       George Gray, Assistant Administrator for Research and Development
       Pai-Yei Whung, Chief Scientist
       Rafael Deleon, Director, Office of Cooperative Environmental Management
       Sonia Altieri, NACEPT Designated Federal Officer

-------
RESPONSE TO CHARGE QUESTIONS

In developing responses to the charge questions, four key themes resonated throughout
the White Paper and  helped  focus NACEPT's discussions:   (1) identifying multiple
perspectives on integrated modeling;  (2) automating  the process for transporting data
from sources to models and between models; (3) managing uncertainty; and (4) building
a modeling community of practice. It is important to review these themes here to provide
context for NACEPT's response to the questions posed.

Identifying Multiple Perspectives on Integrated Modeling

Environmental systems are inherently linked through an intricate web of processes and
feedbacks. Integrated modeling can be defined as a continuum of increasing complexity:
conceptual integration  of existing models (without feedbacks), full coupling of existing
models  (with  feedbacks), and integrated system models  (developing  fully-coupled
discrete models). Institutional integration also  is important to  facilitate communication
and coordination among the disciplines.

Automating the process for  transporting data from sources to models and among
models

A key source of delay in the use of integrated modeling (IM) is the need to move large
amounts of data from  diverse, often incompatible  databases into a large environmental
modeling system. When an integrated modeling system has been developed by linking
models, there also is a need to  pass output data from one model to another model as input
in a compatible form. Much  of the delay and opportunities for error can be  reduced
substantially by harmonizing and automating the process by which data are collected and
transferred. The water quality exchange (WQX) effort  for  EPA Office  of  Water's
STORET  database is  a good example of how data  collection and exchange can  be
facilitated  and improved.

Managing uncertainty

Multiple sources of uncertainty arise in modeling systems and can be a problem for
managing  IM  systems.  Sources  of uncertainty  include  variation in measured  data,
incompatibility  of scales (spatial,  temporal, or functional) among model  components,
disagreement between  alternate  sources  of  information,  under-determined  model
parameterization (i.e.,  more   unknown coefficients  than  stated  variables),  natural
heterogeneity,  the  selection  of one  model  over another, simplifications of model
structure, extrapolation (prediction) errors, and value judgments. Uncertainty in IM can
compound that of simple models. When one links two models together, the uncertainty in
the integrated model can exceed  that in either model alone.  Choosing the appropriate
model complexity to  suit the problem definition and the available data  resources is
crucial. If the model is  too simple, it will not answer the management questions and
therefore will have low utility.  If the model is too complex, it likely will have insufficient

-------
data for calibration  and confirmation, leading to a level  of uncertainty that  is not
acceptable, thus having low utility for decision support.

Building a community of practice (CP)

A community of practice  is  an organizational concept  in which practitioners share
knowledge, leverage experiences,  and improve individual  and collective capacity to
contribute  to  the success  of the  organizations  with  which they are  affiliated. A
community of practice may help  to address such issues as managing uncertainty, how to
apply IM through single and linked  systems models, and  how  to make science and
technology transparent  to decision  makers. Building a community of practice will help
institutionalize  skill  sets and knowledge bases,  including advice on how to avoid
misapplying models that might work in one situation but not another.

1.  Does NACEPT  agree with the White Paper's  focus  on the importance of
    integrated modeling to inform EPA decision making?

NACEPT agrees  that integrated  modeling is an important component of EPA decision
making. However, follow up work is needed to make decisions on how it can be better
utilized within EPA to solve  environmental problems. The importance of IM could be
better understood if the range of IM was  articulated with  examples of appropriate
applications to ensure a common understanding of the forms of integrated modeling and
their use. A concern shared by NACEPT and the authors of the  White Paper is that some
readers will assume that linking together a  set of established and widely evaluated models
should provide a reliable IM system. However,  integrated modeling is  only useful if the
model is appropriately matched to the problem it is intended to solve.

Another issue in need of further elaboration is the difference between scientific (research)
and regulatory models.  Research modeling seeks to advance the state  of the science by
identifying knowledge or data gaps, by posing and testing hypotheses, and by generally
investigating how processes function and interact in environmental systems. Regulatory
models use the best practical science, requiring some compromises or  simplifications to
be made for the models to be useful for regulatory purposes. When there is incomplete
information, regulators often  must  make decisions based on conservative assumptions,
default options, or worst-case scenarios. In other  words,  regulatory models should be
using the "best practical science".

2.  Does the White  Paper make  a compelling case for the need to implement an
    organization-level   solution  to  promote  consistency   and  repeatability  of
    integrated modeling analyses?

NACEPT believes that the White Paper makes a compelling case for implementing an
organization-level solution to promote consistency and repeatability of IM. A community
of practice will go a long way towards providing consistency. The most effective and
efficient  organizational EPA structure within which  IM can  thrive  is  less clear.
Organizational options could be explored by piloting a few complex IM applications and

-------
letting "form follow function" (and, if a more formal process  is desired, the Kaizen
Method could  be employed). Good pilot candidates would be  cross-media  issues
involving a systems approach to problem solving such as:

   •   Biofuels:  Examine full life cycle costs based on impacts to air quality, climate
       change, water resources, and energy.

   •   Gulf of Mexico hypoxia:   Examine atmospheric and land-based nutrient inputs
       and their  fate and  transport characteristics,  and  develop hypoxia  reduction
       scenarios based on  the optimization  of costs  and benefits of various control
       strategies.

   •   Community health/comprehensive environmental assessment:   Assess multiple
       stressors on community health (EPA/CDC joint project) or an ecosystem such as
       the  Great Lakes, and model various outcomes  based  on  different control
       strategies.

There  is general  agreement  that a CP is  needed to  formulate  guidelines  to  share,
disseminate, and ensure proper use of models  that EPA provides or has developed. For
example, a  portfolio  of  archetypal models  (a form  of  "generic" models)  might be
considered  with respect to a CP. For EPA's modeling work to be made more broadly
available,  community practice  guidelines  such  as  precautionary  notes  regarding
adjustments that are needed for accurate application would be instructive to the user.
No group of experts can offer one set of rules about the correct modeling approach. The
knowledge  needed to determine the best approach  in each case must be built  through a
CP, which is tied into the science  as  well  as  the  modeling process. Much of the
infrastructure needed for the CP is forming:  CREM, EPA's internal modeling program,
is  building  a community  and  culture. This is important because  the group can provide
continuity and technical knowledge and become a repository for case  studies to  help
create guidelines for understanding capabilities.

Most environmental organizations are still stove-piped based on  media or management
focus (e.g., toxics modelers versus eutrophication modelers). The community of practice
concept could  be effective in  helping  modelers come  together  to  solve  what  are
increasingly cross media  problems. EPA should define a  framework that establishes a
community of modelers or interdisciplinary science teams and builds a community that
serves  as a reservoir of knowledge and skills on how to  approach model design. The
Agency needs an  institutional memory, and to develop one EPA needs to think about
building a cohesive group of people with problem solving skill sets. Examples follow:

   •   The CREM has been building a Models Knowledge Base on the web that could be
       continued and enhanced and integrated with the CP concept.

   •   The Science  Policy Council  (SPC), under the OSA, consists  of senior-level
       officials from  the different EPA program  and regional  offices, and  'examines
       cross-cutting   issues  in  science  and   technology.  This   group  makes

-------
       recommendations to the Science Advisor, and could  be more involved in the
       discussions on IM. The SPC is in a unique position because its  membership
       includes senior EPA staff members, many of whom have  moved among different
       program offices and up the chain of command. These people may be in the best
       position to understand how useful IM could be in the various aspects of priority
       setting and problem solving. The EPA Administrator could refer this advice letter
       to  the  SPC, and that the SPC could report  back to the Administrator  with
       recommendations. It would be useful to apply a pilot project such as bio fuels IM
       using the  SPC  as  a  forum  in which  to  discuss the model  development and
       application, and as a source of recommendations for organizational options to
       support IM within EPA.

   •   At the National Aeronautics and Space Administration (NASA), new science is
       enabled through satellite missions; forecasting models are created and handed off
       to the National Weather Service (NWS). These are adapted to various regions.
       There is always a transition when attempting to add the new state-of-the-science
       developments that will improve the forecast into complicated forecast models that
       have developed skill  in their particular regions.  This is  done in major releases
       rather than sharing a continuous model, and there is  a  transfer of science and
       technology between developers and users. At EPA, this approach can be applied
       by having the research team develop an integrated model and at the point when
       there is confidence  in its ability to predict, it would be handed off and tuned per
       region as groups become familiar with the  model and adapt it for their localities.
       When the science team creates another major release, they need to work with the
       applied community to determine when they have an improvement, and how to
       understand  the  nature  of the improvement's  impact on  regulations.  The
       developers  also  will  ensure that the models  are  applied  to  the  problem
       appropnately at the regional level, and incorporate feedback from the regions into
       model refinement. Localized knowledge can be  added to each model. EPA should
       examine the NASA/NWS modeling process, and perhaps the  SPC could be the
       coordinating body if the NASA/NWS approach  is adopted at EPA.

3. Is the concept of integrated modeling  clearly captured in terms of its meaning
   and its emerging role in EPA and greater scientific community?

In general, the White Paper adequately captures the meaning and emerging role of IM,
and is  quite forward thinking in its  approach; however, the White Paper  could benefit
from more in-depth analysis of how IM could be effectively implemented. For example,
most problems require site-specific model  adjustments determined on a case-by-case
basis  to  address  specific  management  issues,  differences  in   environmental  system
properties, time constraints, and other factors.  A set of pre-packaged models cannot be
pulled off the shelf and linked to address any question. NACEPT  shares this concern with
the authors of the White Paper. The Agency, through  the CREM, has already drafted a
guidance document for regulatory model  development and application; this document
also applies to integrated modeling.

-------
Another important issue is the assessment of whether and how to include feedback in
integrated models. Feedback  between models  and within models, where  appropriate
based on the ecosystem being modeled, is an ideal, desirable property of an integrated
modeling system. In practice,  however, it  is not always feasible to implement complete
feedback. The Chesapeake Bay Program (CBP)  is an example of an integrated modeling
application that  features various degrees of  feedback  at multiple  levels of model
integration. At the topmost, whole-system level, CBP links the airshed, watershed, and
estuary water quality models in a linear (sequential) pattern with little or no feedbacks.
Each of these component models is fully integrated in  itself,  however, with complete
feedbacks built in. For example,  in the estuary water quality model, the  water column
production submodel  provides organic matter deposition to  the  sediment  submodel,
which in turn exerts sediment oxygen demand on, and provides nutrient fluxes back to,
the water column. Even the water column submodel includes feedbacks representing
nutrient-food chain interactions. Thus, at the component model and submodel levels, the
CBP modeling system exhibits appropriate feedback. The panel recommends that careful
consideration be given to the need for feedback processes and to the formulation of those
processes in the development of integrated  modeling systems.

4. Are  the  challenges associated  with  assimilating  and  applying  integrated
   modeling methods and technologies complete and accurate?

Although we recognize that integrated modeling is an investment in better science, it is
not without challenges. The White Paper  presents three broad categories of challenges
associated with IM: science, technology,  and institutional. Science challenges include
design of the integrated  analytical  framework, development  of a better, quantitative
understanding of processes  in environmental systems that integrate heretofore separate
environmental problem domains,  model evaluation and uncertainty analysis, education,
and knowledge management. Technology  challenges  include achieving interoperability;
automated data access, retrieval, and processing; and the development of decision support
interfaces. Institutional challenges include organizational stove-piping and an imminent
shortfall in human capacity.

As noted in the forward to  the White Paper, EPA's  challenge is difficult:  to produce
scientifically compelling  and pragmatic  linkages between fate and  transport,  human
exposure,  risk,  ecological  effects, and/or  economic  models  in  which  few  such
connections  currently exist. This requires an understanding of  the  way  all  of the
components fit  and how to optimize system  performance and characterize resultant
uncertainty. This is a tremendous challenge worthy of our concerted effort.

The Executive Summary of the  White Paper  notes that "the major challenge facing
environmental protection today is to eschew the single-chemical, single-medium, single-
pathway view that existing  environmental  statues have fostered, in favor of a view of
organisms and  the environment as they  really  are:   an integrated  whole. Yet the
realization of the benefits of this approach  is hampered by a number  of challenges,
including the lack of a community-wide interchange and consistency and  harmonization
across these various efforts."

-------
The White Paper also notes "the disconnect between the manner in which EPA programs
are organized, and the fluid interconnectedness among pollutants and their environmental
consequences across all  media. This dichotomy  presents a significant challenge  for
scientists and decision makers to develop effective and efficient environmental protection
strategies  and  to  understand  and  communicate environmental  outcomes".  Our
recommendations may help to address some of these challenges.

5.  Are  the  recommendations  for  advancing awareness,  appreciation,  and
    application of integrated modeling effective? Will they be sufficient to overcome
    the challenges? Are there other recommendations you would suggest?

NACEPT believes that the White Paper's recommendations are thoughtful and likely to
be effective, but they will not be sufficient to overcome the challenges without adequate
funding, higher visibility within EPA, and a concerted effort to sort out the next level of
inquiry on how IM is most effectively implemented. A very important and early science
challenge  - developing a quantitative  understanding for the processes  that require
integrated multi-media and multi-disciplinary modeling - has  not been adequately
addressed in the White Paper's recommendations. NACEPT believes that  EPA  should
consider supporting a research program to address this critical science challenge to ensure
successful IM implementation. By itself, a community of practice will not be able to meet
this challenge.

In addition, NACEPT firmly believes that the following actions are key components to
the success of integrated modeling within EPA:

    •  An evaluation should be made of the up-front investment needed to implement the
      White Paper's recommendations;

    •  Within a short period of time, a community of practice should be established and
      all modelers within EPA should be expected to follow those practices;

    •  Integrated   modeling   should  be   showcased,   and   organizational  issues
      simultaneously addressed,  by applying integrated modeling to  a  high profile,
      complex, cross media issue such as bio fuels or the Gulf of Mexico hypoxia. One
      or  two such "case  studies"  can  be  the  focal point for establishing protocols,
      examining organizational relationships and identifying  key stumbling  blocks
      along  the  way.  Scientists  and  modelers  with  cross-media  perspectives and
      problem solving skills from across offices within EPA should be pulled together
      work on this, and they should work with the Science Policy Council to evaluate
      the process in "real time."
                                                                               10

-------
NACEPT Integrated Modeling (IM) Workgroup Members
NA CEPT Members

Mr. Joel Bolstein
Environmental Law Practice Group
Fox Rothschild LLP

Mr. Robert Gruenig
National Tribal Environmental Council

Mr. Stan Laskowski
Masters of Environmental Studies
Program
University of Pennsylvania

Mr. Erik Meyers
Sustainable Programs
The Conservation Fund

Ms. Arleen O'Donnell
IM Workgroup Chair
Massachusetts Environmental Trust

Ms. Jennifer Nash
Product Stewardship Institute, Inc.

Dr. Bradley Smith
Huxley College of the Environment
Western Washington University

Dr. Dan Watts
York Center for Environmental
Engineering and Science
New Jersey Institute of Technology

Supplemental Members

Dr. Joseph C. Coughlan
Intelligent Systems Division
NASA/ARC

Dr. Joseph V. DePinto
LimnoTech
Dr. Jon Goodall
Civil & Environmental Engineering
University of South Carolina

Dr. Thomas E. McKone
Indoor Environment Department
Lawrence Berkeley National Laboratory

Dr. Olufemi Osidele
Geosciences & Engineering Division
Southwest Research Institute

NACEPT Chair

Mr. John Howard
Vinson and Elkins, LLP

EPA

Ms. Sonia Altieri
NACEPT Designated Federal Officer
Office of Cooperative Environmental
Management

Ms. Lara P. Autry
IM Workgroup Designated Federal
Officer
Office of the Science Advisor

Dr. Noha Gaber
Workgroup Liaison
Office of the Science Advisor
                                                                          11

-------