UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
September 17, 2007
EPA-SAB-07-012
Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
SUBJECT: Science Advisory Board's Homeland Security Advisory Committee Consultation
on the EPA's Emergency Consequence Assessment Tool and Incident-based
Microbial Risk Assessment Framework
Dear Administrator Johnson:
The Science Advisory Board (SAB) Homeland Security Advisory Committee (HSAC)
held a public meeting on May 30 and 31, 2007, to provide consultative advice on the Agency's
Emergency Consequence Assessment Tool (ECAT, September 2006) and the Draft White Paper
on Incident-based Microbial Risk Assessment Framework (MRA, May 2007).
The HSAC, augmented by additional experts from the SAB's Radiation Advisory
Committee and the Drinking Water Committee, is composed of a remarkably diverse and
accomplished group of experts. There was a tremendous amount of enthusiasm and energy
displayed by their willingness to serve and the intensity of their involvement is a tribute to the
Agency and the importance of the missions that it has undertaken for our nation's security.
The Agency has a long history of requesting early input from independent experts and the
SAB welcomes the opportunity to be part of that tradition. The HSAC was very impressed by
the hard and thoughtful work done by the Agency's scientists.
As this was a consultation regarding programs in their formative stage, there will be no
consensus report from the SAB. However, on behalf of the HSAC, we would like to note several
key points that arose in the consultation on these two topics. Written comments from individual
Committee members are provided in the official minutes of the consultation.
-------
Comments on EPA's Emergency Consequence Assessment Tool (ECAT)
EPA's National Homeland Security Research Center (NHSRC) within the Office of Research
and Development is developing an interactive on-line risk assessment software tool designed to
provide health advisors and other emergency response officials with rapid access to critical
information during an environmental emergency or training exercise. The EC AT is designed to
assess and provide site-specific numeric estimates of health risks for selected chemical,
biological and radiological threat agents; and identify which response actions might be
appropriate to mitigate human health risks. The SAB was asked to review the preliminary
version of this tool and provide advice and recommendations for its future development and
application.
We offer the following thoughts regarding the EC AT:
1. The ECAT could be useful for EPA's second phase responders, risk assessors, and risk
managers. It has particular promise as a training tool, if developed and evaluated according to
the appropriate scientific standards (found in educational assessments, human-computer
interactions, and related fields). However, its use by first responders, in the initial hours of an
emergency would not be feasible.
2. The next developmental phase for the ECAT should include one or two fully developed threat
scenarios. Those complete applications of the ECAT should be independently evaluated in terms
of their contributions to health protection. One scenario for air contamination and one for water
contamination would be good for examining generality. These demonstrations might use the
Human Exposure Measurement results from the Urban Dispersion Program tracer field studies
and drinking water system tracer studies.
3. For these applications, EPA should show how the ECAT's outputs will affect specific
decisions. Those demonstrations should identify the impacts of specific information, accessed at
specific times during a specific emergency, reaching specific decision makers, used in specific
decisions, disseminated to specific audiences, interpreted in specific ways, and leading to
specific protective actions. Evaluating the usefulness of information is an essential element in
sound decision making and risk communication.
4. EPA should study the challenges in using the ECAT with actual events. That research should
consider issues like choosing the right hazard with dissemination events (where an unknown
agent is quietly introduced), determining source terms for models, and communicating to diverse
audiences. The research should develop decision rules that consider the expected impacts of
possible diagnoses and misdiagnoses. The research should focus on the test cases.
5. EPA should explicitly evaluate the ECAT's potential usefulness before extending it to other
domains. That evaluation may conclude that some areas should be eliminated (e.g., because
usable models cannot be created), that some areas are only viable if they can use data sources
maintained by other organizations, that some areas can be used if their models are validated
using tracer studies, and that some areas should only provide access to consulting experts.
6. EPA should develop a dissemination plan for the ECAT, addressing issues of coordination,
cost, trust, liability, duplication, etc. That plan should be informed by the relevant science
-------
regarding organizational behavior, political science, and public administration. The plan should
consider selective release of the EC AT elements; for example, the collection of databases (in the
left-hand toolbar) may have particular value. Risks with unintended users (both friendly and
hostile) must be considered.
7. EPA needs to have a robust science program on risk communication. Scientifically sound risk
communication entails identifying the information most critical to users' needs and delivering it
in a demonstrably effective way. Poor communication can harm citizens, by undermining their
ability to protect themselves; it can harm organizations, by undermining public faith in them.
Without rigorously developed and evaluated communications, the EC AT may provide no value
or negative value. The markers of sound communication science are (a) familiarity with the
current research literature, (b) formal analysis of the information needs of specific decision
makers facing specific decisions, (c) empirical evaluation of communication impacts, and (d)
review by peers.
Comments on EPA's White Paper on Incident-based Microbial Risk Assessment Framework
EPA's NHRSC has prepared a white paper describing issues regarding the development of a
decision framework for assessing health risks associated with exposure to microbial agents after
an incident and developing cleanup levels associated with a decontamination response. The SAB
was asked to provide advice on the development of such a framework.
We offer the following thoughts on the draft white paper:
1. The document needs clear opening statements with its strategic goals and underlying
assumptions, along with concluding assessments of the sensitivity of its conclusions to those
assumptions and the limits to its scope.
2. The white paper covers broad topics in very general style. Little specific assessment
methodology was provided, thereby limiting the basis for comment. For example, its response
parameters were too general to elicit a meaningful exchange of ideas between the HSAC
members and the Agency scientists. When a more specific and detailed methodology is
established, a follow-up review by the HSAC would lead to a fruitful exchange of thoughts.
3. The white paper embodies a highly simplified view of crisis management. Much more
complex conditions are likely to exist and this plan must recognize them and be capable of
providing flexibility to address them. Because EPA will not be leading responses in the first 24
hours, it must consider local roles and objectives. In some cases, EPA may not play a direct role
in response for consequence management. Thus, clear process recommendations for use by
other regulatory entities are necessary.
4. As with any risk assessment, there are numerous limitations, uncertainties, and roadblocks
associated with the process. Such challenges should not be considered as insurmountable as
described in the document. Rather, the EPA should write the document in a "can do manner"
and deal with the limitations separately.
-------
5. The microbial risk assessment framework should be developed to give as quantitative a
measure of risk as possible, given the available data, just as one would employ when assessing
chemical or food safety (also facing data limits). Sources of uncertainty should be assessed as
part of risk characterization, in order to estimate the impacts of assumptions and defaults. The
report should consider the risks of decontamination strategy as well as agent risk.
6. Development of background data for biological contaminants is essential and will play a
central role in the development of cleanup benchmarks for various environmental settings.
Background data already play a comparable role for remediation of chemical contaminants,
particularly in complex environments like urban areas. Collection of background data must be
an important part of the overall research agendas of EPA and other federal agencies.
7. Performance assessment of analytical methods for environmental detection of microbial agents
is an area that should be given considerable attention by the EPA, however, it is not discussed in
the submitted white paper. This is a critical step that follows the implementation of the remedy
in the immediate-, short- and long-term.
General Comments
1. The HSAC could serve as a consultative body to the Agency as a whole in developing a
scientifically sound risk communication program. The HSAC has the three essential kinds of
scientific expertise: (a) domain knowledge, for many specific hazards; (b) risk and decision
analysis, for identifying decision-relevant information; and (c) social science, for developing and
empirically evaluating communications.
2. The HSAC could also assist the Agency in evaluating its overall homeland security research
program in order to identify knowledge gaps and to strengthen future programs.
-------
3. Much better feedback mechanisms are needed in order to take full advantage of HSAC
members' expertise and to maintain their commitment. Members offered their availability for
more frequent consultations, for the committee as a whole or subgroups.
Thank you for your attention and the opportunity to serve the Agency and its mission.
Sincerely,
/S/ /S/
Dr. Baruch Fischhoff, Chair Dr. Rebecca Parkin, Co-Chair
Homeland Security Advisory Committee Homeland Security Advisory Committee
/S/
Dr. Granger Morgan, Chair
Science Advisory Board
------- |