March 15, 1996
EPA-SAB-RSAC-LTR-96-004
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M. Street, SW
Washington, DC 20460
Subject: Review of the Strategic Plan for the Office of Research and
Development by the Research Strategies Advisory Committee
(RSAC) of the Science Advisory Board
Dear Ms. Browner:
In December 1995, the Agency submitted its draft Strategic Plan for the Office of
Research and Development (The Plan) to the Research Strategies Advisory Committee (RSAC)
of the Science Advisory Board (SAB). The Agency requested a review of the Plan on an
expedited schedule. To facilitate the review, several members of the Committee held a planning
teleconference on December 18, 1995 to clarify issues related to the review and the Plan itself.
As a result of that discussion, the Agency produced a set of issues and questions for the RSAC to
consider. These issues and questions form the Charge to the Committee (see below).
On February 23, 1996, the RSAC held a public teleconference in Washington, DC to
review the Plan. At that meeting, RSAC provided individual written and oral comments on the
Charge and the Plan to the Office of Research and Development (ORD). Following the
teleconference, the RSAC developed this brief, formal Committee report outlining the major
points of consensus regarding the issues raised in the Charge.
The Charge to the Committee is as follows: (a) Strengths of the Plan - What aspects or
elements of the Plan does RSAC find to be particularly useful, important, and/or worthy of
Agency attention and support? (We have also identified weaknesses of the Plan); (b) Strategic
Message - The Plan lays out a set of Strategic Principles, a Mission Statement, a set of Goals and
Objectives, and a Management Process designed to establish priorities and translate priorities into
effective programs. Are these appropriate for ORD as they are defined in the Plan?; (c) Clarity -
Is the Strategy, especially the process for setting priorities, clear?; (d) Criteria for Priorities - Are
the criteria identified for setting priorities appropriate and useable? Would RSAC like to help
refine them now or in the future?; and (e) Utility of the Plan - Does the Plan offer a potentially
useful roadmap for decision making and policy framework for managing ORD's research and
development programs?
-------
The general sense of RSAC was that the Plan represents a monumental undertaking and an
important step forward. We congratulate ORD and EPA for producing a well-written document
that responds to advice given to the Agency by other external review groups, such as the SAB
and the National Academy of Sciences. The Plan clearly states the vision and mission of ORD,
articulates the principles underlying EPA research, delineates long and short term research goals,
and presents criteria for priority setting. The existence of the Plan, coupled with the desire of the
Agency, and specifically ORD management, to implement it, will provide ORD with much needed
guidance for setting its immediate and future research agenda. In the following paragraphs, we
have provided our responses to the questions raised in the Charge. We have also voiced our
concerns about specific parts of the Plan, and where possible, we have offered recommendations
that we believe will further strengthen the Plan. This letter report does not contain minor
comments or editorial suggestions - such responses are contained in the individual written and
oral responses submitted to the Agency as part of the February 23rd teleconference.
l.a. Strengths of the Plan - The EPA ORD staff has developed a well written and well
thought out document that successfully articulates the ORD Strategic Plan. It contains the
critical elements that should be included in a strategic plan and addresses all of the
elements in a comprehensive manner. The Plan clearly delineates an implementation
approach and clearly shows the relationships between vision and mission and long term
goals and objectives. The Plan concludes by describing six high priority areas of research
that will form the focus of future ORD activities and, of equal importance, identifies topics
that ORD will not address. The planning process has actively sought out and brought in
the important customers for the ORD work (i.e., program offices and regional offices),
integrating them into the planning process. The Plan is built around the risk
characterization/risk management paradigm, as was suggested in the past by various
review bodies, and includes communication as a necessary step. Associated changes in the
organizational structure and operation of ORD which also follow this paradigm will
greatly aid in the implementation of the Plan. The Plan provides a means to coordinate
and integrate research within EPA, which in turn should lead to substantial resource
savings and more effective environmental protection strategies.
l.b. Weaknesses of the Plan -
(1) Although the use of the risk characterization paradigm is perceived as a
strength of the Plan, it also constitutes a major limitation by focusing research on issues
that constitute known, existing risks. The Plan may not have the ability to detect
emerging environmental problems, integrate new approaches and technologies, and
address prevention of environmental problems. Further consideration of how to capture
these important anticipatory issues in the strategic planning process may be beneficial.
(2) The success of the Plan ultimately will be judged by its ability to produce high
quality research of importance for the Agency's missions. Therefore, it would be
beneficial to incorporate in it an explicit mechanism for evaluating the success of the Plan
in achieving its stated goals. Some form of review on a continuing basis to provide
-------
feedback as to how the Plan is working and how it could be improved should be included.
(3) The Plan would be enhanced by giving attention to linkages with other
planning activities within EPA (such as the Agency-wide Strategic Plan), other agencies in
the United States, international activities, and the issue of global sustainability.
(4) The Plan needs to address more specifically how ORD plans to provide
national leadership with regard to emerging environmental issues (Mission statement #3).
Given the small budget of ORD and the very limited travel budget of ORD scientists and
engineers, how will the Agency determine what other research activities are underway,
both nationally and internationally, on the environmental issues and then position itself to
take the leadership role?
(5) The Agency needs to address the development of scientific and engineering
core capabilities and indicate how it will support the development of first class scientific
talent. While EPA does have some first class scientific talent, the Plan is not as effective
in articulating the process whereby EPA will identify the kinds of expertise that will be
needed over the long term, as it is in articulating the scientific and research issues.
(6) The Plan needs to take care to define "risk"; in some places it is not clear what
is meant, i.e., risk to the population, risk to the maximally exposed or most sensitive
individual?
(7) The terminology used in Figures 1 & 2 of the Plan specify obtaining "dose
response" information as part of the impact assessment. Since there are many instances
where it may not be possible to obtain dose response information, other approaches (such
as those indicated in Appendix A) will also have to be used.
2. Strategic Message - The Strategic Principles, the Mission Statement, the set of Goals
and Objectives and the Management Process for establishing priorities are very
appropriate and should provide a sound basis for setting research priorities. Mr.
Alexander's comments at the meeting that the Plan builds upon the culture and approaches
that have been developed within EPA are very appropriate here. A strength of the Plan is,
again, that it is based upon a risk-based analysis. Some criticisms and suggestions on the
strategic message are as follows: (a) The attempt to integrate health and environmental
risks is admirable, but may prove to be difficult, if not impossible to implement, (b) The
Plan needs to provide a better approach for how emerging issues will be identified and
how success in addressing emerging issues will be measured. While the Plan emphasizes
the critical need to be proactive, rather than being completely reactive to short term
program needs, the plan could be improved by better defining the methods to be used in
becoming more proactive. It is not clear how projects classified as exploratory research
will be handled; in any research program there is a need to provide for less targeted
projects that may anticipate future problems. The near term research priorities (Appendix
A) clearly demonstrate some long range thinking, but the mechanics of how these
priorities were derived is not clear. Figure 2 indicates that the process begins with a
public policy or initiating event, which is a reactive mode. Figure 4 indicates that the
customers will identify research topics. Again this process appears to be reactive. Other
methods of bringing in topics such as "look out panels", SAB reducing risk type activities,
-------
researcher initiated ideas, and others such as interagency review meetings could he
employed. In addition, the metric chosen for success for the emerging technologies is the
"public as the ultimate judge". The reliance on public opinion as an indicator of far
reaching environmental issues does not seem to be appropriate. Peer Review from outside
sources and the SAB may be a better approach, (c) Goals 3 (managing risk) and 4
(evaluating risk) should be given in reverse order to be consistent with the sequence of the
process.
3. Clarity - The document as a whole, not just the strategy and process for setting
priorities, is very clearly written. Using two columns, having many side boxes, and the
graphics all contribute to a clear understanding of what the Strategic Plan is about. In
fact, the process by which this document was created should be a model for other
organizational elements of EPA as they grapple with strategic planning.
4. Criteria for Priorities - Although the criteria seem clear, it is not readily apparent how
they were applied to produce the list of six near-term research priorities. This is
particularly of concern because the six issues differ greatly in scope. For example,
"endocrine disrupters" is a very specific topic, whereas "ecosystem protection" covers an
enormous range of issues. In fact, little sense of what research is to be done on ecosystem
protection or human health effects can be gleaned from the document. It would be helpful
to see a matrix containing the six topics and the major priorities, as well as some of the
topics that were considered but fell outside of the top six. A related issue pertains to the
absence of any connection between this prioritization process and the budget for ORD.
How further prioritization within the six identified issues will be achieved is not clear; this
is particularly important in an era of fluctuating and decreasing budgets.
The Strategic Plan should better define the methods used to determine how ORD
can make a significant contribution. It is not clear from the discussion relative to Figure 4
how ORD makes the decision about whether it can play a significant role. Are other
government agency activities evaluated relative to the issue? Are ORD's existing core
competencies and funding levels the determining factor? If the criteria used in evaluation
indicate that this issue is particularly important but the Agency doesn't have the expertise
or money to address it, then the Agency should use this information to secure additional
expertise and funds. Rejecting a topic (or hoping that the program offices address the
issue) may not be appropriate in all cases.
While the criteria are appropriate, it is not clear how well the criteria will be used,
and whether management can develop new criteria or modify existing criteria as the need
arises. It would be appropriate if these criteria were reviewed on a regular basis (e.g.,
every two years or so) by an independent peer review group, such as RSAC.
5. Utility of the Plan - The utility of the Plan depends upon the skill and determination of
those who manage ORD. Nevertheless, the fact that ORD has such a Plan in the first
-------
place is a very positive and encouraging step. The credibility and communication of the
Plan are only raised in relation to particular programs, but they need to be discussed in the
strategic implementation of the plan. Although the steps for translating the Plan into a
research program are laid out on pages 11-12 of the Plan, it would be useful to remind the
reader on page 23 of the Plan what these next steps are. This issue gets to one problem in
that the Plan, although well written and clearly presented, is rather complex. One Member
noted that her reading of the material was broken up by frequent reference to useful
figures and tables, and she did not leave the document with the impression of where the
next steps are. Perhaps this could be achieved by a short epilogue following the long-term
goals.
The Plan needs a "close-out" mechanism for programs and activities that have been
completed. These are the procedures by which ORD determines that an issue should not
receive continued attention, either because the risks are determined not to be substantial or
else sufficiently well understood. Such disinvestments are often difficult, and a clear
delineation of how this will be accomplished is crucial to the overall success of the
planning process. The utility of the Plan would be significantly aided by the development
of improved management systems for fiscal and technical information. This is addressed in
a very limited way on page 14 of the Plan in the "data tracking system". In the past, the
RSAC has been frustrated during reviews of the ORD budget by the inability to obtain
usable data regarding expenditures and progress in technical program areas. We suggest
that the Plan would be more informative if it included in Appendix B a summary listing of
the current expertise of the EPA staff (e.g., 30 ecologists, 5 epidemiologists, etc.).
We appreciate the opportunity to review and provide advice on ORD's Strategic Plan.
The Committee would be pleased to continue its review of the Plan and to help to refine the
criteria for setting priorities. We look forward to your response to the advice contained herein.
Sincerely,
~M. ^
Dr. Genevieve M. Matanoski, Chair Dr. Margaret Kripke, Chair
Executive Committee Research Strategies Advisory
Committee
-------
NOTICE
This report has been written as part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related
to problems facing the Agency. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or
commercial products constitute a recommendation for use.
-------
US ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
RESEARCH STRATEGIES ADVISORY COMMITTEE
CHAIR
Dr. Margaret Kripke, Chair, Department of Immunology, M. D. Anderson Cancer Center,
Houston, TX
MEMBERS
Dr. Judy A. Bean, Professor and Director of Biostatistics, University of Miami, Miami, FL
Dr. Stephen L. Brown, R2C2 Risks of Radiation and Chemical Compounds, Oakland, CA
Dr. Edwin L. Cooper, Department of Anatomy and Cell Biology, School of Medicine, UCLA,
Los Angeles, CA
Dr. Theo Colborn, Director, Wildlife and Contaminants Project, World Wildlife Fund,
Washington, DC
Dr. Kenny S. Crump, Ruston, LA
Dr. Joan M. Daisey, Indoor Environment Program, Lawrence Berkeley Laboratory, Berkeley,
CA
Dr. Virginia Dale, Environmental Sciences Division, Oak Ridge National Laboratory, Oak
Ridge, TN
Dr. Richard Kimerle, Monsanto Corporation, St. Louis, MI
Dr. Paulette Middleton, Director, Science & Policy Associates, Inc., Boulder, CO
Dr. W. Randall Seeker, Senior Vice President, Energy & Environmental Research Corp., Irvine,
CA
DESIGNATED FEDERAL OFFICER
Mr. A. Robert Flaak, US Environmental Protection Agency, Science Advisory Board, 401 M
Street, S.W., 1400F, Washington, DC 20460
-------
DISTRIBUTION LIST
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Science, ORD
Director, Office of Science Policy, ORD
EPA Regional Administrators
EPA Laboratory Directors
EPA Headquarters Library
EPA Regional Libraries
EPA Laboratory Libraries
Library of Congress
National Technical Information Service
Congressional Research Service
------- |