United States Science Advisory Board EPA-SAB-EEC-97-011
Environmental Washington DC September 1997
Protection Agency
4>EPA AN SAB REPORT: REVIEW
OF THE NATIONAL RISK
MANAGEMENT RESEARCH
LABORATORY (NRMRL)
PREPARED BY THE NRMRL
SUBCOMMITTEE OF THE
ENVIRONMENTAL ENGINEERING
COMMITTEE (EEC)
-------
September 30, 1997
EPA-SAB-EEC-97-011
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, SW
Washington, DC 20460
Subject: Review of the National Risk Management Research Laboratory
Dear Ms. Browner:
At the request of the Office of Research and Development (ORD), the Environmental
Engineering Committee (EEC) of the Science Advisory Board (SAB) conducted a review of
the Agency's National Risk Management Research Laboratory's (NRMRL) program. The
EEC's charge was to review NRMRL'S strategic directions and approach to research within
the context of both the 1995 realignment of ORD's organizational structure and its strategic
plan. The EEC review focused on:
a) Examining and critiquing the research programmatic directions such as
whether NRMRL is pursuing the most appropriate research problem areas;
b) Commenting on strategic directions, e.g. use of its core technical
competencies, transition from primarily extramural to an intramural R&D
organization, leveraging with other agencies and organization;
c) Reviewing and commenting on the effectiveness of NRMRL's approach to
science management, e.g. measures of success and science quality,
soundness of peer review process;
d) Examining and critiquing the relationship of NRMRL's risk management
research and its intended role in the risk assessment/risk management
paradigm; and
e) Reviewing and commenting on the strategic balance for the next decade
among pollution prevention, technology development, remediation, and risk
management assessment activities.
The EEC met September 25-27, 1996 at the National Risk Management Research
Laboratory in Cincinnati, Ohio to review the written materials provided and the presentations
made. A writing subcommittee was formed to prepare this report. The EEC approved the
NRMRL report June 23, 1997 and the Executive Committee approved the report on July 24,
1997, subject to vettor review and approval (which was obtained on September 30, 1997).
-------
The Committee notes that the NRMRL management diligently prepared for the review and
successfully communicated the strategic research and management plans for the
laboratory. Therefore, the NRMRL staff are commended for their attention to this review at
a time of considerable uncertainty in budgets, laboratory reorganization and the overall
strategic planning process at ORD.
Major findings and recommendations of this review are stated below:
a) The Committee concludes that both ORD and NRMRL have made significant
progress in research planning since the EEC's strategic research planning
commentary (SAB, 1994). It finds that NRMRL's plans are conceptually
consistent with ORD's "Strategic Plan". The "Strategic Plan", in turn, is
conceptually consistent with risk assessment/risk management paradigm
emphasized in the 1983 National Academy of Sciences publication "Risk
Assessment in the Federal Government: Managing the Process". Therefore,
the Committee concludes that NRMRL has in place a well-understood and
coherent intellectual framework for strategic research planning.
b) Regarding research program directions, the Committee concludes that the
twelve areas NRMRL has selected are appropriate. They come from the
ORD's 1996 Strategic Plan and are consistent with the laboratory's mission.
However, the decision-process could benefit from a focused statement of how
these topics were selected as opposed to others. Because priorities will
change overtime, as will resources, NRMRL needs to establish a
recognizable decision process detailing those factors that will lead to the
addition or subtraction of research elements as needed.
c) Regarding science management, NRMRL has developed certain key
elements such as measures of success, peer review and use of teams.
However, NRMRL will need to refine each element as the new laboratory
paradigm takes effect. Measures of success should be carefully chosen and
wisely used.
d) Regarding the utility of risk management research, the Committee commends
NRMRL for identifying factors that determine the value of the Agency's risk
management goals and the extent to which NRMRL's research can influence
or be influenced by them. Two research elements would enhance the
capacity of NRMRL to provide the evidence of risk reduction: (1) development
of compliance assurance models and methods; and (2) development of
measures of environmental and public health improvement.
e) There are opportunities for NRMRL to play a dominant leadership role in both
traditional and emerging research areas. Hence it will be prudent to assess
these opportunities and to define strategies responsive to EPA's increased
emphasis on community-based environmental decision making. For
example, drinking water research appears to rank high as a focus in
NRMRL's plans. However, its community impact may be more impressive if
-------
concentrated on infrastructure improvement rather than incremental
advances in treatment technologies alone.
f) The current de-emphasis on waste management in NRMRL's plans, structure
or vocabulary appears to be somewhat short-sighted, particularly when waste
management is and likely will remain a significant societal issue and one that
occupies the attention and energies of both the public and private sectors.
Although pollution prevention or waste avoidance is often considered superior
to the remediation approach implicit in most waste management programs, it
seems inappropriate to almost abandon the search for innovative solutions
within the research agenda of the Agency.
g) Core competencies in technology development and remediation within
NRMRL and linkages with similar research programs should be maintained
so that the menu of options for treatment and control can continue to expand.
Such linkages provide vital feedback mechanisms to the research
community, and serve as a reality check and a measure of overall success.
The Committee wishes to note that two important difficulties facing the NRMRL, also
face the rest of ORD. First, moving from a largely external research program to a largely
internal one is a great challenge in terms of staffing. The laboratory staff that has managed
research contracts in the past must now be transformed into scientists and technologists.
The question remains - how difficult is this challenge! Second, collaborative research is
only one of several means to make this transition. And collaborative research with outside
institutions is valuable in its own right. However, the current administrative barriers to doing
collaborative research are quite high. These barriers should be reduced as they have
become an impediment to conducting the right research the right way.
The Committee appreciates the opportunity to review NRMRL and looks forward to a
written response from the Assistant Administrator for ORD.
Sincerely,
/signed/
Dr. Genevieve M. Matanoski, Chair
Executive Committee
/signed/
Dr. Ishwar P. Murarka, Chair
Environmental Engineering Committee
/signed/
Dr. Frederick G. Pohland, Chair
NRMRL Subcommittee
-------
NOTICE
This report has been written as part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related
to problems facing the Agency. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or
commercial products constitute a recommendation for use.
-------
ABSTRACT
At the request of the Office of Research and Development (ORD), the
Environmental Engineering Committee (EEC) of the Science Advisory Board reviewed
(on September 25-27, 1996) the Agency's National Risk Management Research
Laboratory's (NRMRL) program. NRMRL's mission is to conduct research to reduce
uncertainties and costs associated with making and implementing environmental risk
management decisions. The EEC's charge was to address the following questions:
a) Examining and critiquing the research programmatic directions such as
whether NRMRL is pursuing the most appropriate research problem
areas;
b) Commenting on strategic directions, e.g. use of its core technical
competencies, transition from primarily extramural to an intramural R&D
organization, leveraging with other agencies and organization;
c) Reviewing and commenting on the effectiveness of NRMRL's approach to
science management, e.g. measures of success and science quality,
soundness of peer review process;
d) Examining and critiquing the relationship of NRMRL's risk management
research and its intended role in the risk assessment/risk management
paradigm; and
e) Reviewing and commenting on the strategic balance for the next decade
among pollution prevention, technology development, remediation, and
risk management assessment activities.
Key comments include: a) NRMRL has in place a well-understood and coherent
intellectual framework for strategic research planning; b) The twelve areas NRMRL has
selected seem appropriate and NRMRL will need a decision process leading to the
addition or subtraction of research elements as needed; c) NRMRL has developed key
science management which will need refinement as each element of the new laboratory
paradigm takes effect; and d) The Committee commends NRMRL for identifying factors
that determine the value of the Agency's risk management goals and the extent to
which NRMRL's research can influence or be influenced by them.
Keywords: research, technology, risk assessment, risk management, peer review,
community-based environmental decision-making
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee
NRMRL Subcommittee
June 1997
CHAIR
Dr. Frederick G. Pohland, Department of Civil and Environmental Engineering,
University of Pittsburgh, Pittsburgh, PA
MEMBERS
Dr. Edgar Berkey, Center for Hazardous Materials Research, The National
Environmental Technology Applications Center, Pittsburgh, PA
Dr. Hilary I. Inyang, Center for Environmental Engineering and Science Technologies
(CEEST), University of Massachusetts, Lowell, MA
Dr. Robert B. Pojasek, Cambridge Environmental, Inc., Cambridge, MA
Dr. Wm. Randall Seeker, Energy & Environmental Research Corp., Irvine, CA
SCIENCE ADVISORY BOARD STAFF
Mrs. Kathleen W. Conway, Designated Federal Official, U.S. EPA, Science Advisory
Board (1400), 401 M Street, SW, Washington, DC 20460
Mrs. Dorothy M. Clark, Staff Secretary, U.S. EPA, Science Advisory Board (1400), 401
M Street, SW, Washington, DC 20460
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee (FY97)
CHAIR
Dr. Ishwar P. Murarka, Environment Group, Electric Power Research Institute, Palo
Alto, CA
MEMBERS
Dr. Edgar Berkey, Center for Hazardous Materials Research, The National
Environmental Technology Applications Center, Pittsburgh, PA
Dr. Calvin C. Chien, Corporate Remediation, E. I. DuPont Company, Wilmington, DE
Dr. Nina Bergan French, SKY+ , Oakland, CA
Dr. Hilary I. Inyang, Center for Environmental Engineering and Science Technologies
(CEEST), University of Massachusetts, Lowell, MA
Dr. James H. Johnson, Jr., School of Engineering, Howard University, Washington, DC
Dr. Jo Ann Lighty, Department of Chemical and Fuels Engineering, University of Utah,
Salt Lake City, UT
Dr. John P. Maney, President, Environmental Measurements Assessment, Hamilton,
MA
Dr. James W. Mercer, GeoTrans, Inc., Sterling, VA
Ms. Lynne Preslo, Technical Programs, Earth Tech, Long Beach, CA
Dr. Wm. Randall Seeker, Energy & Environmental Research Corp., Irvine, CA
Science Advisory Board Staff
Mrs. Kathleen W. Conway, Designated Federal Official, U.S. EPA, Science Advisory
Board (1400), 401 M Street, SW., Washington, DC 20460
Mrs. Dorothy M. Clark, Staff Secretary, U.S. EPA, Science Advisory Board (1400), 401
M Street, SW., Washington, DC 20460
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 5
2.1 Background 5
2.2 Structure of the National Risk Management Research Laboratory 6
2.3 Environmental Engineering Committee Review of the National Risk
Management Research Laboratory with Charge 7
3. COMMENTS ON NRMRL'S STRATEGIC DIRECTIONS AND APPROACH 9
3.1 General Comments 9
3.2 Research Programmatic Directions 9
3.3 Strategic Directions 11
3.3.1 Core Competencies 11
3.4 Science Management 13
3.5 Utility of Risk Management Research 15
3.5.1 Beyond Engineering 15
3.5.2 Options for Research Organization 16
3.5.3 The Next Step 16
3.6 Strategic Balance for the Next Decade 17
REFERENCES R -1
APPENDIX A - MATERIALS REVIEWED A -1
-------
1. EXECUTIVE SUMMARY
At the request of the Office of Research and Development (ORD), the
Environmental Engineering Committee (EEC) of the Science Advisory Board reviewed
the Agency's National Risk Management Research Laboratory's (NRMRL) program.
The EEC formed a NRMRL Writing Subcommittee to prepare this report, which has
been approved by the EEC and by the Executive Committee of the SAB.
In the "Strategic Plan for the Office of Research and Development" (EPA,
1996a), ORD described the relationship of risk assessment to the risk management
process, and emphasized the need for scientific and engineering research to enable
sound risk management decisions and actions. Within the framework of that strategic
plan, NRMRL's mission is to conduct research to reduce uncertainties and costs
associated with making and implementing environmental risk management decisions.
NRMRL has therefore developed a research agenda to reduce risk uncertainty that
also focuses on those important, relevant issues where it can make a difference.
The EEC was charged with:
a) Examining and critiquing the research programmatic directions such as
whether NRMRL is pursuing the most appropriate research problem
areas;
b) Commenting on strategic directions, e.g. use of its core technical
competencies, transition from primarily extramural to an intramural R&D
organization, leveraging with other agencies and organization;
c) Reviewing and commenting on the effectiveness of NRMRL's approach to
science management, e.g. measures of success and science quality,
soundness of peer review process;
d) Examining and critiquing the relationship of NRMRL's risk management
research and its intended role in the risk assessment/risk management
paradigm; and
e) Reviewing and commenting on the strategic balance for the next decade
among pollution prevention, technology development, remediation, and
risk management assessment activities.
According to its 1996 strategic plan for research, ORD has determined that its
overall priorities for the next few years are: a) drinking water disinfection; b) particulate
matter; c) endocrine disrupters; d) ecosystem risk assessment; e) health risk
1
-------
assessment; and f) pollution prevention and new technologies.
Other areas of high importance include: a) air pollutants; b) indoor air; c) global
change; d) drinking water (in addition to disinfection issues); e) waste site
characterization; and f) waste management and site remediation.
NRMRL's research focuses on characterizing pollutant sources that require
management; and on identifying, developing, and evaluating cost-effective tools and
technologies for prevention, control, restoration, and remediation of environmental
contamination that presents high risk, high cost, or that lack effective management
alternatives (EPA, 1996a).
In general, the Committee concludes that ORD and NRMRL have made
significant progress since the EEC's Strategic Research Planning Commentary (SAB,
1994). The 1994 Commentary recommended that EPA adopt and implement a
consistent, reliable and comprehensive approach to strategic planning for EPA
research and development. The SAB's Research Strategies Advisory Committee
favorably reviewed ORD's "Strategic Plan for the Office of Research and Development"
(SAB, 1996). NRMRL's plans are conceptually consistent with ORD's "Strategic Plan".
The "Strategic Plan", in turn, is conceptually consistent with the widely cited "Risk
Assessment in the Federal Government: Managing the Process" (MAS, 1983).
Therefore, there is a well-understood and coherent intellectual framework in place for
NRMRL's decision-making.
NRMRL management diligently prepared and clearly communicated the
laboratory's strategic research and management plans. The participating personnel
uniformly supported the laboratory planning and management process. Therefore, the
NRMRL personnel are commended for their efforts at a time of considerable
uncertainty in budgets, laboratory reorganization and the overall strategic planning
process being implemented by ORD. NRMRL has made efforts to develop the
conceptual elements of risk management. NRMRL's efforts and the interim efforts of
the SAB's Risk Reduction Opportunities Subcommittee have been used to create
Figure 1 at the end of this report.
In responding to the charge, the Committee commented upon:
a) research programmatic directions;
b) strategic directions;
c) science management;
d) utility of risk management research; and
-------
e) the laboratory's strategic balance for the next decade.
Crosscutting themes evident in these comments include the benefits of a rational,
transparent, decision process; the utility of metrics; the need to maintain core
competencies; and the continuing need for improved technology. While much attention
has been given to refinement and implementation of the risk assessment paradigm and
process, refining the understanding of the risk management paradigm and its
implications for research has received less attention.
Regarding research program directions, the 12 areas NRMRL selected are
appropriate. However, the decision process-particularly as it relates to balancing
short-term and long-term goals-could benefit from a focused statement of how these
topics were selected as opposed to others, together with the desired specific research
goals. The decision process should be documented well enough that anyone reading it
can understand not just what was decided but how and why those decisions were
reached. Because priorities will change over time, as will resources, NRMRL needs a
recognized decision process detailing those factors leading to the addition or
substraction of research elements. These factors should also be chosen both for the
perceived criticality of the thrust area and for the potential for NRMRL to actually make
a difference.
There are opportunities for NRMRL to play a dominant leadership role in both
traditional and emerging research areas. Hence, it would be prudent to assess these
opportunities on a continuum, and to define strategies not only consistent with the
strategic plan, but those responsive to EPA's increased emphasis on community-based
environmental decision making.
NRMRL did not define the current and future core competency needed for EPA
in environmental risk management. Nor did NRMRL compare the capabilities of the
NRMRL staff with the abilities needed to undertake necessary future research. It is
critically important that NRMRL continue with and advance the core capabilities of its
staff, so that it can respond to future (and unforeseen) environmental challenges. It is
also important that NRMRL address current environmental problems. Therefore,
NRMRL should define the current and future core competency needs of the Agency in
environmental risk management, and then compare the NRMRL staff capabilities with
the types of research prescribed by its agenda. (A useful discussion of core
competency may be found in Appendix D to SAB, 1995a).
Regarding science management, NRMRL has developed certain key elements
such as: measures of success; peer review; and the use of teams. NRMRL will need to
refine each element as the new laboratory paradigm takes effect. Measures of success
need to be flexible and applied with purpose clearly in mind. Different measures will
apply to different activities and the individual researchers or teams of researchers
-------
involved. Measuring success should be a reasoned and thoughtful process that must
be sustained by the scientific and technical community, whether focused on the extent
to which the activity achieves an assigned research task, or whether the activity
contributes in the broader context of mitigating human and environmental risks.
Moreover, effective measurement policies should provide a basis for self-improvement.
Regarding the utility of risk management research, the Committee commends
NRMRL for identifying the factors that determine the value of the Agency's risk
management goals, and the extent to which NRMRL's research can influence or be
influenced by them. Accordingly, the NRMRL risk management plans conceptually fit
into the risk assessment/risk management paradigm. The laboratory has clearly
identified its role and the focus of its research agenda.
The next logical step will be selecting research topics that fit the agenda. The
ultimate challenge is demonstrating that health and environmental risks have been (or
will be) reduced. Two activities would enhance the capacity of NRMRL to provide
evidence of risk reduction-1) development of compliance assurance models and
methods; and 2) development of measures of environmental and public health
improvement.
Regarding strategic balance, NRMRL must consider that regional environmental
assessments will increase during the next decade as environmental protection moves
towards community-based approaches. Dominant factors are likely to be demographic
fluidity, infrastructure and natural hazards which, when extended to global dimensions,
introduce an array of environmental issues for which control schemes are not currently
available. Indeed, the solution to some of the critical environmental and health
problems will reside in geographic regions over which the Agency has no jurisdictional
authority.
In response to this trend, NRMRL should expand the scope of its pollution
prevention and risk management assessment activities. Significant contributions can
be made in waste minimization, source characterization, cost analysis, technology
verification, and technology transfer.
The current de-emphasis on waste management in NRMRL's plans, structure or
vocabulary appears to be somewhat short-sighted, particularly when waste
management is and likely will remain a significant societal issue and one that occupies
the attention and energies of both the public and private sectors. Although pollution
prevention or waste avoidance is often considered superior to the remediation
approach implicit in most waste management programs, it seems inappropriate to
almost abandon the search for innovative solutions within the research agenda of the
Agency.
-------
Core competencies in technology development and remediation within NRMRL
and linkages with similar research programs should be maintained so that the menu of
options for treatment and control can continue to expand. Such linkages provide vital
feedback mechanisms to the research community, and serve as a reality check and a
measure of overall success.
-------
2. INTRODUCTION
2.1 Background
In late 1995, the EPA Office of Research and Development (ORD) realigned its
organizational structure using risk assessment and risk management as organizing
principles for a nationwide laboratory system, including the National Risk Management
Research Laboratory (NRMRL). Since publication of "Risk Assessment in the Federal
Government: Managing the Process" (MAS, 1983), much attention has been given to
refinement and implementation of the risk assessment paradigm and process,
respectively. Although many of the activities of EPA and other regulatory agencies at
the Federal, state and community levels involve risk management, refining the
understanding of the risk management paradigm and its implications for research has
received less attention.
In the "Strategic Plan for the Office of Research and Development" (EPA,
1996a), ORD described the relationship of risk assessment to the risk management
process, and emphasized the need for scientific and engineering research to enable
sound risk management decisions and actions. NRMRL's mission is to conduct
research to reduce uncertainties and costs associated with making and implementing
environmental risk management decisions.
NRMRL's research focuses on two important areas: first, characterizing pollutant
sources that require management; and second, identifying, developing, and evaluating
cost-effective tools and technologies for prevention, control, restoration, and
remediation of environmental problems that are high risk, high cost, or that lack
effective management alternatives. NRMRL catalyzes development and commercial
application of cost-effective management alternatives through joint efforts with public
and private sector partners, through programs to verify performance and cost of
innovative technologies, and through independently-conducted research activities.
NRMRL also provides technical assistance and technical information transfer in
support of improved risk management decisions.
The NRMRL research program is in a state of transition. In addition to the
organizational realignment mentioned above, other factors have suggested major
redirections for the research program. These include changes in ORD program
priorities, e.g., emphasis on airborne fine particle research and de-emphasis on
hazardous waste research; major funding reductions over the last two years; the loss of
key personnel with reduced opportunity for replacement; the need to shift emphasis
from management of research and development conducted extramurally to the conduct
of research and development intramurally; the use of teams to facilitate research
planning and implementation; and the implementation of expanded guidelines on peer
-------
review and quality assurance of ORD research.
2.2 Structure of the National Risk Management Research Laboratory
In realigning its organizational structure using the risk assessment and risk
management paradigm, ORD used the following "building blocks" to form the new
National Risk Management Research Laboratory (NRMRL):
• Air and Energy Engineering Research Laboratory in North Carolina
• Risk Reduction Engineering Laboratory in Ohio
• Risk Reduction Engineering Laboratory in New Jersey
• Robert S. Kerr Environmental Research Laboratory in Oklahoma
• Headquarters Liaison Office in the District of Columbia
Moreover, NRMRL remains geographically dispersed, and now consists of:
• Water Supply & Water Resources Division
• Land Remediation & Pollution Control Division
• Sustainable Technology Division
• Air Pollution Prevention & Control division
• Subsurface Protection & Remediation Division
• Technology Transfer & Support Division
• Technology Coordination Office.
Many contemporary environmental problems require interdisciplinary solutions,
thereby encouraging the coordination of the combined talents of a team. While
NRMRL includes some researchers who find success mainly in individual endeavors,
most are productive on team(s). Often the specificity (or cross disciplinary breadth) of
a research topic is the criterion of assignment or assumption of a topic to or by either
an individual investigator or a team of researchers.
The EEC observed several indications that NRMRL was increasing the use of
teams, especially multi-disciplinary teams. Some of these observations were:
a) the teamwork evidenced by the laboratory leadership at the review
meeting;
b) the laboratory's efforts to develop a meaningful risk reduction paradigm;
c) the laboratory leadership is interdisciplinary;
d) the Table of Organization names the four members of NRMRL's Research
Planning Coordination Team who serve on the ORD Research
Coordination Teams (RCT), and the RCTs work across ORD and with the
-------
program offices to develop research strategies implementing the ORD
Strategic Plan.
2.3 Environmental Engineering Committee Review of the National Risk
Management Research Laboratory with Charge
The National Risk Management Research Laboratory (NRMRL) requested that
the Environmental Engineering Committee (EEC) of the Science Advisory Board (SAB)
review its strategic directions and approach to research within the context of both the
1995 realignment of ORD's organizational structure and its strategic plan. On
September 25, 26 and 27, 1996, the EEC met in Cincinnati, OH to consult with
representatives of NRMRL and its component divisions, i.e., air pollution control,
subsurface protection and remediation, sustainable development, technology transfer
and support, and water supply and water resources. The EEC designated a NRMRL
Writing Subcommittee, consisting of EEC members, consultants and a Designated
Federal Official from the SAB, listened to overview presentations, discussed the
documentation accompanying the presentations, and provided a verbal synopsis of
findings and recommendations, to Agency staff before adjourning. This report presents
the Committee's major findings and recommendations and responds to the original
charge to the SAB/EEC posed by NRMRL, namely:
a) Examine and critique the research programmatic directions being taken
by NRMRL, such as whether NRMRL is pursuing the most appropriate
research problem areas within the framework of the ORD Strategic Plan,
and how well NRMRL's research relates to EPA's increased emphasis on
community-based environmental decision-making.
b) Review and comment on NRMRL's strategic directions as a research
laboratory organization, including NRMRL's use of its core technical
competencies and whether they require expansion, addition, or
consolidation to accomplish NRMRL's mission; NRMRL's approach to
transition from a primarily extramural management organization to an
intramural research and development organization; the roles of
partnerships in NRMRL's R&D program and the soundness of NRMRL's
approach to using them; and NRMRL's ability to leverage its programs
and resources with other agencies and organizations having similar but
not identical research missions (e.g., DOE, DOD, state agencies, and
agencies of other nations).
c) Review and comment on the effectiveness of NRMRL's approach to
science management, such as its measures of success and science
quality; the soundness of NRMRL's peer review process; the use of teams
to facilitate research planning and implementation; and NRMRL's use of
8
-------
its facilities in the conduct of its R&D program.
d) Examine and critique the relationship of NRMRL's risk management
research to the intended role of risk management in the risk
assessment/risk management paradigm.
e) Review and comment on the strategic balance needed in NRMRL for the
next decade among pollution prevention, technology development,
remediation, and risk management assessment activities (e.g., source
characterization, cost analysis, technology verification, and technology
transfer).
-------
3. COMMENTS ON NRMRL'S STRATEGIC DIRECTIONS AND
APPROACH
3.1 General Comments
Based upon its review of materials made available and the quality presentations,
it was clear to the Committee that the NRMRL management diligently prepared for the
review and successfully communicated the strategic research and management plans
for the laboratory. Therefore, the NRMRL personnel are commended for their efforts at
a time of considerable uncertainty in budgets, laboratory reorganization and the overall
strategic planning process being implemented by ORD. Moreover, the participating
personnel presented a united front in its support of the laboratory planning and
management process. What follows is a consensus position established by the
Committee after due consideration of the resources made available and discussions
with the NRMRL staff.
3.2 Research Programmatic Directions
The creation of the ORD Strategic Plan provides a baseline for future decision-
making on research programmatic direction. Within the risk assessment/risk
management paradigm in which ORD currently operates, NRMRL has attempted to
select its research agenda to reduce risk uncertainty in areas that pose the greatest
human and environmental risks, and to focus on issues that are important, relevant and
where it can make a difference. Accordingly, NRMRL plans to emphasize certain of
ORD's (EPA/ORD 1996) overall priorities; these are:
a) drinking water disinfection;
b) particulate matter;
c) endocrine disrupters;
d) ecosystem risk assessment;
e) health risk assessment; and
f) pollution prevention and new technologies.
10
-------
Other areas of high importance include:
a) air pollutants;
b) indoor air;
c) global change;
d) drinking water (in addition to disinfection issues);
e) waste site characterization; and
f) waste management and site remediation.
The distribution of resources described in NRMRL's FY 97 Research Plan
appears adequate to address the specific topics selected for emphasis. However, a
focused statement of how these topics were selected as opposed to others, together
with the desired specific research goals, would have been beneficial, particularly in
regard to schedules for attaining a balance between both short-term and long-term
goals. Such an initiative of selecting goals and setting schedules of achievement will
need to include attention to engineering issues and implementing tactics, which can
also serve as one index of the relative success of the research agenda. Moreover, in
anticipation of changes in priorities consonant with available resources and topical
urgency, a recognized decision process should be in place which details those factors
leading to the addition (or substraction) of research priorities. NRMRL documents
reviewed by the EEC were clear and well written, but neglected to document the
decision process. As a result, it is possible to identify the laboratory's priorities and to
have some sense of the factors which influenced those choices, but it is unclear how
the priorities were established. These factors should also be conditioned not only by
the perceived criticality of the thrust area, but by the potential for NRMRL to actually
make a difference.
There are opportunities for NRMRL to play a dominant leadership role in both
traditional and emerging research areas. Hence, it would be prudent to assess these
opportunities on a continuum, and to define strategies not only consistent with the
strategic plan, but those responsive to EPA's increased emphasis on community-based
environmental decision-making. This linkage with community-based environmental
decision-making is still at an early stage and will require considerable effort, because
such decision-making involves many non-technical aspects that may be more critical to
the ultimate success of the concept. Whereas drinking water appears to rank high as a
focus in NRMRL's plans, its community impact may be more impressive if concentrated
on infrastructure improvement rather than incremental advances in treatment
technology. Hence, issues that could constitute the arena of such a focus are, for
11
-------
example, system rehabilitation techniques, application of geographic information
systems (CIS), maintenance cost modeling, system deterioration rate modeling, city
planning effects on distribution efficiency, and pipe deterioration effects on drinking
water quality, all of which would invite and promote collaboration with local and
regional constituencies.
Within a more global perspective also embraced by the ORD strategic plan, but
not explicitly evident in NRMRL's research plan, are similar opportunities for
community-focused initiatives. For example, natural hazards can impose dramatic
impacts on human welfare and natural ecosystems. Beyond the Horizon (SAB, 1995b)
provides such a focus, with, for example, floods being shown to impose major
environmental/health impacts that could serve as a basis for a prospective EPA
program initiative and leadership position.
Finally, the current de-emphasis on waste management in NRMRL's plans,
structure or vocabulary appears to be somewhat short-sighted, particularly when waste
management is and likely will remain a significant societal issue and one that occupies
the attention and energies of both the public and private sectors. Although pollution
prevention or waste avoidance is often considered superior to the remediation
approach implicit in most waste management programs, it seems inappropriate to
almost abandon the search for innovative solutions within the research agenda of the
Agency. Preventive programs are unlikely to be effective as sole control measures for
all wastes, and NMRML could exploit its past emphasis and established expertise to
help develop, share and apply those innovations for waste management necessary to
"prevent, mitigate and control pollution".
3.3 Strategic Directions
As an extension of the previous theme of searching for logical opportunities to
provide focus to the overall research agenda, particularly as NRMRL transitions from
an external research management organization to one performing intramural research,
it is important to recognize the significance of this shift and the challenges it portends.
"Fad" research topics or topics that complemented in-house research were historically
managed through extramural research programs. The advantage was the ability of
EPA to control and focus the research, such that programmatic goals and milestones
were met. The disadvantage was that there was not always open competition, and that
research talents often remained dormant or unproved as Agency "researchers" honed
their managerial skills. Personnel that directed and managed research conducted by
extramural investigators may not be able to become productive researchers within an
in-house system, even with appropriate mentoring.
3.3.1 Core Competencies
12
-------
A key missing link in the planning exercise undertaken by NRMRL was a
definition of the current and future core competencies needed for Agency in
environmental risk management, and a comprehensive comparison with the capabilities
of the current NRMRL staff and their ability to undertake the types of research that will
be necessary in the future. In some ways, it is more important for NRMRL to continue
with and advance the core capabilities of its staff so that it can respond to future (and
unforeseen) environmental problems than it is to address specific environmental
problems. For example, the study of formation and control of chlorinated dioxin from
combustion sources is not only important today to support the regulatory development
for hazardous waste combustion rules, but because it enhances the expertise of the
laboratory on the formation and control of all types of trace air toxics in all types of
combustion systems. Because combustion will remain the primary world energy source
for the foreseeable future, this expertise must be maintained and expanded. In the
same vein, the NRMRL should define the other core competencies that are required
both now and in the future to carry out the total mission and goals of the laboratory.
The laboratory plan then should address how to move in the strategic directions as well
as how to maintain and cultivate the core competencies critical for the future. It is
equally essential for the laboratory to assess its current capabilities. The EEC is
aware of the fact that the NRMRL has begun to survey its staff to determine what
capabilities are thought to exist, but the EEC did not review the processes being used
in this survey. This survey must be well thought out and directed towards obtaining a
critical assessment of the real capabilities. It needs to address whether the laboratory
has the right type of people, how will the people align with the future high priority
research areas, and what types of core competency need to be nurtured and/or
developed for the future.
The laboratories that now constitute NRMRL have some highly competent and
recognized researchers, with staffing levels commensurate with the previous role of
research management. About 23% of the degreed staff have Ph.D. degrees, with a
broad degree distribution in the supporting disciplines, i.e., 49% in engineering, 18% in
physical sciences, 9% in life sciences, 5% in business administration, 4% in natural
resources, 3% in social sciences and 12% in other specialties. It is also commendable
that in-house expertise has been supplemented with a large number of visiting
scientists and post-doctoral fellows. Moreover, as budgets shrink, establishing
partnerships with other agencies will become even more important. Therefore, NRMRL
should define the core competencies and alliances that are required both now and in
the future to carry out its mission and goals. Such a laboratory plan will be crucial in
addressing implementing tactics consistent with the overall strategic directions.
Considering the emerging focus of NRMRL on ecosystem-type environmental
issues, recruitment of or fortification with appropriate disciplines should be an element
of future personnel allocation efforts. Here again, resources are unlikely to be
available to NRMRL to satisfy all of its important research mandates. Hence,
13
-------
collaborative programs with other agencies, and extension of participation in inter-
agency science forums beyond policy-level personnel to researchers and technical
analysts, will be necessary. Some examples of where NRMRL's research interests and
laboratory plans intersect with those of other agencies include:
a) Natural Hazards (NIST and USGS)
b) Global Change/Tropospheric Ozone (NASA and NOAA)
c) Ecosystem Restoration (USDA and USER)
d) Indoor Air (Center for Indoor Air)
e) Contaminated Sediments (US Army Corps of Engineers, WES)
Several forces such as budget cuts, the growth of the extramural grants
program, and scrutiny of contracting procedures are acting to push the NRMRL into an
in-house research posture with substantially fewer cooperative agreements and less
contractor support. The Committee finds that these forces and the consequent
reactions could insulate NRMRL intramural research from the benefits and synergism
that extramural research provides, and slow the pace and credibility of scientific and
technological advances. This is a particularly vexing issue, given that the pendulum of
change from extramural to intramural research has apparently swung so far as to stifle
efforts to provide some moderation. Hence, the laboratory needs to develop a plan that
fosters a balance between the in-house agenda and cooperative, collaborative and
integrated research with outside researchers as a top priority. Increased funding of the
grants program has directly reduced the ability of the ORD to conduct cooperative
research, and NRMRL must rethink how to use all available and potential funding
mechanics to accomplish its agenda and avoid the insularity that discourages
researchers from working with their external scientific and engineering counterparts.
This will require development of implementing mechanisms beyond the use of requests
for proposals or workshops ensuring interaction of various researchers on specific topic
areas.
3.4 Science Management
NRMRL's approach to science management contains some key elements that
include measures of success, peer review, and the use of teams for research and
planning. This is a step in the right direction, but each of these elements will need to be
refined as the new laboratory paradigm takes effect. Hence, the Committee concluded
that the organizational goals and metrics of success need to be restructured in order to
be more effective in planning NRMRL activities. The organizational goals defined in
the NRMRL plan are not really goals, but rather are more a listing of activities that will
14
-------
be undertaken by NRMRL. For example, increasing and strengthening in-house
research is what will be done, not the actual goal. Therefore, NRMRL would be better
served by the plan if the goals were explicitly defined to address the underlying
rationale for these activities, e.g., why does NRMRL want to strengthen in-house
research? Is it to improve the core competency of the staff or to better serve the
Agency? Is it to minimize the amount of extramural expenditures, or to improve
efficiency and flexibility in responding to the needs of the Agency? Why does NRMRL
want to implement an internal grants program? Why does NRMRL want to develop
strategic partnerships to leverage work? Why leverage with agencies of other nations?
By defining why the laboratory wants to increase and strengthen in-house
research, the laboratory can more effectively define the appropriate metrics of success
to indicate whether goals are being achieved. For example, if the actual goal is to
increase core competencies so that NRMRL can more effectively respond to Agency
needs, then increasing in-house research will be measured in a unique manner related
to this particular goal. Definition of the underlying rational for each of the activities
(currently called organizational goals) is a necessary step in defining the
appropriateness of associated activities and measures of success.
Measures of success also need to be flexible and applied with purpose clearly in
mind. Different measures will apply to different activities and the individual researchers
or teams of researchers involved. For example, numbers of peer-reviewed
publications, numbers of citations, status in the research community and impact of
contributions on policy or implementing actions can all be applied as measures of
success. These metrics need to be tempered, however, by a realization that certain
research, e.g., computer model development, is prone to yield many opportunities for
publication, while other efforts, such as field testing or laboratory exploration, may yield
fewer papers but yet have a profound effect on the advancement of science and
technology. Hence, measuring success should be a reasoned and thoughtful process,
that must be sustained by the scientific and technical community, whether focused on
the extent to which the activity achieves a assigned research task, or whether the
activity contributes in the broader context of mitigating human and environmental risks.
Moreover, effective measurement policies should provide a basis forfollowup program
planning and resource allocation in the final analysis.
Finally, success and its measure can apply individually as well as collectively.
Often the specificity (or cross disciplinary breadth) of a research topic is the criterion of
assignment or assumption of a topic to or by either an individual investigator or a team
of researchers. The current laboratory staff and facilities are reflective of both types,
with certain researchers finding success mainly in individual endeavors, while others
are most productive as a team. There is probably still room for both, although many
contemporary environmental problems require interdisciplinary solutions, thereby
encouraging the coordination of the combined talents of a team.
15
-------
Within an outcomes assessment perspective, research issues of narrow scope
may be best explored by individual scientists or engineers, with the results subjected to
broad peer review. Similarly, crosscutting issues, such as ecosystem restoration, are
likely best dealt with by multidisciplinary teams, but again subjected to the rigors of
broad peer review. In either case, special attention needs to be placed on selecting
and sustaining the peer review process on a continuum, and providing the facility
support requisite for successful achievement.
3.5 Utility of Risk Management Research
It is particularly commendable that NRMRL has identified the variety of factors
that determine the value of the Agency's risk management goals, and the extent to
which NRMRL's research can influence or be influenced by them. Accordingly, the
NRMRL risk management plans adequately fit into the risk assessment/risk
management paradigm, at least in concept. As shown in Figure 1 (an adaptation, with
modifications, from NRMRL's presentation materials), the laboratory has clearly
identified its role, with asterisks presumably indicating the focus of its research agenda.
3.5.1 Beyond Engineering
Figure 1 describes activities suited to a risk management laboratory rather than
a highly focused engineering laboratory. Since the SAB report, Future Risk, was
published in 1988, NRMRL has conceptualized a broader approach to risk reduction.
NRMRL has not explored all these options, or any one of them in depth. However,
NRMRL has broadened its understanding of the need for both engineered and non-
engineering options over the last decade. Recent EEC reviews of the SITE program
and the Pollution Prevention Research Strategy document captures some of the
progress and some of the difficulties. Among the ten recommendations highlighted in
Future Risk were:
a) EPA should shift the focus of its environmental protection strategy
from end-of-pipe controls to preventing the generation of pollution;
and
b) to support this new strategy, EPA should plan, implement, and
sustain a long-term research program.
In discussing the second recommendation, Future Risk, identified candidate core
research areas related to risk reduction, including: risk communication, incentives for
risk reduction, education and technology transfer, and environmental management and
control systems (the latter are largely engineering).
In 1990, the SAB published Reducing Risk. While this report is perhaps most
16
-------
frequently cited for supporting comparative risk, the Strategic Options Subcommittee
considered ways to reduce risk. Among the Subcommittee's recommendations were:
a) EPA should establish priorities based on the potential for risk
reduction;
b) In order to reduce risk and prevent pollution in a significant way,
EPA must substantially broaden its kit of environmental protection
tools.
17
-------
3.5.2 Options for Research Organization
It is legitimate to ask whether EPA should take on the diverse and more general
risk management areas or be a highly focused engineering laboratory. Alternatively, a
new risk management laboratory could be created-even a "virtual" laboratory-to
address the new areas. However, the EEC did not directly address these questions,
because it was not charged to do so.
The EEC recognized within the context of Figure 1, further evidence both that
ORD has been responsive to the SAB advice, and that the laboratory management has
developed a clear grasp on major trends in risk reduction. The latter was perhaps most
striking to those EEC members who also served on the SAB's Integrated Risk Project's
Risk Reduction Options Subcommittee. Management's participation in the NRMRL
review was very impressive. Each of the many managers present voiced an
understanding of the laboratory's mission, priorities, and challenges in a coherent and
consistent way. The management and research staff gave every appearance of being
able to operate as a true team.The EEC was more concerned with helping this team get
on with the work at hand than in conceptualizing alternative approaches to structuring
laboratories.
The EEC recognizes that ORD faces certain challenges. Chief among these are
the difficulties of moving from a largely external research program to a largely internal
one, and reducing the institutional barriers to collaborative research between ORD
laboratory and external organizations.
NRMRL also faces the challenge of a broader notion of risk management. The
EEC favors this broader notion and is encouraging NRMRL to grow along these
multidisciplinary lines. The EEC did not consider another reorganization as a means to
perform risk reduction research.
The EEC favors NRMRL adopting the broader view of risk reduction shown in
Figure 1. The EEC anticipates that the multidisciplinary work will prove stimulating to
the NRMRL. Clearly it has the potential to improve the utility of the engineering work
and, in some cases, provide non-engineering alternatives.
3.5.3 The Next Step
The next logical step will identifying research topics that fit into this agenda. The
ultimate challenge in this process will be to demonstrate that health and environmental
risks have been (or will be) reduced when the agenda and its research tasks are
implemented. Hence, two activities identified as components of the risk
assessment/risk management paradigm that would enhance the capacity of NRMRL to
provide evidence of risk reduction include: first, development of compliance assurance
18
-------
models and methods; and second, development of measures of environmental and
public health improvement.
The risk reduction options listed with the effort to "Identify and Evaluate Risk
Management Options" in Figure 1 reflect the option categories identified by the Risk
Reduction Options Subcommittee of the SAB's Integrated Risk Project. This project is
ongoing, and the results are likely to be useful to NRMRL in its efforts to select specific
and cross category options for risk reduction.
3.6 Strategic Balance for the Next Decade
As highlighted in the recent NACEPT report on "Promoting Innovative
Approaches to Environmental Protection" (EPA, 1996b), a community-based approach
mandates a reorientation of traditional media-based programs to the more holistic,
multi-media imperatives of specific environmental settings. Accordingly, the NRMRL
must strive to strike a strategic balance between the various facets of such a scenario,
with the inevitability that regional-type environmental assessments will increase during
the next decade. Dominant driving factors are likely to be demographic fluidity,
infrastructure and natural hazards which, when extended to global dimensions,
introduce an array of environmental issues for which control schemes are not currently
available. Indeed, the solution to some of the critical environmental and health
problems will reside in geographic regions over which the Agency has no jurisdictional
authority.
In light of the scenario outlined above, it will be important for NRMRL to expand
the scope of its pollution prevention and risk management assessment activities,
because significant contributions can be made in these areas, whether concentrated on
waste minimization or source characterization, cost analysis, technology verification, or
technology transfer. Although this would suggest a decreasing emphasis on
technology development and remediation within NRMRL, core competencies in
technology development and remediation within NRMRL and linkages with similar
research programs should be maintained so that the menu of options for treatment and
control can continue to expand. Such linkages provide vital feedback mechanisms to
the research community, and serve as a reality check and a measure of overall
success.
19
-------
o
?
re
+*
0)
_0>
Q.
E
E
2
o>
o
0)
'
•o
0)
"o ^^
> PT
= 0>
= ;?
(/) Q.
(/) HI
•C E
« '5.
5
re co
E «
O) Q
5 Di
2 O
PB o
Q. *J
*J O)
c -S
g -S
5 =
O) J
re N-
c o>
re o>
E £
(-
re E
•^ E
t
1= «
= c
(0 O
'"
o
• .*;
0) -o
9- 0)
= fc
-------
REFERENCES
EPA. 1996a. Strategic Plan for the Office of Research and Development (EPA/600/R-
96/059), Office of Research and Development, US Environmental Protection
Agency, Washington, DC. May 1996.
EPA. 1996b. Promoting Innovative Approaches to Environmental Protection: A
Summary of Recommendations from the National Advisory Council for
Environmental Policy and Technology, (EPA 100-R-96-013), National Advisory
Council for Environmental Policy and Technology (NACEPT), US Environmental
Protection Agency, Washington, DC. June 1996.
EPA. 1997. 1997 Update to ORD's Strategic Plan (EPA/600/R-97/015). Office of
Research and Development, US Environmental Protection Agency, Washington,
DC. April 1997.
MAS. 1983. Risk Assessment in the Federal Government: Managing the Process,
National Academy of Sciences, National Academy Press, Washington, DC.
March 1983, 191 p.
SAB. 1994. Strategic Research Planning Commentary (EPA-SAB-EEC-COM-94-004).
Science Advisory Board, Environmental Engineering Committee, US
Environmental Protection Agency, Washington, DC.
SAB. 1995a. Future Issues in Environmental Engineering. (EPA-SAB-EEC-95-004),
Science Advisory Board, Environmental Engineering Committee, US
Environmental Protection Agency, Washington, DC.
SAB. 1995b. Beyond the Horizon: Protecting the Future with Foresight. (EPA-SAB-EC-
95-007), Science Advisory Board, Environmental Futures Project, US
Environmental Protection Agency, Washington, DC.
SAB. 1996. Review of the "Strategic Plan for the Office of Research and
Development" (EPA-SAB-RSAC-LTR-96-004), Science Advisory Board,
Research Strategies Advisory Committee, US Environmental Protection Agency,
Washington, DC.
R-1
-------
APPENDIX A - MATERIALS REVIEWED
The following materials were available to the Environmental Engineering
Committee for use in the review of NRMRL.
1. Documents Relating to Community Based Environmental Protection
a) Edgewater consensus
b) ORD document
c) NACEPT Report, "Promoting Innovative Approaches to Environmental
Protection," 1996
2. Documents relating directly to the NRMRL review
a) August 12, 1996 letter transmitting:
1) Charge for the NRMRL review
2) Strategic Plan for the Office of Research and Development
(EPA/600/R-96/059, May 1996)
3) Review of the Strategic Plan for the Office of Research and
Development by the Research Strategies Advisory Committee
(RSAC) of the Science Advisory Board (EPA-SAB-RSAC-LTR-96-
004, March 1996)
4) Promoting Innovative Approaches to Environmental Protection: A
Summary of Recommendations from the National Advisory Council
for Environmental Policy and Technology (EPA 100-R-96-013,
June 1996)
5) Federal Register notice for the meeting
b) September 4, 1996 letter transmitting documents on:
1) Overview of NRMRL
2) In-House support
3) Research teams
4) Peer review
5) Internal grants
6) QA/QC
7) ORD Research Planning/Prioritization
8) Current NRMRL Research Focus & Program Directions
9) Measures of Success/Outputs
c) Overheads distributed September 26, 1996
d) Leader's Handbook for Quality Integration of the Atlantic Fleet provided
by Dr. Pojasek.
e) Sub-agenda for Division Director Panel, distributed by Dr. Oppelt
A-1
-------
DISTRIBUTION LIST
The Administrator
Deputy Administrator
Regional Administrators, Regions 1-10
Assistant Administrators
Directors, Office of Research and Development Laboratories
Director, Office of Air and Radiation
Director, Office of Atmospheric and Indoor Air Programs
Director, Office of Policy Analysis and Review
Director, Office of Environmental Processes and Effects Research
Director, Office of Policy Analysis
Director, Climate Change Division
EPA Headquarters Library
EPA Regional Libraries
National Technical Information Service
Congressional Research Service
Library of Congress
------- |