REPORT OF THE QUALITY-SCIENCE TEAM
U. S. ENVIRONMENTAL PROTECTION AGENCY
     NATIONAL PERFORMANCE REVIEW
              August 6, 1993

-------
                  i C.-
                   Quality Science Team Members
Co-Chairs:  Wil Wilson - OAR,  i Bill Raub -  OA

Tyrone Aiken - OPPTS
Kay Austin - OPPTS
Aabika Bathija - OPPTS
Timothy Backstron - OGC
Steven Bayard - ORD
Joseph Breen - OPPTS
Douglas Campt - OPPTS
Jose Cisneros - Region 5
Mary Clark - OAR/NAERL
Larry Cupitt - ORD/AREAL
Charles Kensley - Region 7
Steven Hedtke - ORD/Duluth
Dominique Lueckenhoff - Region 3
Edward Ohanian - OW
Ronald Patterson - ORD
Sally Perreault Darney - RTP
Rosemarie Russo - ORD/Athens
Stephanie Sanzone - OA/SAB
Rita Schoeny - ORD/Cincinnati
Robin Sega11 - OAR/RTP
Babasaheb Sonawane - ORD
Silvia Termes - OPPTS
Raaona Travato - ow
Darlene Watford - OPPTS

-------
                  TEL:                    .   Rug  27.93   9:23 No.001 P.03
X.  VISION
              that USEPA be recognized throughout the world
        as a science agency, as veil as a regulatory agency,
                             by virtue of

    1)  decisions consistently based  on high-quality science;

    2)  leadership In all major areas of environmental science; 1 and

    3)  affective communication and  coordination of  environmental-
       scientific Issues and policies.

ZZ.    BACKGROUND/CURRENT STATE

Rule-Baking and  other policy decisions  at USEPA almost invariably
rely on science,  with  the nature and  extent of that reliance  varying
considerably from one  instance  to   another.   But  one  factor  is
invariant:  quality must be a first-order concern  in deciding  what
scientific  information  is relevant,  how  it is presented, and how: it
is  used.    Quality  science  is a  condition  precedent  for  quality
decisions.

USEPA's science activities range from fundamental, basic research to
highly  applied,  problem-solving  efforts.   Basic  research  may  be
driven by critical, media-specific environmental problems;  or it may
create  the  foundation for unprecedented,  multimedia approaches  to
the nation's most critical environmental  challenges.   While  applied
research  may be  used  to  fulfill  immediate  needs,   it  may  also
contribute  to attaining  long-term strategic goals.

Scientific  activities at USEPA are  performed both by  its research
laboratories and  by   its  program  and  policy  offices2.    These
activities  directly impact the national and  international environ-
mental-science  community.  For example, USEPA's analytical methods,
predictive   models,  and  risk-assessment  guidelines  are  used  as
standards by other agencies, both nationally and abroad.

USEPA is on the cutting edge of many  areas of  environmental science.
and contributions of  its scientists  and  other technical personnel
are respected world-wide.  However,  the role of quality science  in
USEPA policies,  decisions, and actions is often questioned.   USEPA'«
major agenda is unquestionably that  of a regulatory  agency  charged
   1 Toe words In this text are Intended to be inclusive, not exclusive. 'Environmental science* is used
from a holistic perspective, encompassing both ecological science and human health science. "Science*
(and 'scientists") is used to represent not only the traditional natural sciences, but engineering (and
engineers), the mathematical sciences, and the many other sciences (e.g., economics, social sciences) thai
nay become more and more important over the next decade.

   * References to "program and policy offices" in this document  are intended to address both
Headquarters and Regional operations.

-------
with promulgation and enforcement of air-, water- and land-quality
statutes.   Science  often is  — or  seems  to  be  — a  secondary
priority.  Zn fact/ USEPA decisions all too frequently are perceived
by OMB, Congress, other  Federal agencies, industry,  or the public as
not grounded  in  sound science.  To the extent that the Agency does
not  support  and use  quality  science,  this  perception  will  be
repeatedly reinforced.

USEPA's ability to conduct an effective and coherent quality-science
prograa  is  impacted  by  numerous congressional mandates  and highly
compartmentalized appropriations.  Program offices within USEPA are
divided  by  media-specific statutes that sometimes erect seemingly
insurmountable barriers to cross-media initiatives. Moreover, about
75% of the budget is  directed toward problems of high visibility and
relatively low risk rather than to environmental concerns considered
by the scientific community to present significantly greater risks
to human health  and  the environment (Problem Area Analysis of the
FY94 Budget,  OPPE, June,  1993).

Another  manifestation of this  balkanized governance is  that the
Agency's research and development is not driven by a holistic, long-
term, strategic  plan.  Instead,  program offices,  dependent upon the
Office of Research and  Development  (ORD) for support of regulatory
actions,  have seen  critical research  cut  back  substantially  or
eliminated  outright, as  ORD  is  forced  to  respond to  new policy
mandates without  a  commensurate increase  in  funds  and  staff.
Neither  ORD  laboratories nor the  program offices historically have
had  sufficient resources to address  allvhigh-priority  short-term
needs and a substantial  long-term research agenda simultaneously.

Contributing  to  the  perception  that USEPA's decision-making is not
always  rooted in sound science is the  Agency's severely limited
ability  to support  and nurture its scientific  staff.   There are
numerous  inconsistencies throughout  the  Agency  regarding  job
functions, training  and development, opportunities  for advancement,
and  recognition  for  scientists.   In the  face of  attrition and loss
of institutional memory in specific subject-matter areas, the Agency
has  fallen behind with respect to training and  career development
programs for its technical staff.  Further, reductions in science
and  engineering  career positions not only have bred undue dependence
on contractors to meet scientific and technical needs but also have
encumbered USEPA scientists increasingly with contracting duties —
thereby  shifting their  focus and their talents away from science
toward administration.

Ineffective communication between USEfA policy-makers and scientific
staff  also tarnishes USEPA's  reputation with respect to science.
Evaluating  and  communicating  the  sources  of  uncertainty  in
predictive exposure and  risk  assessment are not deeply  ingrained
practices across the Agency.  Scientists do not always fashion their
recommendations  and/or  risk   assessments   so  that  all   major
assumptions and  findings are made explicit.  Likewise,  many managers
do  not  frame their  questions  to enable scientists to address the
issues in the best  way.  Neither risk  assessors nor risk managers

-------
                 TEL:                      Rug  27.93   9 = 23  No.001  P.05
consistently convey their findings, decisions,, and  the associated
uncertainties  using the  language  and communication  tools  most
appropriate* for the target  audience.   USEPA cannot be  said  to be
fulfilling its mission if it cannot adequately convey levels of risk
to the public.

Communication  and  coordination  among  USEPA's science  programs —
both internally and externally  — are beset by problems  as  veil.
Inadequate coordination among programs  and'fehe laboratories has led
to science being practiced in isolation,  and perhaps,  redundantly.
There  is  no  comprehensive,   cross-referenced  data   base   that
identifies in-house expertise and  on-going activities for access by
Agency scientists.  Scientists working on similar or closely related
issues or projects find difficulty in establishing  peer review and
support  elsewhere  within  USEPA  because  there is  no  accessible
Agency-wide   listing   of   scientific  personnel,    training,   and
expertise.   The same  applies  to USEPA  publications,  scientific
findings, and decisions.

Communication and coordination of efforts with  scientists in other
Federal and State agencies and international organizations also-are
inconsistent.  USEPA research efforts often have no apparent impact
on the. research  agendas of  other  agencies,  nor  is there  a  well-
established mechanism  for USEPA staff to access  the results of their
counterparts  at  other  agencies.     while  there  are  examples of
successful collaborative efforts between  USEPA  and  industry,  (such
as pollution prevention and  product stewardship,)  interactions with
academia and industry should be considerably strengthened.

Inflexibilities in the contracting mechanism, relative to contract
types, inhibit the  Agency's  ability to  adequately address specific,
research needs. Currently, the governing process for all contracts,
whether  scientific or  non-scientific,  is  the  same.    Laboratory
scientific and technical staff  are  required to purchase equipment
utilizing the  same procurement criteria  as staff with completely
differing needs.   Due  to the vastly differing nature of  laboratory
scientific needs from other types of  procurement needs, utilizing
universal procurement  criteria  for  both of these areas  greatly
stifles the timely actions and operations of laboratory scientific
and technical staff.

Against this sobering backdrop of needs, opportunities and problems,
USEPA must take deliberate steps if it is to achieve its vision of
being  recognized   as  a science agency.    The remainder  of   this
document presents  an overall strategy  and some specific proposals
toward that end.

III. DESIRED STATE/GOALS

Not  only  can science support  the   over-arching  environmental-
protection strategies of today;  it also can define the strategies of
tomorrow.  Thus, changes  in the way USEPA staff  do science should
include, but not be limited to,  a greater  emphasis on the following
strategic   themes:  risk-based   priority  setting;     pollution

-------
prevention,  ecosystem  approaches,   and  product  stewardship;  and
environmental  education  and  equity for  all  population  groups
Underlying it all should be a strong, sustained program of basic and
applied research  in  all major domains of environmental science.

•   Cross-media issues

Today, environmental issues are more complex than ever before; and
environmental  protection  strategies  must   cut   across   all
environmental  media  and  all  routes of exposure.   Rarely is  it
sufficient to deal with environmental media individually.   Rather,
the  quality  of science  across all  program offices  that  exercise
statutory authority over specific media should  reflect a consistent,
holistic approach to protecting  and enhancing both ecosystems and
human  health  and,  at  the  same   time,  should  ensure  adequate
environmental resources to  meet future needs.

•   Quality data and  application

To ensure that science within USEPA  is of the highest quality, USEPA
must  improve  the quality  of data and  promote  its  appropriate
application in decision-making processes.  This  can be accomplished
by improving the  methodologies used  to acquire data; ensuring that
models properly reflect the  latest scientific information, data, and
concepts  —   including  uncertainty  analysis;  applying  models
responsibly and consistently;  and incorporating current scientific
knowledge into risk-based decision-making.

•   Communication  and Coordination

Effective communication and  coordination among USEPA scientists,
between scientists and policy-makers, and between the Agency and the
rest of  society  are essential  to   achieve high-quality  science.
USEPA must  improve  across  the board in this area.   Interactions
between risk managers and risk assessors warrant special emphasis,
as  do  USEPA interactions with other federal  agencies, state and
local agencies, OMB, Congress, academe, industry and international
organizations.  Above all, the Agency must improve its communication
to the general public.

e  Recognition  and Resources

Public recognition, including that of the Congress,  is essential for
USEPA  to achieve the  above  vision.    USEPA  requires  not  only
effective environmental legislation  but also resources sufficient to
identify  »isk-based  priorities  and  to  implement  appropriate
environment protection and  research  strategies.  USEPA's resources
must  also  include  a  well-trained, well-equipped,  and  highly-
motivated scientific and technical work force.
•  Planning

-------
                 TEL:                      flug  2?.95   9 = 2-5  Nc.Oul r.Gr
Strategic planning and appropriate resource allocation are necessary
to ensure:  1) the success of both short-term and long-term projects
and   2)   adequate  environmental   protection   for  the   future.
Ultimately, US EPA can do the right science and do the science right
only when the policies,  methods,  and infrastructure, as veil as the
public, actively support the undertaking.

XY. TARGET/ACTION

To  achieve   its  vision  that  quality  science   undergird  all
environmental decisions  and to  strongly support its leadership role
in all areas  of environmental  science,  USEPA must  identify those
elements forming the basis of credible science and make a commitment
to  implement specifically  targeted  initiatives  to  achieve  the
vision.  Broadly interpreted, a quality science agenda addresses the
following:

•  Clearly identifies and prioritizes  the most important scientific
   questions to be addressed;

•  Identifies   and   uses   the  most   appropriate  and   powerful
   experimental and analytical designs;

•  Employs state-of-the-art experimental techniques;

•  Makes  accurate measurements to generate reliable data;

•  Evaluates  and interprets the results  appropriately  to obtain
   accurate answers to the questions;

•  Uses   the  results  in  mathematical  models  to  apply  science
   knowledge efficiently and coherently;

•  Subjects  the methods, results, interpretations, and  models to
   field  validation and  independent review processes;  and

•  Provides  an  infrastructure  that recruits, retains  and rewards
   high   quality   scientific  personnel  and  an   on-going  core
   capability.

V. TARGET/ACTION  CATEGORIES

Development and implementation of an Agency-wide  quality science
agenda can be  accomplished through  the  target/action  areas  and
associated do-ables/initiatives that are summarized in this section.
Details on  each of  the  do-ables/initiatives are  presented in the
next section.

A. ACHIEVEMENT  OF QUALITY SCIENCE IN USEPA

TARGET/ACTION STATEMENT:  Over the  last few years,  several expert
panels have conducted thorough reviews  of  science at  USEPA  (e.g..
Safeguarding  the  Future;   Credible Science. Credible Decisions.
Reducing Siaki  Setting  Priorities and Strategies for Environmental

-------
Protection,  and Future Risk;  Research  Strategies  for the       .
These panels have Identified what needs to be done to  improve both
the  quality and the relevance  of science  at USEPA.   Now, USEPA's
leadership  needs to implement the recommendations.

Several  recommendations have consistently  emerged  from the expert
panel  reviews:   (i)  To  deal  effectively  with  today's  complex
environmental problems,  the Agency needs  to develop  a coherent
science agenda that moves  toward cross-media,  anticipatory research;
(2)  the  Agency  should  take steps to ensure that  qualify science
enters  the  decision-making  process early  and consistently  for
effective   environmental   protection  programs;   and  (3)  Agency
leadership  Bust undertake  a deliberate and  continuing  effort to
create the  climate,  culture,  and  incentives  necessary to encourage
superior science.

SPECTFTC
    1.  Quality-Science as a Continuing Policy Objective

    2.  Procedures that Promote Scientific Excellence

    3.  Science Advice within Program and Regional Offices

    4.  Science/Congressional-Liaison Team

B.  THE SCIENCE/POLICY INTERFACE

TARGET/ACTION  STATEMENT;  Efficacious use of science in support of
policy  decisions is the responsibility  of  both  policy-makers and
scientists.  The credibility of USEPA decisions,  the efficient use
of USEPA resources, and the cost-effectiveness of USEPA regulations
are  all  critically  dependent  on  the  maintenance  of  a  clear
distinction and  a proper balance  between science  and  policy.   The
place where scientific analysis ends and policy choices  begin is not
always  easy to determine.   In the context  of a  regulatory agency
with many pressing responsibilities, the tendency to blur the line
between science and policy  is difficult to resist for scientists and
policy Bakers  alike.

The key to crafting  and maintaining an appropriate  relationship
between scientific  inquiry  and policy making is  the nature of the
interaction  between   policy   makers  (or  "risk  managers")   and
scientists (or."risk assessors").  The timing, frequency, and nature
of such communications are all important. Scientific analyses that
are informative  and policy  decisions that are informed can best be
achieved through implementation of specific practices and procedure*
governing communication across the science/policy interface.

SPECIFIC PQ-ABLES

    1.  Preparation of Policy-Makers for the Science/Policy Interface

    2.  Preparation of Scientists  for  the Science/Policy Interface

-------
                 TEL:                      flug 27,93  9 = 23 No.001 P.09
   3.  CASE BOOK on the Science/Policy Interface

   4.  Management of Differential Risks among Population Subgroups

C. WURTURING QUALITY SCIENTISTS

TARGET/ACTION  STATEMENT:    As  USEPA works  to  advance the  high-
quality performance of science and technology in its organization,
it aust develop, increase and support the skills and talents of its
scientific workforce.

SPECIFIC PO-ABLES

   1.  Job enhancement and Resource Support for Scientists

   2.  Career Ladders and Promotion/Incentive/Reward Systems

   3.  Employee-Empowered Scientific Recognition Committee

   4.  Training For Scientists

   5.  Recruitment of Scientists

D. COMMUNICATION AND COORDINATION

TARGET/ACTION  STATEMENT;     USEPA  has  the   responsibility  to
communicate  its  environmental scientific  findings  to  properly
reflect  the high-quality  scientific  basis  for  decisionmaking.
Communication of findings and decisions must take into account the
specific audience for whom the information is intended and include
the  appropriate language and  tools to  attain the  most effective
results.    The Agency must coordinate  its  activities to  avoid
replication of tasks and attain  coherent products  (regulations,
guidelines, regulatory decisions, research findings).  Risk managers
and  scientists must establish good  communication to  ensure  that
regulatory decisions are based on  high-quality science.

SPECIFIC DO-ABLES

   1.  Prioritization of Intra-Agency Communication and Coordination

   2.  Exchange of Scientists (internally 4 externally)  For Purposes
       of Exchanging  Science

   3.  Communication  and  Coordination  with Other Federal Agencies,
       State and Local Agencies, and International Organizations

   4.  Communication  and  Coordination with Academia  and Industry

   5.  Communication and Coordination  with  Congress, OMB,  News
       Media, and the General Public

72.  DO-XBIES/INITIATIVZS

-------
A.  Achievement ef Quality Scienca in USEPA

   A.I    Quality-Science as a Continuing Policy Objective

VHATi   Make a public commitment to use  and  support high-quality
•cience in all of the Agency's deliberations and actions.

This would consist of several actions:

   (1)    The  Administrator  would   publicly  adopt  the  vision
          statement  of the  Quality  Science  Team  and  commit  the
          Agency to accomplishing the vision statement.
                   •
   (2)    The Administrator would establish a process by which to
          implement  the  changes  needed to ensure that  USEPA  is a
          science agency with high-quality science underpinning all
          of its actions.

WHOi   The Administrator must publicly affirm the vision for quality
science at USEPA.  She must also establish and empower the Agency-
wide Committee for Quality Science, and task it with overseeing the
Agency's "deliberate and  continuing  effort to create the climate,
culture, and incentives necessary to  encourage superior science.!1

TZMZ LIKE:  One legacy of the National  Performance Review should be
the establishment and empowering of an Agency-wide team, a Committee
for  Quality  Science,  to  develop  and   implement  policies  and
procedures to enhance Quality Science at USEPA.  This Committee for
Quality Science should be part of USEPA's' effort to "reinvent1* the
Agency.  The Committee will be "continuing", providing suggestions
for  changes  and  assessing  progress  made  in   improving  quality
science.  The  Committee  should  incorporate and build upon efforts
already underway,  including the Council  of  Science  Advisors,  the
Sci/Tech Committee,  and the Quality Science Team itself.

BARRIERS:  There is skepticism that USEPA, with its regulatory role,
can also be a "science" Agency,  overcoming this skepticism within
USEPA itself, in the Administration,  and in Congress will be a major
challenge.  Adoption of  the vision statement by the Administrator
will not be adequate to bring about needed changes: the same vision
Bust  also be  shared by  the AAs and  other  upper-  and mid-level
managers.    Demoralizing  rumors  of   lab  consolidation should  be
replaced  by   better  administration  of  decentralized  science.
Finally,  the vision must  bring  about  changes  in  attitudes  and
actions.

COSTS:  There are  no additional costs  to  be incurred by having the
Administrator affirm the vision statement and  establish a Committee
for  Quality Science to  advise  her.   Undoubtedly,   some actions
recommended by the Committee to improve the "climate, culture, and
incentives" will have investment costs: quality science does cost,
both  in terms of  funds  and personnel.   Nonetheless,  there  are a
number  of specific  recommendations  can  be  accomplished  with  no
additional costs.

-------
                 TEL:                     flug 27.93  9 = 23 No.001  P.11
BENEFITS:  The benefits to supporting the quality science vision are
significant. Quality science used to support Agency decision-making
will increase Agency credibility with the scientific community, the
public, impacted industries,  and Congress.  Improved credibility of
USEPA as a  science  organization with Congress and  OMB  will likely
lead to increased resources and greater flexibility.   The recogni-
tion resulting  from increased  public confidence will  improve work
force morale, which will, in turn, provide further improvements in
the Agency's rules  and decisions.   Only  when high-quality science
undergirds  the  Agency's  deliberations  can more effective and less
costly solutions be identified.  Better solutions  to environmental
problems will provide significant savings to the affected industries
and will  improve the  national economy while  still protecting our
health and  ecology.

MEASURES 0? SUCCESS: This initiative will be  successful only  if the
vision  is  put  into action.   The  ultimate goal is to  improve the
quality of  the  Agency's  science  —  not quantity — and quality is
much  more  difficult   to measure.   Actions recommended by  the
Committee for Quality  Science should each contain their  own measures
of  success.   The  success  of  the  overall  effort to  improve-the
quality  of science can  be  estimated  through periodic surveys- of
public  (including  scientific  community,  affected  industries, and
Congress) perceptions and confidence in the Agency's decisions.

    A.2   Procedures that Promote Scientific Excellence

WHAT:   Reaffirm USEPA's commitment to basic scientific  research,
peer review, quality assurance, Good Laboratory Practices, and other
policies that are  designed  to  promote excellence in science.

Sustained  excellence  in  science does not come about spontaneously.
Quality science must be promoted regularly across the Agency through
the application  and  continuing  refinement  of  procedures created
especially for  this purpose.

WHO:    Administrator,  Assistant   Administrators,   Regional
Administrators, Council  of  Science  Advisors

 TIKE LINE:  continuing;  evaluate  progress after 5  years

 BARRIERSt   Although the vast  majority of USEPA managers and their
 staffs  advocate   quality   science  in  principle,   considerable
 disagreement remains  as  to how that goal can  best be achieved.  Many
 regulatory officials are concerned that,  in some instances, expanded
 use of peer review and the other quality-promoting procedures will
 constrain operations unduly or require unnecessary expenditures of
 time and noney.  Further, even in instances when managers are eager
 to invoke these procedures, budgets and staffing  constraints often
 preclude it.   There  is currently more  emphasis  on  administrative
 procedures than on achieving flexibility.

 COSTS:  No adequate basis exists for estimating the absolute costs
 of  this  proposed  initiative.   Historical   USEPA-wide  records for


                                 10

-------
budgets and  staffing  levels  do not break out the costs associated
with oeer review and the other quality-promoting procedures.  OARM,
in cooperation with the program offices, ORD and the Regions, should
begin  a  systematic review of the  current dollars and  FTEs  that
constitute the  current operating level of these quality promoting
activities.  An estimate of the FTEs and dollars needed to eliminate
the  operating  shortfall  and  a  short-term strategy  for shifting
resources would be developed and implemented over  18 months.  After
the shortfall is erased, a systematic, nulti-year effort to increase
the  scope  and  intensity  of  the  quality-science  thrust could take
place.

BENZ7XT8I    Broader' and  more  intense  promotion  of  excellence in
science  will be  nanifest almost  immediately in the  quality and
credibility  of  the technical work-products that  support decision-
Saking throughout the Agency.  This should lead to regulations and
policies that are cost effective and thus gain broad acceptance from
the  regulated  communities.    And, if  decisions  are  easier  to
understand,  less  likely to invite  legal  challenge, and less likely
to be  reversed in court  when  challenged,  then costs of  litigation
and  enforcement should go down.

MEASURES  OF SUCCESS:  Success would be indicated by  progressively
more  numerous  decisions  that  win   broad  support   for  their
cost/effectiveness in achieving environmental protection.

    A.3    Science Advice within Program  and Regional Offices

WHAT:  Continuing the development of a  strong science advice program
within the Agency.

One   role  of  science   in  EPA  is  to  reduce  uncertainties  in
environmental decision-making. Because laboratory-animal testing is
the  nodel used most  frequently  to predict the potential for  human
health effects due  to exposure to  a  chemical pollutant,  there  is
uncertainty in extrapolating from effects seen in  laboratory animals
to predict effects in humans.  Other areas of uncertainty in policy
 decision-making  arise  as  a  result  of  limited knowledge  about
 exposure and the effect  of numerous chemicals or chemical mixtures
 on environmental components.

 While policy  decisions  require a  strong science base  to reduce
 uncertainty;   USEPA  policy   or  regulatory  decisions  are  often
 Shelved is not based on the relevant science.   The science advice
 Knc?ion?  ensuring  that policy  decisions are  based  on  •  c ear
 understanding  of the relevant science, is currently not well-defined
 S?  systematically organized within USEPA.    This would fnv'ude a
 science advisor  within  ORD,  each  program office «nd region.   The
 science  advisor'*  function,  analogous  to that  of  an  occ  legal
 advisor to  a specific project or program, is to ensure 'that USEPJ
 policy  decisions  and  regulatory  actions  are  based  on  «uP«r^
 Science  and a clear  interpretation  of  relevant science  and are
 scientifically defensible.
                                  11

-------
                   TEL:                      flug 27.93  9 = 23 No .001  P.13
The science  advisor within  ORD,  each  program  office and  Region
should be a senior scientist  or recognized expert who works closely
with  the Science  Advisor  to the  Administrator and  serves  as  a
program-office liaison to the Agency's Science Advisory Board.

WHOi  Science Advisor within ORD and Each Program Office

TIKI LINZi  Develop and implement plan by fall 1994

BARRIERSi  Decision  makers  nay  object to what  is perceived as an
additional layer of bureaucratic oversight.   Programs which do not
function  in traditional  line management  approach  may   find the
additional  review  inefficient   in  meeting  statutory  deadlines.
Resource allocation  for science  advisor positions may compete for
continuing education, training, and travel for program office career
scientists.

COSTS:   Variable, but  a  senior scientist  or science expert may
require GM/GS-15 and above salary range.

BENEFITS:  The development of a strong science advice program within
the Agency would  result in  consistency in Agency policy  decision-
making  and  the  perception  that  USEPA  is  a  "science"  Agency.
Decisions in one program office would  not adversely impact those in
other program offices.  Another  benefit would be more direct use of
ORD science  and less reliance on contractors to program offices.
Career scientists would view their contribution  to science at USEPA
as valuable, if work products were reviewed  by a well-trained peer.
This would  increase  morale  and  incentives to do superior science.
Then  the public  would  be less  likely to challenge  USEPA  policy
decisions,  avoiding  costly  delays  (economic  and  environmental
quality)   in  implementation.    Additionally,   enhanced  career
development and recognition would be provided for career scientists,
both  vithin and  outside  the  Agency,  for  their contribution to
excellence in science.

MEASURES OF SUCCESS:  Policy decisions based  on clear interpretation
of science leading to more consistency  in decision-making.  Public
support,   news  media  perception   enhanced,  state  and    local
environmental  sectors  support  USEPA  decisions.    Congressional
support, increased resource allocations.

    A. 4    geienee/Congrsssional-Liaison Team

WHAT:  Establish  a Congressional-liaison team representing Quality
Science  throughout the Agency.

The team would communicate  actively  with Congress  and  its  staff
about  important issues of environmental  science.   Science issues
should be communicated by scientifically-trained experts, not left
solely to Public Relations personnel.   USEPA should identify topic-
area  technical  experts, train them  to interact  with Congress, and
encourage  them to communicate with Congress about  their areas of
technical expertise.  Only when Congress and their staff understand


                                 12

-------
                   T£|_:                      nug^r.ao   y • ^ J H u . u« i r . . - •
the  scientific  and   technical  issues  behind  today's  critical
environmental problems can they deal effectively with these complex
environmental problems.  USEPA scientists should play a major role
in educating Congressional staff about environmental science.

WHO:   The Assistant  Administrator for Research  and Development,
together  with the  Science Advisory  Board,  should  champion  this
•ffort; however, the team should represent the interests of science
throughout the Agency, including the Program Offices and Regions.
Topic-area *experts  should be drawn  from all sectors of the Agency.

TIKE LINEi   The  liaison staff would be established as part of the
National Performance Review effort to reinvent government and should
remain a permanent  part of USEPA.

BARRIERS:  There are possible legal restrictions against "lobbying"
Congress.  Controls may need to  be  established to ensure that this
initiative is clearly a component of public education, outreach and
communications.   Other science  agencies  seem to  be able  to use
articulate,  respected scientists to  communicate  effectively  with
Congress:   USEPA should  do no  less.   Potential  barriers  may be
overcome by using NOAA or NASA as a model.

COSTS:  Costs will be minimal.  Current public relations or liaison
staff could be re-assigned and augmented with a few  additional FTEs.
Topic-area experts would be recruited from current scientific staff,
trained to communicate effectively  with Congress and the public at
reasonable costs and provided travel funds.  The topic-area experts
would not be permanently  assigned to the  Liaison Team,  since they
have to  remain involved with their research programs  in order to
retain their expertise and credibility.  Programs sponsored by the
liaison  staff (for example,  periodic briefings  for Congressional
staff on critical  environmental issues) should  not be expensive.
Travel  costs for  regular visits  by topic-area experts  to USEPA
Headquarters  and Congress  could be absorbed within the Agency's
current budget.  •

82VZ7XT6:   Two  major benefits  will be:  enhanced  recognition by
Congress of  the  high  quality  of  scientific and technical expertise
present in USEPA; and improved environmental legislation.  USEPA has
the  foremost experts  in many  areas of environmental science, and
this effort will help them be  recognized for  their expertise by
Congress, other  federal sectors, and the public. . As USEPA becomes
respected as a science agency by Congress, it will  be treated more
like a science agency.  This effort v*!! also help  educate Congress
(and Congressional  staff and the  public) about the scientific issues
involved  in today's  environmental  problems.     Such  increased
understanding, together with the  availability of  identifiable topic-
area experts to provide counsel and advice,  should  make for  better,
aore-effectiva,   and  less-costly  environmental  legislation.
Legislation that is well-founded in science will  provide substantial
savings  for the U.S. through more effective programs  with  fewer
ineffective,  but costly,  controls.   Better legislation will also
save considerable time and effort by reducing the political  battles


                                 i-a

-------
                   TEL:                      Rug 27.93  9 = 25 iso.uui  r.i5
 entailed  in   frequent  revisions  and   re-authorizations  of
 environmental  legislation.

 KEXSUR2S OF 80CC888:  The goal of enhanced recognition of USEPA as
 the   premier  environmental   science   agency  by   Congress  and
 Congressional  staff  can be  assessed  through periodic  surveys
 Success should also be  reflected in increased requests for Agency
 scientists to  testify before Congress  on  technical environmental
 issues.
B.  Science Policy
    B.I    Preparation  of  Policy-Makers  for  the  Science/Policy
Interface                                                        J

WHAT i    Establish  a  training  program  to  help  policy  makers
communicate more effectively with scientists.

EPA  policy makers  may  fail  to indicate  the specific  technical
questions  that  need to be answered to  achieve a  particular risk-
aanagement objective; and scientists charged with preparing relevant
analyses  may  fail  to seek  such clarification.    Even when  both
parties are of one mind as to the key questions, the decision-making
process  often does  not  allow sufficient  time  for  scientists  to
prepare a thorough assessment.  Moreover, scientists rarely have the
opportunity to help determine how their analyses  actually are used
to shape policy.

When any of these scenarios occurs, EPA science  is not kept in the
decision process.  Over time, some scientists have  come to feel that
they are not an integral part of the decision-making process.   The
latter is a commonplace outcome whenever scientists focus on issues
they personally believe are important and later learn that the risk
managers  consider   those  issues  to  be peripheral  to  the  main
question (s) .

WHO:  Agency-wide work group  appointed by the Administrator; review
by Council of Science Advisors

TIKI LXHEs  Training program should be developed  and instituted by
the spring of 1994.

BAARIERSt   The  idea of  yet another  training  program  may  meet
resistance in principle,  especially from those who feel that prior
efforts  in  other   subject-matter  areas  such  as  Total  Quality
Management  have  diverted   staff  from important   work   without
commensurate positive effect on the quality of the decisions being
nade.  Others may believe that the recommended training would merely
restate at length ideas that really are only  common  sense.   still
others may  fear that, by  adopting the recommended  communication
practices  and  procedures,   they would  be  inviting  unwarranted
incursion into their programmatic discretion.
                                14

-------
 Policy makers will need to be assured that the 'objective is not t«
 lrl£*nt S?:Jr decHi8Kior»a*i"9 prerogatives but rather ?o en™re !
 proper relation and balance between science and policy.  TO promote
 cooperation by policy makers  in  the training,  it Bust incorporate
 substantial  flexibility  and  use  the   time   of  policy  Sakers
 efficiently.  The best time for an  individual to take theVSpoHS

                                                   "
       4   Jhf  C08t  °f d«v«l°Pi"9  the training  program would  be
 approximately one person-year of effort.  This could be accommodated
 within the normal duties of the staff involved.            wnca««a

 BENEFITS:  If policy makers were to make even  modest modifications
 in their standard  practices and  procedures  in  response  to  the
 recommended training,  the  benefits could  be  very  large.    When
 scientific analyses are properly  focused on the  precise questions
 presented by  the enabling  statute  or regulations,  the  resultant
 decision will  be  less vulnerable during judicial review,  scientists
 also  will be  more productive and not waste time and  resources ~on
 peripheral issues.   If  outside scientists  and the public believe
 that  EPA  scientific analyses have  not been unduly influenced  by
 policy/political requirements, the credibility of EPA decisions will
 be enhanced.   Moreover,  the  benefits to the public of even a single
 regulation or action that   is  more  cost-effective  in  achieving
 Environmental  goals than otherwise would have been the case in the
 absence  of the proposed  training program would far exceed the  cost
 of developing  and implementing  the proposed program.

 MEASURES  O?  SUCCESS:    Because  the  recommended procedures for
 communication   across  the   science/policy  interface  are largely
 objective,  specific decisions can be examined to determine whether
 policy  nakers  and scientists .have  followed them.   For example
 scientists can be asked whether  they were advised early  in the
 regulatory process concerning the specific questions their analyses
 should focus on.  In addition to such objective measures, changes  in
 attitudes will  also  be  important.    If  the  program  succeeds
 scientists will be less  likely to complain that their work has  been
 distorted by policy requirements and  policy makers  will express
 greater  confidence in the objectivity of the analyses they receive.
 This would result in a more  coherent and relevant research agenda.

    8.2     Preparation of Scientists for the Science/Policy Interface

WHAT:  Establish a training program for scientists to assist them  in
 communicating  effectively and appropriately with policy makers.

USEPA needs  the best scientific and technical  information to make
credible  and scientifically  defensible  decisions.  To ensure that,
the scientist must present to the policy maker a comprehensive risk
characterization  (scientific analysis) and not  just numbers.

The scientist  must present  both  the qualitative  and quantitative
features  of the assessment and  also identify any uncertainties  in

-------
the assessment based on confidence in the data and methods used to
develop  assessments.    The  scientist should  present  consistent
assessments  of  risk  ranges,  principal  assumptions  SnS
                              ,                 upons    n   teir
 underlying rationale.  Also, where possible,  he/she should quantify
 the uncertainties along with comment about their influence  on  the


 The policy nakers should specify whether the  traditional scientific
 extrapolation  models are  appropriate  in  specific cases;  strive  for
 e55^pee? r"viev'  both inter-  and  intra-agency; and  identify when
 additional information would be  beneficial.    Scientists  should
 stress that the risk assessment process employs a varying number of
 judgmental  positions   to  help  bridge  gaps  in  data  and  in
 understanding  of  underlying biological processes.   Risk  assessors
 should not attempt to overstate  or understate risks, but  rather oive
 estimates  and  the range of uncertainty.

 WHO i  Agency-wide work group appointed by the Administrator;
 review by  the  Council of Science Advisors

 TIME LINE:  Develop the guidance by fall of 1993.

 BARRIERS:   Some  policy  makers  seem  reluctant to  interface with
 program office or ORD scientists prior to communicating major  risk-
 based regulatory decisions to the public.  Also, they may choose  not
 to  reveal  the  degree of  uncertainty or what  influence uncertainty
 had on their final  decisions.   At  times, scientists are not  given
 adequate  time  to  collect  acceptable  data  to perform credible
 assessments.  Research scientists are often brought  into the process
 very late,  as  reviewers,  rather than  early on, as  developers.  At
 times,  USEPA  is  pressured by  external forces  into making rush
 decisions that are not based on good science.  Many times scientists
 are not  advised  what specific questions the decision makers need
 answered early on in the process. Some policy makers are resistant
 to  involving scientists in  the  decision  making process  because of
 their  perception that  scientists  should  be  excluded   from  risk
 management-based policy decisions.

 COST:  $100,000 for workshops, consultants and contractor assistance
 to  develop the guidance document (s).

 BENEFITS:   Adherence  to  Agency wide guidance  will  improve the
 understanding  of  Agency risk assessments, lead to more informed
 decisions,  and improve  the credibility of Agency  assessments and
 decisions and ultimately decrease costly challenges to regulations
 and enforcement actions.  Recognition of USEPA risk assessments will
 improve scientists morale  which  in turn will lead to their enhanced
productivity.

MEASURES or SUCCESS:  Decisions based on credible risk assessments
will decrease  the  number of  challenges from  environmentalists,
 industries and the public.

-------
    8.3    CASE BOOK on the Science/Policy Interface

 WHAT:  Create  CASE BOOK, a collection of agency  cases  documented

                                                            ~
    Exemplary cases of science and policy interfacing.
    Examples of problem solving approaches.
    Orientation workbook for risk assessors and risk managers.
    A guide to help new employees assess and manage risks.
    A primary reference for science/policy approaches.

 WHOi  Council of  Science Advisors.  Contributions to CASE BOOK will
 be prepared by  each program, that is, the offices serving under the
 various statues,  (e.g.,  TSCA,  FIFRA,  etc.).             9

 TIME  LINE:   Continuing;  evaluate program after 2 years. The time
 for completion  of the first edition of CASE BOOK will be one year
 after the  agreement  that  this  proposal  is  to 'be  accepted  and
 Implemented. Approximately,  September 1994.

 BARRIERS:  Some risk managers and risk assessors may be  reluctant to
 relinquish  their  analysis  for  agency  wide  review.  Additionally, a
 manual  that is circumspect of  decisions  may  cause  paralysis  of
 analysis, that  is,  CASE BOOK may cause risk assessors/managers to
 equivocate  rather than make well-timed decisions.  These  are common
 barriers and should not prevent the  idea of the CASE BOOK  from being
 successful.     Most   scientific   organizations   compile  reference
 information for  use when  making   risk  assessments  and  doing
 research/experiments.

 COSTS:  $25,000 for consultants and publication assistance.

 BENEFITS!    Circulation  of   a  high  quality  manual  containing
 instructive Science and Policy Case Studies will benefit the agency .
 CASE BOOK will  provide substantive examples of what is expected of
 risk assessors and risk managers.  Specifically,  risks assessors and
 risks managers who know before  hand what is expected of  them have an
 advantage over  those that do  not.    Communication  is  critical  in
 order to facilitate problem solving.  CASE BOOK will be  a lexicon of
 Science and Policy and  reveal the  process of  science and  policy
 interfacing.                                                r    '
                                                    •
A well-prepared inventory of science and policy  issues will require
 few resources compared to  the  benefits.   Circulation of exemplary
 science and policy decisions will raise the morale of Scientists and
Policy makers.  The agency  standards  will  be open to scrutiny  and
 forced to stand the  test  of critique from  agency  scientists  and
policy nakers.  The peer review process, good laboratory practices
and other practices  designed to promote quality science would  be
enhanced  by CASE BOOK.   Promoting  quality  science  and  policy
interfacing  and publishing  examples  will gain support  from  the
Congress and the public through better risks communication.
                                17

-------
                   TEL:                      Pug  27,93  9 = 23 No.001 ?.19
 CASE BOOK will  be an excellent tool for orientation of Risk managers
 and Risk assessors, both new and existing employees.  In concision
 the substance of CASE BOOK is already contained within the
 used and cited in the continual efforts to manage and asselJ risk
 Feedback and  input into CASE BOOK will identify the effectiveness of
 the document.  Commentary sections on each case  cited in CASE BOOK
 SS5dJ!S?l?Wl*??tJtht ««"tiveness of the document.   Similar-Tv  °*
 CASE BOOK IS  Cited as a guideline in future examples  of Science and
 C9UCY Interface in  the  agency.  we will know it  is working.

 Subgroups *a"geBent  of  »i«««ntUl   Risks   among   Population


 WHAT!     Enhance  the capabilities  of   health-risk   assessors  to
 differentiate among  population subgroups wherever  practical  by
 developing a  long-term research plan and associated risk-assessment
 strategy.

 Attempts to estimate  the likelihood of adverse human-health effects
 from known or  putative  environmental  hazards  traditionally  have
 focused on the risks  faced by  the  population  on the average;  the
 principal exception  has  been selected reference concentrations  or
 doses  that are calculated using arbitrary uncertainty factors  to
 protect  potentially  sensitive  populations.     yet  experience
 increasingly  is  making  clear  that,  for  any  given  hazard,   some
 population subgroups  may  be  at substantially greater  risks  than
 others  as a consequence  of stage of development, gender, or other
 manifestation of genetic make-up.   Moreover, current research  in
 cell  biology, molecular  genetics, and  epidemiology  is offering
 evermore powerful means  to assess  subgroup-specific  risks;  and an
 jxpanded •ffort would yield significant return.   The time has  come
 for USEPA risk assessors,  as well as the  other scientists whose  work
 supports them,  t.o obtain and use  subgroup-specific  data wherever
 practical  as  a matter  of policy.

 For  example,  in  view of  the  possibility that  infants and children
 aay be  a greater risk from certain pesticides than are adults, the
 National  Academy of  Sciences  recently  recommended  revamping the
 pesticide-registration process so that it  includes better tests and
 better data relevant  to this subgroup.   This line of reasoning can
 be extended to include the possible differential vulnerabilities of
 infants  and  children to  hazards  other  than   pesticides.    And
 essentially the  same  logic  can be applied  to other  population
 subgroups  under  various  circumstances  —  especially  the elderly
women,  and racial and ethnic minorities  — for a wide spectrum of
 environmental  hazards.

WHO: The Administrator could appoint an agency-wide working group to
 develop  a  two-part  product:  (l) a  research agenda to expand the
                                10

-------
 knowledge base  relevant  to  subgroup-specific risks  and  f•>^
 TIKE LZKEi  Appointment  of working group by January, 1994.


                                  ^
 costs and time required for risk assessments could rise unduly

 COSTSi   $100,000  for a workshop  and consultant fees  related
 developing  the  research  agenda  and  upgrading  !in?mm
 strategy.  USEPA staff costs' would be absorbed Is 'part '""
BENEFITS: The initiative would introduce a new level of scientific
rigor  and   sophistication  into  health-risk  assessment     Trie
initiative also  would stimulate research toward  better  tests and
better data pertinent to differential risks.  Subgroup- spec if *ic 'risk
   '                     °  btter tailorin9' of protuepct?v  measure's
 f                  *                                      measures
 J!*i?£  y?CJ£l 1eedS  °f  vulne^abl«  subgroups,  to  improved public
 health,  and thus to more cost/effective regulation.          P^DHC
 MEASURES OF SUCCESS:  Progressively more emphasis on  subgroups  in
                                                 of
 C.   MurturinQ Quality Se
    C.I   Job Knbaneemeat and  Resource Support for Scientists

                                            to
   *   K        in Pronotin9 quality science at USEPA is to ensure
that  th«  jobs  we  create  are  of  high  quality  - stimulating
rewarding,. and as free from administrative distractions as pSssibll'
This Bay require shifts in job assignments among other components of
the work force.  More scientific projects should be done  in-hSSse to
enable our scientists to share scientific contributions  and to link
science to Agency policy,  rather than merely reviewing  the work Sf
others, as with contract activities.
              * BU8t;  fin$V5e that' as W€ nurtur« quality scientists
SIMM   *    Ve  prfvid«  th
-------
                   TEL:                      Rug 27.93  9 = 23 No.001 P.21
 procedures?  ^  pr°Vided  f°r lamenting  these  standards  and


 WHO:   Scientists  and  engineers throughout the Agency

 TIKE  lIMEi   By fall of 1994,  review all relevant job assignments-
 develop and implement plan to decrease reliance on contract lupport •
 develop and  implement set of standards and QA plans  for scientific

                                rMOV*  instit^io"^ barriers  to
 BARRIERS:  There  are  currently many inconsistencies throughout the
 Agency  on  the functions and grade levels of scientists.  Although
 ORD has a  career  advancement systera in place, it cannot be applied
 to  other  USEPA program  scientists  due  to differing  duties and
 functions.  Limitations on federal positions and resulting reliance
 on  contractors  continues  to  be  a  serious problem.    In-house
 expertise  and  public confidence  in  Agency decisions cannot  be
 nurtured  while  amen  of  the   scientific  work  is  delegated  to
 contractors.   While ethical standards  exist, their emphasis  is on
 legal  and regulatory constraints rather than on opportunities.
 Similarly, QA is  being developed for" laboratory procedures  rather
 than  as a framework  for  Agency  scientists  to   produce  quality
 products.   Constraints on  travel  have  a particularly devastating
 effect  on  scientists, since exchange of scientific information and
 the enhancement of the Agency's image are the two primary objectives
 of  participation  at  environmental conferences  and  professional
 meetings.

 COSTS i  Converting much of the budget for the contract work to in-
 house resources should have no effect on the budget.   Establish a
 travel budget of at least §15,000 per  scientist.

 BENEFITS:  Decreased staff turnover and increased  job satisfaction
 throughout the Agency

MEASURES  0?  SUCCESS i    Increased public   confidence  in  Agency
decisions,  policies, and regulations and  increased representation of
 EPA scientists on national panels.

   C.Z    Career Ladders and FroaotioB/Xncentive/Revard  Systems

WHATi    Create r system  *hat  links  grade assignments  to  the
contributions of the individual.

A traditional approach to  position classification  is  no  longer
responsive to the Agency's need for flexibility  in  creating career
ladders linked to organizational  goals  and individual  achievement.
If we  agree  that "science" forms  the  backbone of  this  Agency,
scientists should  be  able to reach senior  levels (through  their
 scientific" contributions and achievements)  in the Agency,  without
having to assume "Management" positions.   Promotion systems should
provide the structure and criteria  for  advancement, but  should not
liait  then  with artificial ceilings (glass or otherwise) .

-------
 Aspects  of the  Merit Promotion Program should  be  re-eyamin-^ <
 order  to support the  concept  that  jobs  are created bvamn^    "
 based  upon programs tic  needs,  rather than £ploy..V

                                          °f  th' W- or
                        each Program, of f ice vd laboratory adopt thi
       -
            ^""^  the «nnu.l appraisal  cycle).   Reward  sstems
            more competitive, innovative and I prestigiousf and  ll™
 routing  "bean-oriented" and  automated.    New awlrd  eatagerlM
 nomination   (including  external   nominations)  and   peer   wvili
 procedures nay be  incorporated.   Incentives  and  rewards  may  be
 «K;??edithr°U?VeliOWShlpS' adv^or/ instructor assignments? min?!
 J?!    5al8'  8Vt4.e"°f'the  art labo«tory equipment,   rotations  or
 time and support to pursue other USEPA aission-oriented  endeavor^.

 WHO:   Task Force  with cross  sectio'n  of  USEPA scientists, Human
 Resources  experts  and  1st  t  2nd  line supervisors  (particular^
 "Science" Managers - scientists managing  scientists)         «i«riy

 TIME LINE:  Develop Agency-wide policy for career paths,  promotions
 and incentive/reward systems to be  implemented  in FY95.

 SS??ivRLSo  "ana5ement  may  have reservations about implementing  a
 policy due  to potential budgetary impacts.  If "benchmarking" all  of
 the various scientific  positions within the Agency  is  used in order
 to  develop  the criteria, this could  be  a very cumbersome process and
 it  may end up falling under its own weight - well before a policy
 statement could be  developed.   There may also be some  hurdlei with
 0PM to get  over.  Supervisors may feel that this only presents more
 obstacles for them.  They would need to receive extensive support
 (training,  incentives,  time,  etc.) to  do this.   It  would also
 require time and resource support to develop the policy and criteria
 for this  type of personnel management system.

 COSTS:     Development:   $100,000,  plus   6  FTEs  for   a   vear-
 lap lamentation:  $75,000 for  training agency-wide
          *  Dec""ed «ta"  turnover;  it's  getting increasingly
difficult for the Agency to recruit and retain quality scientists.
More  emphasis on  rewards rather  than punishments  would improve
morale.* Increased  job satisfaction and sense of career direction
SS?I!a?C?  Perc«Ption»  of  "hidden"   rules  for  advancement  of
ISJ! 5*JT*    ^hc  A9«ncy.   increased importance placed  upon our
scientific workforce; increased pride.  Improvement in the quality
Of science in USEPA.
                                21

-------
                    TEL:                      flug  27.93   9 = 23 No .001 P.23
KZASURZS 07 SUCCESS:  An Agency-vide Policy and Criteria in place.
Adoption  by  all USEPA  programs  (HQ's  and  Regions)  and  labs.
Increased job satisfaction,  pride and sense of career/direction -as
gained through regular employee surveys.

   C.3    Employee-Empowered Scientific Recognition Committee

WHAT:   Improve the way  we  reward scientists and  technicians for
innovation,  creativity  and continuing  scientific excellence  by
iStablishing employee-empowered scientific Recognition Committees at
the Divisional level.

WHO:  Create a "reinvention  lab" to test this concept.  This do-able
is modeled on an existing committee in ORD/HERL/DTD  (contact person
Dr.  Robert  Kavlock) ,  but  should  be  extended  to other sectors
 (Regions, HQ's)  of  USEPA

TIKE LIME:  The model  for this concept became operative vithin about
 6 months of conception and  is now in its second year.  Briery, the
 Division Director appoints a chairperson and, with ^the chair's help,
 drafts  a committee consisting of 2  technical  staff, 2  principal
 investigators  (non-supervisory)  and  two  supervisors  n(tWelted II
 members replaced each year).   Award nominations are •olicij ted at
 ?east twice a year-anyone can submit a nomination at any time. The
            is  convened  as   nominations  come  in.      It   maxes
                ^
                £eK£u£.  but i.P consistent with  «*"£«»»*

                STS»^
 groups within USEPA but each group would  have  to  decide their own
 criteria.
 BXWMZRS-    Inflexible  interpretation  of  current  USEPA  award
 Si?"?*!  Resistance of managers to  empower staff at all level, of
 the organization.
 COSTS:    None:    funds  for  these  awards would  come from  the
 established award pool (for on-the-spot and special acts).
 BBKBTITB-   Two years'  experience with such a committee in DTD has
                                  22

-------
competition   and  eliminates   the  suspicion  of   "supervisor's
favorites."   5)   A potential  long range goal  would  be to use this
approach  to recognize  and  reward good science,  and  remove cash
awards associated with  the official performance appraisal process.
Then,  the  performance  review  — unencumbered  by  the cloak  of
financial intrigue — could be used productively to evaluate whether
goals have  been aet and to set new goals  and IDPs  for the coming
year.

MEASURES OP SUCCESS:  Annual review at  DTD staff meeting to obtain
feedback on success of recognition program.  Increased productivity
measured by established methods.

    C.4    Training Tor  Scientists

WHAT:   Establish  innovative  career-development program to ensure
that scientists remain  challenged and stimulated while  fostering  a
greater  exchange  and interaction between USEPA and the academic
research  communities.

The Agency  needs to develop a  system that  maximizes the human
potential of  our scientists and  thus  the quality  of science at
USEPA.  The system should establish, in  advance, a cohesive training
plan and  budget for each  position that  is filled.

The prooram will require the establishment of a team of scientists
 (working  with Human Resources) to develop a step-by-step mapping of
advance career-development activities for each scientific position
which  should include: (1) methods to ensure" active  participation in
the rotation of USEPA scientists to outside science organizations to
Soaden the knowledge base of the individual,  as well as the Agency;
 (C financial support by  the Agency  for  employee membership to
 scientific  professional  organizations;  (3)   ^ablishment  of an
Aaencv-wide  long-term training  program  for scientists  who  are
 invested in advanced training and remaining with the Agency;  and
 Sr^tablishment of agreements with local colleges and universities
 to offer a wider variety of scientific  courses at USEPA.

 WHO:  USEPA management

 TIKE LIKEt  Develop training plans for scientific positions by April
 1994 and begin  implementation by September 1994.

 BARBXERSf   Tight  budget constraints  exist,  and  USEPA  lacks  the
 Support  of eonVr... to provide  the needed funding.   Manager, may
 Suspending funds on training scientists and opt for •P«nding  it
 on travel or oversight of  extramural projects  (especially  non-
 SSientifTc manner-) -    Manager, may  discourage training  when  it
 involves  allowing the  employee to use  government time  and may make
 it difficult  for  scientist,   to  participate  in   a rotational
 assignments:  Aggressive means of exchanging technical information
 "?hTnt7rn;tio9n9al  scientist.,  especially   if it  ^olves travel
 funds  or  other resources,  nay  be  viewed as only benefiting the
 scientist, and not the manager or the  Agency.


                                  23

-------
                    LL :                      nag ^fi^O  -3-^0 iNO.uoi
COSTSt    Development:  $200,000,  plus 4  FTEs;  Implementation:  $0
rotational assignments;  $50 fee for  each  member of  professional
associations; $300-1000/training course.

BENETITSs  Recognition of USEPA  as  a  high-quality  science Agency;
development of a strong base of top quality, stimulated scientists;
retention  of  quality scientists  with  institutional  knowledge;
decrease  the  turnover  rate   of   quality  scientists;  increase
productivity  of  quality   scientific  outputs   through  advanced
training; and, eliminate partial training of select scientists.

MEASURES O?  SUCCESS:   Gain public  confidence; decrease turn-over
rate of  quality scientists; increased morale; produce defensible
quality outputs; decrease in number  of scientific challenges to the
Agency.

    C.5    Recruitment of Scientists

WEATs    Develop  a  short-term  and  long-term  strategy  for  the
recruitment   and   retention  of  qualified  in-house  scientists,
engineers and technicians.                                      ~

Such  a  strategy  is  crucial  if USEPA  is   to  achieve  status  and
credibility  as  a  "Science  Agency."   Deficiencies in sustaining an
adequate scientific staff are  increasingly  evident as the Agency is
rapidly  losing   its   institutional  technical  knowledge  through
attrition.   The strategy must  sanction  the recruitment of junior
scientists,  engineers and technicians, who,  under the mentorship of
senior scientists, will conduct  in-house research and develop into
Agency experts.

Participation in the recruitment and hiring processes by peer-level
scientists  from  inside  and outside  the Agency  should be  a  key
element of the recruitment policy.  The strategy must also encourage
cooperation  and coordination with external  researchers.  USEPA must
include  in its  long-term recruitment  strategy a more sophisticated
plan   for   improving  cultural   diversity,  including  fostering
relationships with colleges and universities to attract top-notch
recruits.  Another important part of the recruitment strategy should
include  the  placement (by  recruitment or promotion  from within) of
several  world-class scientists  (including current EPA experts) into
key policy  and  decision-Baking positions, as  well  as critical
research areas that are essential to reducing uncertainties in USEPA
decision-Baking.

WHO:   Task  force consisting  of key  Agency personnel and outside
scientists appointed by  the Administrator

TIME LINE:    Develop  an  implementation plan by the 2nd quarter  of
 1994

BARRIERS:     Barriers  to  the  successful   implementation  of this
recruitment  strategy  include  the  following: limited  FTE  ceiling,
 liaited dollars  for FTEs  allocated  by  Congress,  non-competitive


                                 24

-------
salaries compared to industry,  limited  career tracks and inflexible
Human Resource Management guidelines for scientists, procurement and
ethics regulations that restrict communication between scientists,
limited travel budgets, limited and un-focused training program for
scientists  across the Agency, recent  policy to  reduce 300*  GS
14/15/SES  positions,  recent  policy  to  suspend cash  performance
awards  for FY94  through  FY98,  very  low morale  of the  current
technical staff.

COSTS:   The political will to approach  Congress  and OMB  with  a
request  for a significant  increase  in FTE ceiling.   Part  of the
argument should address switching R4D dollars for PRO dollars to pay
for FTE  ceiling increase.   This  approach would not  increase the
total dollars allocated to the Agency.

BENEFITSi    The  following benefits  would  be   derived  from  the
implementation of this recruitment strategy: increased respect for
Agency  decisions,  increased  respect  for  Agency  scientists  and
engineers,  increased  confidence in  Agency data,  recognition and
acceptance  of USEPA as a science  agency,  increased morale  and
productivity within  the Agency technical workforce.

MEASURES OF SUCCESSt  The ultimate  measures of success would result
from environmental scientists around  the world recognizing USEPA as
the source of expertise in environmental science and technology, the
primary training ground  in  environmental science and technology,  a
reliable  source  of  quality  environmental  data,  a  leader  in
environmental methods and technology  development, and a promulgates
of regulations with  sound technical basis.

D.  Communication and  Coordination '

   0.1    Prioritisation   of   X&tra-Agency  Communication   and
Coordination

WHATi  Give special emphasis to communication and coordination of
scientific  activities  within USEPA.

Effective intra-Agency communication and coordination is essential
to achieve high-quality science products (research,  regulations,
guidelines,  policies  and  decisions).    The  scientific  resources
throughout  the Agency must be identified to facilitate direct and
productive  interaction of scientists  working on related issues.  In
addition,   readily   available  information on  projects/activities
currently  on-going  within the Agency (Headquarters,  Regions,  ORD-
Laboratories)  must  also  be identified and  periodically updated.
Xdentific tion of resources and projects/activities  will serve to
coordinate  scientific  efforts  within the Agency.

Thus, it becomes essential that coordinators be designated in all
programs to   foster  communication  among  scientists  and  avoid
replicate   efforts.     Communication   between  risk  managers  and
scientists  involved  in risk assessments requires  that risk managers
convey clearly what  information they need from scientists, so that


                                25

-------
                   T c L •                      n u 3 ^ r . 3 o
•dentists  can  respond  clearly  and  factually  (including  the
uncertainties  in  the  risk  assessment).    Training of  scientific
personnel in media communication should be  made  available (and in
some cases,  mandatory) to allow proper identification of the needs
and level of communication required by the  target  audience.   Non-
scientific  personnel  should  be encouraged  (and  in  some  cases,
required) to receive scientific training  as part  of  their career
development.

WHO:  USEPA-vide task initiated by the Administrator and implemented
at each program,  Region and ORD-Laboratory.

TIKE  LIME:   This -should  be a  continued  effort supported  by the
Agency.    The  first  phase  of  this  task  (identification  and
coordination of scientific resources and projects/programs) should
be in operation by Fall 1994.

BARRIERS:   Major  barriers  in  communication  and/or  coordination of
scientific efforts would be those imposed by statutory restrictions,
which would limit the amount and nature of the information that can
be  shared.   Although these  are intrinsic  and  may not  be easily
removed, the need for improved   communication  and  coordination of
scientific  efforts within  the Agency should be  recognized by the
Administrator, Assistant Administrators and other upper- and middle-
level managers as a vital key to achieve high-quality science.

COSTS:

a.  Low Cost- Identify (through  a survey)  the scientific resources
    within  the Agency.   Identify the components of  the different
    programs/projects  in Headquarters,  Regions  and  Laboratories.
    Create a document, database  or other form of transmission media
    for disseminating the  gathered  information,  with  periodical
    update of the information.   This could build on the Scientific
    Registry being piloted  by the Sci/Tech Committee.

b.  Medium   to  High  Cost-  Training  of  non-scientific  personnel;
    Training in risk communication; Training in communication skills.
    Designation of intra-agency  coordinators  in FTE positions.

BENEFITS!    The  benefits  of  communication  and  coordination of
scientific  activities within the Agency are  significant.  Replicate
•fforts  can be avoided, which would result in savings in time and
costs.   Improvement  in communication and  coordination within the
Agency (among  scientists and between scientists  and managers) will
improve  the quality  of science.  This will have  a direct  effect in
the  quality  of  the  decisions,  regulations,  research  and  risk
assessments generated by  the   Agency.    In  turn,   the  Agency's
credibility by  the  public,  the  scientific  community  in  other
agencies and academia, industry  and Congress will  improve.  The
increased credibility of the USEPA as a high quality science  agency
and a leader in all aspects of environmental science could result in
increased funding to further expand the Agency's capabilities.

-------
MEASURES O? SUCCESS:  The success of an improved communication and
coordination system within the Agency can be measured by the gains
in the quality of science products generated by the Agency.

The ultimate measure of this success would  by  the recognition by
ethers  (the scientific community,   industry,  other agencies  and
organizations, Congress and the general  public)  that the USEPA is a
high  quality  science agency  and the  leader  in  all  aspects  of
environmental science.                    *
                                        ^
   0.2    Zxchang*  of  Scientists (internally  ft  externally)  For
Purposes of Zxcharigiag  Science

WHAT:  Increase scientific contact and openness within USEPA as veil
as with other government and private sector  organizations.

The   coordinated  exchange  of   USEPA  scientists within   USEPA
laboratories, Headquarters, and the Regions,  as  well as among other
federal agencies,  academia,  and international organizations,  will
nake  its scientific and technical efforts more widely known.   Such
an  exchange  program  would  ensure  that  USEPA  scientists  are
continually exposed to  current knowledge and methodologies.
       •
To accomplish this the Agency should promote  and increase the number
of details and rotational opportunities  within USEPA.  USEPA should
require that every laboratory and program  office, whether in Region
or  in Headquarters,  mandate that  rotational assignments/details
outside of  a  scientists own immediate  program  take place  at some
optimal level of their career.   These rotational assignments should
last  for  a  minimum  of 30 days.   Administrative  and  management
procedures  should  be checked to  ensure that  no barriers  exist to
Inhibit these rotational/detail activities.

In promoting the exchange and sharing of science between USEPA and
other government and private sector organizations, the Agency should
implement the recommendation contained  in Safeguarding the Future;
Credible Science. Credible Decisions.. The Quality  of Scientists at
USEPA,  Recommendation   3  (pp.31-32).     The   details   for  the
implementation of this recommendation would be left to the Agency to
determine.

WHOt  USEPA-wide,  coordinated by  personnel offices in the Regions,
laboratories and Headquarter*.

TIKI  LIME:   Develop  an implementation plan  for the  intra-Agency
rotational/detail  assignments  by the  end  of  FY94.   Develop an
implementation  plan  for the exchange  of scientists between USEPA
and the outside scientific community by early FY95.

BARRIERSi   Coordination with other  programs  to allow  the optimal
exchanges will be complicated.   Some may object to the disruption of
their staff.  The rotations involving relocation may require special
attention,  due to  cost considerations.   Some entrenched employees
may object  to  the  mandatory nature of the rotations.  Concerns may


                                27

-------
                   TEL:                      Rug 2^,3.5  5:2.5 NO.UUi .-
exist over  the opening of  USEPA offices to  non-USEPA  employees,
especially over Confidential Business Information (CBI).

COSTS:    Costs  will  be  minimal  for  internal  rotations,  with
relocation  and per  diem costs  being  the  largest  out-of-pocket
expense.   Two  (2)  Billion  dollars  sight  be set  aside for  the
external rotation program for the first year;  that budget could be
altered as the success and size of  the program is determined.

BENEFITS!    Cross-program   understanding;  a  more  multi-media,
pragmatic workforce;  greater future  sharing  of information  as  a
result of networking.  All of these would help improve the quality
of science in the Agency.

XEA8URES  OF SUCCESS:    Improved  sharing  of  information  across
program; high  number of rotations/details  reflecting interest in
program;   better  relationship   with  other  governmental   and
private-sector organizations.

   D.3    Communication  and   Coordination   vith   Other  Federal
Agencies, state and Local Agencies, and International Organization

WHAT:   Create  frequent opportunities  for USEPA scientists  to be
exposed to  current  knowledge,  data,  and  methodologies  from other
agencies and organizations,  as  well as to  communicate  findings and
advice to them.

Among the federal agencies,  USEPA is fifth-in  the dollars spent on
environment research.  The  states have  five times more  scientists
monitoring  the  status  of the  environment than does USEPA.   Many
international organizations  (WHO, OECO, World  Bank,  Japan,  United
Nations, etc.)  look to USEPA for  scientific  advice.

Within  USEPA,   scientists  are  separated by  organization  (four
programs, ten regions, and  the Office  of  Research and Development
(ORD)) and location  (35+ laboratories facilities).  State agencies
often  cannot  explain  the   science  behind  the  regulations  and
regulatory actions to the public.  The public  is willing to comply
if  they  could  understand  the  rationale behind  the  Agency's
decisions.     Communication  of   USEPA  scientists  with   their
counterparts in international organizations, e.g., OECD,  is crucial
to the USEPA efforts  on guideline harmonization.  The communication
to other agencies/organizations should be  consistent, accurate, and
frequent to build credibility as a  strong  scientific agency.

USEPA should plan and co-host meetings and workshops on scientific
issues  with other   federal  agencies  which  senior USEPA  senior
management  would  be  expected to attend.    Scientists from across
USEPA would attend  in numbers  proportional to the  other federal
agencies.  Regions would host similar meetings  for states and field
offices for federal  agencies.  USEPA should develop a focal point
for  scientific  expertise   to  ensure  that  we provide  accurate,
current, consistent advice to international  organizations.
                                28

-------
WHO:  USEPA-vide task force appointed by the Administrator. The task
force vould  include  scientists from ORD, programs, and Regions.

TIKE  LINil   First  meeting planned for  February 1994  with full
implementation  plan  by October 1994.

BARRIERSi  The  greatest barrier  vould be the budget and managers'
approval to travel. Travel dollars to attend these types of meetings
do not  exist in the present  budget. The  present culture will not
invest  the time and  funds in this kind  of  long term investment.
There also would  be  resistance  to the formation of  another task
force.

COSTSs   For  four  national  meetings and  ten  regional  meetings per
year - $60,000. Travel funds for these meetings -  $500,000.

BENEFITSt    Improved  communication between  USEPA scientists  and
scientists   in   other   federal,   state,   and   international
agencies/organizations would result in more credible environmental
science, better compliance with  our regulations, and  reduction e?
wasted and duplicate efforts.                                    -

MEASURES OP  SUCCESSt   Survey developed by the task force for USEPA
scientists,   other  federal   agencies'   scientists,   and   state
scientists,  to  be completed  by  February  1994.   The  same  survey
instrument would be used again in February 1995 and  February  1997.

   D.4    Communication and Coordination vitb Academia and Industry

WRATs   Develop  Environmental  Technology Re-invention  Laboratories
based on collaborative efforts of the Agency with universities and
industry.

USEPA's environmental protection strategies are undergoing important
paradigm shifts.   The most important  include:  risk-based priority
setting,  pollution  prevention  as the   preferred  approach  over
pollution control, and increased attention to  reducing ecological
risk.

USEPA should encourage and legitimize  the open collaboration  of
Agency scientists  with academia and industry in the development of
innovative approaches to pollution prevention and  cleaner production
technology.   Such creative and  non-traditional  efforts  will help
ensure an American scientific, technological and industrial presence
'in the international  marketplace.  Open and unfettered communication
and coordination with industry, in particular, will serve as the key
to success.

VBOs  The Administrator's Science Advisor, in collaboration with the
Office of Prevention,  Pesticides and Toxic Substances  (OPPTS) with
its links  to industrial  and agricultural  chemistry  and Design for
Environment Program  (DfE), and the ORD Office of  Science,  Planning,
and Regulatory  Evaluation  (OSPRE) would serve as the Agency-broker
on communication  and coordination for a  FY94 pilot program.   The

-------
                   TEL:                     flug 27.93  9=25 Nc.OOi r.5i
program  would  be  conducted  jointly  with  the  National  Science
Foundation's  (NSF)  Environmentally  Benign Chemical  Synthesis  and
Processing  Program  and the NSF  Industrial-University  Cooperative
Research Center at the University of  Washington's Center for Process
Analytical Chemistry.  The industrial sector would be represented by
the Dow chemical - Perkin-Elmer  Strategic Alliance  Ventures Group
recently  established  to  extend  the  practical  use  of  advanced
analytical technology into applications of real-time industrial and
environmental monitoring.
                                                        «

TIKE LIMEs   Fifteen months to  produce a report to Administrator
Browner and Vice President Gore on  the  feasibility,  limits  and
opportunities  of  USEPA/NSF   coordinating  innovative  Design  for
Environment and Pollution Prevention programs with environmentally-
based advanced technology ventures in the  private sector.

BARRIERSt  Three types of  barriers to success  exist:

   Historical and cultural reluctance of USEPA to aggressively,
   openly,  and effectively collaborate as  equal  partner  with
   industrial  sector subject to  Agency  regulations, and  with
   technology-based  firms  who  may  benefit  front   close  and
   privileged Agency ties.

   Znteragency turf battles where USEPA and other science agencies,
   each protective of  their own  perceived sphere of influence and
   control, will stifle coordination.

   Intra-agency  resistance between  and among  Program  Offices  and
   ORO.   Current  USEPA measurement  and  monitoring programs  are
   predicated on pollution control with virtually no experience or
   expertise in pollutant transformation processes.   Reluctance of
   Air,  Water,  Solid Waste and  RCRA programs to adapt and adopt
   Performance-based  measurements  as opposed to  USEPA  Approved
   Methods-based measurements.

COSTSi    Extramural  investment  modest —  primarily  to  support
meetings and report development and production plus travel budget to
Dow,  Perkin-Elmer  facilities  and to  CPAC.    Intramural  resources
modest — senior scientists  from Administrator,  Program  and  ORD
Offices and Labs, plus travel  budget to support staff.

BENEFITS<  Effective and efficient communication and coordination of
pollution prevention activities between the Agency and the industry-
university  communities will significantly leverage  modest federal
resources with  private sector  investments.   Creating a  mutually
supportive culture between USEPA and industry supportive of national
environmental  and economic  goals  will  reap long-term  economic
benefits.

MEASURES  OF SUCCESS:   The   success  of  effective  and  efficient
communication and coordination of this effort may be measured in a
series of agreements between  USEPA  and industry to  adopt  specific
advanced analytical technology to replace outdated and costly Agency


                                30

-------
methods  and procedures.   The October  1994  report would  clearly
identify quantifiable measures of savings, where appropriate! or the
generation  of better  data for  decisionmaking and compliance  at
comparable costs.

   D.s    Communication and Coordination  with  Congress,  OK£,  Kevs
Media, and the General Public

WHAT:  Develop new and improved means to present its science-based
activities 'to lay audiences.

USZPA's  greater emphasis  on  cross-media planning  and  scientific
•fforts will result  in a more  coherent and efficient effort by the
Agency to resolve complex environmental problems.  This cross-media
emphasis will require enhanced cross-media budgeting. This presents
a serious communications challenge with respect to OMB and with the
various  Congressional  oversight  and appropriations  committees and
subcommittees.   The issue of how  the  communication of  USEPA's
research and  science accomplishments affect  the  budget  process is
critical to the financial  health  and well-being of  the Agency's
science program and  its scientists.

Specifically, USEPA  need to take the following steps:

   when  presenting  its  programmatic and research  projects  to
   OMB,  reflect  the cross-media  perspective  rather  than  the
   perspective  that fragments  environmental  protection  into
   different media
                                           t
   adopt a communication strategy that aggressively supports the
   Agency  budget — i.e.,  obtain and disseminate to the  public
   and  affected industrial  and  agricultural  sectors the  best
   possible  scientific  information  and ensure  the  nation's
   scientific expertise is enlisted  in  the  service of  USEPA's
   mission

   increase the number and quality of Congressional briefings —
   offering  Congress  an   open  invitation  to  utilize  USEPA
   resources, as needed, to gain further information  on topics
   of interest

   increase the number of  its personnel/scientist* on  loan to
   Congress/White House

   interact more positively  with the news  media  on  releasing
   important environmental issues/decisions  associated with the
   Agency, emphasizing press  releases that reflect high-quality
   of  science, convey  it   in  a  clear manner,  .and 'contain
    information that precludes conjecture and avoids negative
   publicity

   hold public forums on science for selected target audiences
    in order to adequately present the information and address the
                                 31

-------
                   TEi_:
   audience's concerns as veil as enable the audience to provide
   necessary feedback to USEPA scientists

   serve as a leader in fostering environmental education for the
   general   public,  taking   advantage   of   community-based
   organizations and placing special emphasis on educating and
   informing the public on issues of exposure and risk analysis,
   risk assessment and management,  and risk communication

   encourage  public outreach  by  its  scientists,   emphasizing
   environmental  education,  in grades  X-12,   as  well  as  in
   colleges and universities (especially  schools  with  scarce
   resources) — this outreach effort would  also include a  strong
   focus  on   stimulating  students  to  follow  science  and
   engineering careers

The Agency  should reexamine its  present  approach  of  using  non-
technical staff versed in communications and trying to bring them up
to speed on the technical issues.   A more effective  approach  would
be to  employ individuals with strong  technical backgrounds and train
them to be skilled in the  effective use of communication and public
relations tools.   There already are'many  scientists at  USEPA who
possess excellent communications skills.  Their abilities should be
tapped  to explain scientific principles as a  complement to the
communications staff.

WHO:  This should be a high-priority team effort on the part of the
Program Offices, ORD, the  Office of Congressional and  Legislative
Affairs (OCLA), and the  Office of Communications and Public Affairs
(OCPA).   The  communication materials  should be prepared in  terms
non-USEPA scientists,  Congress,  and the public can understand.

TIME LINE:  A Workgroup should be convened to develop a workplan and
schedule  to  meet  the  near-term  and  long-term  objectives  of
communicating with Congress, OMB, the  news media, and  the general
public.   The workplan should be submitted  to the  Administrator by
February  1, 1994.   The lead Offices  would be  OCLA  and  OCPA with
scientific staff detailed  from ORD,  OPPTS and other Program Offices.

BAWUERS:   Historical  roles of professional communicators versus
professional  scientists,  engineers,  and  technologists  will  make
implementation  of come  objectives  problematic.    Top management
support will be required if true change is  to be accomplished.
                                             «
COSTS:   Significant intramural and  extramural  resources  will  b«
required  over the long-tern.  In the short-term, current staff may
be assigned to scope out the program and develop the workplan.

BENEFITS:   The  short-  and long-term  benefits  to  the Agency are
significant.   Improved understanding  of  the Agency's scientific
basis for decisiorunaking by Congress and OMB is desirable  in its own
right.   The improvement  of  communication  with  the  news media is
obviously in the Agency's own self  interest.   Communication  in an
                                32

-------
•ffactive and meaningful manner with the general public is an Agency
obligation.                                                  *   *

NEASUftzs OF SUCCESSi  Short-terw measures of success may be measured
in numbers  of  events, i.e., briefings,  articles,  public meetinos
where  USEPA  science  is presented  and  discussed.    Longer-term
measures will require some  normative assessment of the perception
positive and  negative, by  Congress, OMB,  the news media  and th4
public on the "quality of science and decis4onmaking"  at USEPA
                                33

-------
                   TEL:                      flug 27.93  9 = 23 No.001 P.35
                             REFERENCES
 "A  Global  Marshall  Plan:    Developing  and  Sharing Appropriate
 Technologies  "garth  In The Balance:  Ecology and  tn«\UBSS  sg^lt
 by A. Gore; Published by Plume, New York, 1993.—

 "Aining  Before We  Shoot:  The  Quiet Revolution  In  Environmental
 Policy"; Address by William K. Reilly, Administrator, U.S.EPA to The
 National Press Club,  Washington, D.C, September 26, 1990.

 "Cabinet Fever,1* Government Executive, published by National  Journal
 Incorporate,  Volume 25, Number 7, Washington, D.C., July 1993.
gpmffun.iea.tinq Risk to Senior EPA Policy Makerat A Focus Group study.
Produced by O.L. Bloom, D.M. Byrne and J.M. Andresen  for the Office
of Air Quality Planning and Standards, U.S. Environmental Protection
Agency; March 1993.

Reducing Risk; Setting Priorities And Strateoiea For  EnvirQnnmr«-«i
Protection. U.S. Environmental Protection Agency,
Science Advisory Board (A-101); SAB-EC-90-021, September 1990.

Safeguarding the Future! Credible Science.  Credible Decisions.
Report of the Expert Panel on the Role of Science at  EPA to
William  K,  Reilly,  Administrator,  U.S, Environmental  Protection
Agency; EPA/600/9-91/050, March 1992.
                                34

-------