UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

                              WASHINGTON. D.C.  20460

                                 August 1,  1S84
                                                                       QFF*ee OF
                                                                   THE AQMINISTRATQB
 Honorable William D.  jRuekelshaus
 Administrator
 U.S.  Environmental Protection Agency "
 401 M Street,  SW
 Washington, D.C.    20460

 tear  Mr, Ruckelshaus:

      The Science .Advisory Board  has  completed  its  review of  the Office of
 Research and Development's  CORD)  university-based  Research Centers Program.
 The Board's review was carried cut by its Subecmmittee on Strategic and
 long-lean Research Planning.  The Subcommittee examined a number of issues
 related to the centers program including the role  of the centers in OBD's
 research program;  the quality of wrk performed by the centers; the EPA
 budget process and  support  for centers ? ORD management of the centers arri
 the adequacy of ORD leadership;  and  identification of options for evaluating
 ard/or reviewing  centers.

      In general,  the Subcommittee concluded that most of the centers it
 reviewed can be judged successful if criteria such as research design and
 quality, and relevance to EPA's  needs are utilized. However, the Subcommittee
 identified a number of shortcomings limiting the ability of these centers
 to be  highly productive research institutions.  Chief among those factors
 were overmanagement of the centers by EPA, resources insufficient to consti-
 tute a critical roass of support, and the poor quality of EPA leadership for
 the centers program.  The Subccnwittee has made a  number of reeotroendations
 for resolving these and other problems,  a«3 we would appreciate your response
 to these ideas,   tn addition, it is our understanding that ORD staff ar«
presently developing criteria for decisions on the renewal of centers as
well as changes in the management of the program,  we would appreciate
receiving a briefing on these initiatives.

-------
     Thank you for the opportunity to present our evaluation of this program.
we believe that if it is sufficiently funded and appropriately managed it
has the potential to be a truly significant and productive component of
ORD's research program to address many of EPA's most important information
needs.  The recommendations by the Board for improving the productivity of
the centers should he regarded as suggestive rather than prescriptive.  The
SAB is interested in being informed on EPA's plans to resolve these shortcomings.

                                Sincerely,
                                Norton Nelson, Chairman
                                Executive Committee
                                John Neuhold, Chairman
                                Subcommittee on Strategic
                                   and Long Term Research Planning
CC:  Mr. Mvin L. Aim
     Dr. Bernard Goldstein
     Or, Herbert Wiser
     Dr. Terrv P. Yosie

-------
                REPORT OF THE
   SUBCOMMITTEE ON S7R4TIGIC AND LONG-TERM
              RESEARCH PLANNING
    REVIEW OF THE RESEARCH CENTERS PROGRAM
  OF THE OFFICE OF RESEARCH AND DEVELOPMENT
United States Environmental Protection Agency
            Science Advisory Board
            Washington, D.C. 20460
                  JULY 1984

-------

-------
                                    NOTICE
     This report has been written as part of the activities of the Environ-
mental Protection Agency's Congressionally established Science Advisory
Board, a public group providing advice on scientific issues*  The Board is
structured to provide a balanced, independent, expert assessment of scientific
issues it reviews, and hence, the contents of this report do not necessarily
represent the views and policies of the Environmental Protection Agency nor
of other agencies in the Executive Branch of the Federal government.

-------
                                TABLE OF CONTENTS


                                                                              PAGE


  I,    EXECUTIVE SUMMARY .....................  	    1

 II.    INTRODUCTION  ....... 	  ...    3

       A,   History of the Centers Program  .................    4
       B.   Subcommittee Review Procedures  ,.»..,,,..,....,.    5
       C.   Outline of this Report	«	...»    6
111.    STRENGTHS AND WEAKNESSES OF THE CENTERS PROGRAM
       A,   Criteria for Establishing Centers and Definition
             of Centers' Mission „.»...,,..,,,.....*,,.    6
       B.   Comparison of Centers Program Accomplishments with
             Original Goals and Criteria .,,,,...,,,,,,,,..    7
       C.   Research Quality Review Criteria  ................    9

             1. Design and Focus of the Research Programs at Four Centers.  .    9
             2. Research Quality and Control . 	 ......  10
             3. Quality Assurance  ..............  	  .  11
             4. EPA-Center Communications  .................  12
             5. Intra^ and Inter—University Linkages ............  13
             6. Non-Center Considerations  .,,.,..,,,, 	  14

       D.   EPA Management of the Centers Program ..............  15

             1.  EPA Guidance for the Centers	15
             2.  Reporting Requirements and Programmatic and Scientific
                   Reviews of the Centers	16
             3.  The EPA Budget Process for Centers	16

 IV.    FINDINGS AND RECOMMENDATIONS  	  18

       A.   Issues from the Charge to the Subcommittee	  .  18

             1. New Center Themes  ...,.,..,»,.,..,,....  18
             2. Scientific Review of the Centers	  20
             3. Centers Program and ORD's Mission  ..... 	  20
             4. Quality of Work	  21
             5. Renewal of Centers ...... 	 ,......,,  21
             6. Communicating Research Results ...............  22

       B.   EPA Management of Centers ....................  23
       C.   The Budget Process for the Centers Program	24
       D.   EPA Leadership for the Centers  ......... ,,..,,,,  25
       E.   Role of the Centers ....,,,.,,..........<..  26
                                        11

-------
V.   APPENDICES

     A.  Charge to the Subcommittee for Strategic and Long-Term
           Research Planning	,	   A-l
     B.  Roster of the Subcommittee  *.....,,.......,...   B-l
     C.  QRD Research Centers	C-l
     D.  Site-Visit Reports  ..... 	 .....  	   D-l

-------
                                Key to Acronyms
AA      -     Assistant Administrator

AECTRC  -     Advanced Environmental Control Technology Research
                Center

CASAC   -     Clean Air Scientific Advisory Committee

EPA     -     Environmental Protection Agency

HERL    -     Health Effects Research Laboratory

IERL    -     Industrial Environmental Research Laboratory

MERL    -     Municipal Environmental Research Laboratory

NCGWR   -     National Center for Ground Water Research

NIEHS   -     National Institute of Environmental Health Sciences

NIH     -     National Institutes of Health

OER     -     Office of Exploratory Research

ORD     -     Office of Research and Development

ESKERL  -     Robert S, Kerr Environmental Research Laboratory

SAB     -     Science Advisory Board

SAC     -     Scientific Advisory Committee

SRC     "•     Scientific Review

-------
 I.   EXECUTIVE  SUMMARY

     This  ts the  final  report  of  the EPA Science  Advisory Board's (SAB)
 review of  the  Office of lesearch  and Development's  (OiU3)  university-based
 research centers  program.   The Board's  review was carried out  "by the
 Subcommittee on Strategic  and  Long-Term Research  Planning which was foriaad
 to provide advice on A  series  of  long-term  research and development issues
 confronting EPA.

     The Subcommittee has  focused on, six major issues  in  its review of
 the  research centers program,   these include:   1) the  role of  the centers
 in OiD's research program;  2)  the quality of  work performed by the existing
 centersj 3) the EPA budget  process and  financial  support  for centers;
 4) EPA management of the centers  program; 5)  the  adequacy of EPA leadership
 for  the centers program; and 6) options for evaluating and/or  renewing
 centers*

     The Subcommittee finds that  neither the  criteria  for evaluating the
 centers performance nor the mission  of  the  centers  program has been clearlv
 established by OSD,  This has  led to a  great  deal of confusion within  OED
 and  between OSJ3 and individual centers  regarding  the appropriate role  of
 centers in OKD's  research  program*   Although  most centers visited by the
 Subcommittee can  be judged successful if one  uses criteria of  research
 design, quality and relevance  to  EPA's  needs,  it  is equally clear that the
 centers program can not he termed successful.   The  centers program exists
 in a vacuum insofar as  BPA's research planning process is concerned and
 thus, it has had  no discernable impact  upon how SPA identifies its  research
 needs*

     There are a  number of ways of resolving  these  problems and achieving
 a clarity  of purpose for the centers  program.   These include;   1) 01D  should
 prepare guidance which  clearly_ establishes  the mission of the  program  and
 defines criteria  by which the  goals  and performance of the centers  can be
 measured;  and  2)  Olfl should define EPA's  high  priority health  and environ-
 mental research needs for the  next five to  ten years and  Identify which of
 these needs can be addressed aost  effectively  by  centers*

     The quality  of work performed by most  of  the four centers visited "hj
 the  Subcommittee  is generally  high.  Factors  such as the  design and focus
 of the research program} research quality and  quality  assurance are given
high priority  by  the center directors, and their staffs who have also evolved
constructive relationships with their respective  Scientific  Advisory
 Committees.  In general, the number  of  linkages between the  centers and
 their affiliated  universities  continues  to  grow,  thus  drawing  a wider
 spectrum of disciplines and talents  under the  centers'  umbrella.

     A major limiting factor to the achievement of  high levels  of research
productivity by the centers is  the lack of  adequate budgetary  support.  The
dimensions of this problem are  two-fold.  First,  the process by which
centers  receive their annual budget  is  counter  productive  to center research

-------
performanee*  After extensive preparation and review of their research
plans by several layers of EPA management, Che centers' budgetary alloca-
tions are made without a clearly defined rationale and at a level of
funding considerably below the target allocations given to ths centers to
guide their research planning for the following fiscal year.  A second set
of problems stem from insufficient levels of support.  The current budget
of §420,QQQ per center per year is simply not adequate to constitute a
critical mass of resources to achieve high levels of research productivity.

     The Subcommittee recotnmends that a number of steps be taken to resolve
problems related to budgetary process and support.  These include:  1) alloca-
tions of resources should be received by the centers at the start of the
annual project period; 2) ORD should identify ways to streamline the multiple
layers of review in the centers' budget cycle, including further delegations
of authority to the level of ORD where the responsibility lies for managing
the centers program; 3) ORD should distinguish between those portions of a
center's budget that constitute core support and those that comprise funds
for research; 4) centers should be free to supplement core support resources
by any means that does not constitute a conflict of interest or run contrary
to the mission of the centers program; and 5) the support needed by most
centers to maintain a high level of research productivity ranges from a
minimum of $800,000—$1,000,000 per year to a maximum of $2,000,000 per
year (including core support and research funds).

     The centers program is both overreviewed and overmanaged by EPA.  The
combination of policy board reviews, Scientific Advisory Committee reviews
and periodic administrative reviews by OED headquarters — In addition to
budgetary reviews --' are excessive given the amount of resources allocated
to the program.  Administrative costs consume a minimum of 20-25% of the
centers' budget which is an excessively large fraction of resources at the
current level of funding.

     The Subcommittee recommends that the following measures be impleiaentecl
to resolve these problems of overmanagement: 1) both the policy board and
Scientific Advisory Committee should meet only once per year, ideally at
the same time; and 2) QRD should clarify a number of institutional relation-
ships associated with the centers program, including:  a) the relationship
between the project officer and the policy board chair—^where possible, the
two positions should be held by the same individual, preferably a senior
laboratory official; b) the stability of the policy board and the knowledge
of Its members.  Board members should be appointed for fixed terms, staggered
so as to facilitate the continuity of knowledgeable members while gradaully
introducing new participants; and c) the role of ORD headquarters and the
Assistant Administrator In annual sign-off support for the centers.  GRD
headquarters review need only occur when a center is up for a renewal
decision*  There is no need for the Assistant Administrator to annually
approve a center's funding.

     One of the most troubling of all the issues addressed by the Subcommittee
in Its review of the centers program was the quality of leadership in ORD's
Office of Exploratory Research (OER).  Throughout the program's existence

-------
 there has been no effective senior spokesperson or advocate for the centers
 within EPA,  particularly at headquarters*   As a result, the centers ate
 treated as orphans whose existence is tolerated, but they are not adequately
 supported or effectively utilised.  The Subcommittee believes that the
 Assistant Administrator must__become the advocate for the program, for this
 is the ^individual who has the scope of authority and span of control to
 successfully utilize the_ scientific,, talents in residence at the centers.

      It Is not clear how knowledgeable or  concerned the current OER leadership
 Is about the many problems that beset the  centers program.  The Subcommittee
 recommends that_future directors.of PER possess scientific competence and
 professional experience in managing a long-term research prograa, have the
 support and  respect of the scientific community,, and senior policy_o£ficials
 at EF^.» ^e atl3-e -to,,,.P.r.gyi-d
-------
Subcommittee on Strategic and Long-Tera Research Planning.  The Subcommittee
was formed by the SAB's Executive Committee on December 9, 1983 co identify
specific areas in which the SAB could advise ORD and the Agency on a number
of long-term research and development issues related to the Agency's mission
to reduce human health and environmental risk from anthropogenic activities*
The review of the centers program is the second of what is expected to be
a series of Subcommittee reports.

      The SAB review of the centers program was requested in December 1983 by
Dr. Bernard D. Goldstein, Assistant Administrator for lesearch and Development.
Dr. Goldstein specifically solicited the SAB's input to assist GW> in its own
evaluation of centers prior to determining whether and under what conditions to
renew existing center agreements.  The specific charge to the Subcommittee was
"1> examining what environmental areas and theiaes will be of highest priority
for the centers program; 2) assisting ORD in conducting a scientific review of
the centers, associated with ORD's scientific and management review of the
program; 3) reviewing the effectiveness of the centers program as a means of
carrying out ORD's mission; 4} advising on the quality of the work performed by
the centers; 5) adi/ising ORD on the issue of options for renewal of centers;
and 6) examining how research results generated by both the Research Centers
Program and the Peer Review/Investigator Initiated Grants Program can be more
effectively communicated to ORD's laboratories and to EPA's program offices*"
The complete Subcommittee charge is included as Appendix A*

      To carry out the review of the centers program, the Subcommittee
recruited a number of scientists and engineers representing a diverse set
of scientific disciplines and institutional affiliations*  The review panel
embodied expertise and experience on both bench research and research
management levels.  The roster of the Subcommittee is presented as Appendix 3.

     A.   History of the_ Centers__Program

     The concept of a research centers program within EPA originated in
the 1970's as the outgrowth of a concern over the direction of the Agency's
research program.  Reports prepared by the Office of Technology Assessment
in 1976, the National Research Council in 1977, and The President's Office
of Science and Technology Policy in 1979 called to the attention of
Congress and EPA policymakers the need to "balance the Agency's shorter-term
needs for technical assistance for regulation development with a commitment
to support research directed to longer-term research projects and programs
that would improve the scientific basis of regulatory decision making*  In
particular, these groups believed that EPA had the responsibility to identify
research gaps and foster advances in the state of scientific knowledge to
ultimately serve EPA's regulatory information needs.  In Fiscal Year (FT)  '78
the Congress required EPA to assess laboratories needed to support long
term research.  On April 5, 1978 the Agency submitted to the Congress a
report that examined a number of alternative approaches for conducting
Ions-term environmental research.  A key recommendation of the report was
    ^See also  the Preliminary Report of the S_ei_enee^Advisory Board  Stud)
Group on Strategic and_Long-Term Research Planning, December 7,  1983.

-------
that "EPA should draw upon and utilize existing institutional resources as
one method of filling research gaps.  Approximately five to ten centers of
expertise should be supported primarily at existing institutions with
specialized expertise.»»» Approximately, $500,000 to $1 million will be
required annually to support each ceater»"

     The Agency followed up this report "by appointing a task force to
identify themes of interest.  A newly formed Office of Exploratory Research
(OEI) prepared concept papers for each thetae.  These became the basis for a
wide solicitation of letters of Intent from interested universities.  After
reviewing the letters of intent, EPA Invited selected universities to submit
formal proposals for the competitive awarding of a cooperative agreement for
each center theme.

     Beginning in FY*79 EPA authorized cooperative agreements with eight
university "based centers*  The centers and their respective starting dates
are listed in Appendix C.  A funding history of the centers program is
presented in Table I.

         Table I;  Funding History of ORD Research Centers Program
     Fiscal,Year                               A^lQt-A^PJ^jfJ-.-t0. Program
                                                    (millions of S)

       1980                                        ^    2.8

       1981                                            4.9

      •1982                                            6,3

       1983                                            3.4

       1984                                            3.4*

       1985                                            3,4**
 * Current estimate of actual dollars to be expended
** Administration's request to the Congress for FY'85
     B.   Subcommittee Review Procedures

     The Subcommittee held three public meetings on February 3» March 14-15,
and May 17-18, 1984.  In addition, the Subcommittee subdivided into four
groups to conduct site visits during the month of April.  The sites visited
included the Epidemiology Research Center, the Advanced Environmental
Control Technology Research Center, the National Center for Ground Water
Research, and the Ecosystems Research Center,

-------
     The Subcommittee analyzed previous reports on the program by SPA
and the centers.  Extensive briefings were provided to the Subcommittee by
EPA staff (including representatives of QRD headquarters and laboratories),
representatives of the policy boards, and center directors and their staffs.
From these briefings the Subcommittee learned about EPA's management of
the program, ongoing and anticipated research activities carried out by
the centers, and specific problems and opportunities associated with the
program as a whole, as well as those related to specific centers.  This
information served as the basis of the Subcommittee's findings and
recommendations.

     Both Agency and center staff were extremely helpful In providing the
information needed to respond to the Issues listed in the charge.  The
Subcommittee appreciates this cooperation and wishes to acknowledge the
contribution of all of the individuals associated with the centers program.

     C.   OutIine_of_^thi_s__Report

     The body of this report consists of two major sections.  Section III
presents an analysis Of the major strengths and weaknesses of the centers
program, encompassing such factors as the scientific quality and relevance
to EPA of research carried out by the centers and EPA's management of the
program.  The Subcommittee's findings and recommendations are presented in
Section IV,

III.   STRENGTHS .AND WEAKNESSES ..OF_THE CENTERS PROGRAM

     A.   Crl_te_r_ia for Establishing Centers and Definition of Centers'  Mission

     The Subcommittee's review of Agency documents and interviews with
Agency and centers* personnel reveal a wide disparity of opinion about both
the criteria for the establishnent of centers and the mission that the
centers were to perform.  For example, ORD staff informed the Subcommittee
of five criteria that were originally used! to review applications and select
centers.  These included:  1) the scientific quality and creativity embodied
within a university's proposal to receive a center; 2) the university's
ability to develop innovative solutions; 3) qualifications of the researchers;
4} university facilities; and 5) experience of the university personnel.

     These criteria contrast with those provided In the OER concept paper
for developing center themes entitled "General Guidance for Centers"*
Examples of criteria Identified in this document include;  1) "centers and
their programs shall have a multimedia and multldlsciplinary orientation...";
and 2) "center programs oust be responsive to the long-cerm needs as perceived
by all EPA laboratories whose activities are related to the center objectives.'1

     A similar diversity of viewpoints characterized the definition of the
centers' mission.  The solicitation announcement for the original centers
emphasized chat they would augment EPA's ongoing long-term research program.

-------
However, discussion  at an  early  policy board  meeting  of  one  newly formed
center indicated  that approximately  10-201  of the center's funds  were  to
be earmarked for  studying  problees of immediate  interest to  the Agency.

     In short,_ both  the  criteria for_e8tablis_hing_the centers  and the
mission of the centers program have  not  been  clearly  articulated  by EM.

     B.   Comparison of  Centers ...Program  Accomplishments  With Original.Goals
          and Criteria

     A, major rationale for the establishment  of  the university-based
centers program was  to carry out  research designed to meet EPA's
longer-term information  needs as identified by the Congress, the  National
Research Council, the Office of  Technology Assessment, and EPA internal
task forces and advisory committees. • When eight centers were  established
between 1979 and  1981, both the  selected universities and many senior  ORD
officials had high expectations  regarding the program's  potential.  EPA
officials, in particular,  saw the program as  a means  to  establish more
formal linkages to the scientific community,  with the hope that such ties
would lead to advances both in scientific knowledge and  in the credibility
of the Agency's research program.

     A major difficulty  in assessing the accomplishnents of  the centers
program lies, as noted in  the previous section, in the absence of clearly
defined criteria against which to judge  performance.   In theory,  the QER
solicitation statement for each  theme describes the original goals  of each
center.  However, as centers were established, it became clear that there
was no unified policy to evaluate the centers1 operations.  Even  solicitation
statements were not  consistently used as policy statements*  Without adequate
st*d consistent guidance  from QRD headquarters, policies  were developed on
an ad hoc basis'for  individual centers,  in response to specific questions
raised by a particular center.   Individual cooperative agreements,  therefore,
may contain policy specifications peculiar to  one university.

     An additional difficulty in weighing the  centers' performance  is the
erratic budgetary history  of the program.  As  seen in Table  I, the  budget
increased steadily from  FY'SO through FY'82,  but beginning In  FY*83 a sharp
decline in resources ensued.  This up and down trend  of  resource  availability
Is disruptive to any research program, but it  is particularly  disruptive to
the planning cycle of research projects  of several years' duration*

     Given this ambiguity  in both OED policy  and funding, what have the
various centers accomplished in  their three to five years of existence,
compared to what was expected by OER or  the leadership of 0S.D?

     It is apparent  that some centers have accomplished  essentially what
EPA had_in mind,  and in  some cases even  more  than EPA should have expected_
or deserved given the_v_icissitudes of the EPA  budget, changing leadership
in OER an_d_CQnEradict_o_ry_ progratrnnatic guidance.  The  Epidemiology Research

-------
Center, however, had no clearly defined mission for the  first  three years
of Its existence and Its performance during this period  reflects  this  lack
of definition.

     The extent to which the centers program is judged "successful" depends
in large part upon, the criteria used in measuring Its impact on research-
related functions of EPA.  The original concept of EPA-funded  centers, as
opposed to university-based centers supported by the National  Institutes of
Health (HIM) and other Federal agencies, was that EPA and the  university
scientists would jointly plan and manage research that could not  easily be
conducted in EPA's own laboratories.  EPA's research would be  enriched,
extendedj and strengthened by long-term and exploratory  research  in the
university—the kind of research that Is difficult to nurture  in  an EPA
laboratory because of regulatory pressures to acquire short-terra  data  in
support of specific programmatic activities.  These principles are embodied
in the concept of cooperative agreement funding and in the role of the
policy boards*

     If one examines the centers program from the viewpoint of:   1) criteria
for joint research planning; 2) feedback of research findings  into the
regulatory process; 3) general enrichment of EPA's research efforts; and
4) exchange of scientists between centers and EPA laboratories, there  is
little evidence of success.  The centers _hay_e__ almost no  disce_rnible impact
on EPA's research jjlanning process .,_and_the_ planning process^ (in  contrast
to the budgeju p_roeess) has very lit^tle__liBpact_ oti the centers*  The centers
ey;ist_ja$_i/ J.B._a__va.cuua and have had no discernible _impact upon how EPA
identifies its research needs*  Thus, & major purpose jof the centers
program has not materialized to a significant extent.

     The value of the centers program and its impact on  EPA should also be
judged by other criteria*  Ideally these criteria would  include the quality
and quantity of the long-term or exploratory research, the relevance of this
research to EPA's long-range needs, and the timely availability of the research
information to "EPA's laboratories and program offices.  Related criteria would
be whether the centers program has attracted nature scientists in. various
departments of a university to engage in EPA mission-'related research  in which
they would not otherwise be engaged, and whether the centers are  attracting
younger faculty members, graduate studentss and post doctoral  fellows  into the
environmental sciences.

     The Subcommittee did not attempt to develop a quantitative assessment
of the centers in terms of these criteria*   Nevertheless, the  site visits
and the meetings with center directors and policy board  chairmen  yielded
abundant examples of fundamental research closely relevant to  EPA's problems,
fruitful interactions between centers and EPA laboratories, recruitment of
both younger and mature scientists, development of new academic courses,
and other activities which demonstrate that an EPA-university  connection
has mutual benefits*  These examples will be cited throughout  the text of
this report.

-------
      In the Subcommittee's view, the, principal factors leading to success
 for this program.are htgh quality o_f_center leadership, comparable quality
 and consistency of policy board and ORD leadership, degree of university
 suppert_and quality of university personnel, and adequate EPA resources.
 Where one or more of these are lacking the center has been less successful
 and even disappointing.  What Is critically needed at the present tine to
 promote the success of _ the eenters_ program as & whole is a continuity and a
 Duality of leadership in QRD headquarters that: understands what a centers
 .program should proyide_tQ_ EPA,, what factors lead to success or failure, and
 what management and budgetary resources are necessary for a successful
 program.  In summary, the centers program cannot be proclaimed a success
 story.   However, the reasons for this,result rests primarily_with the
 inadequacy of ORD leadership and support father _than with the universities.

      C.   Research__Quality Review Criteria

      The research quality review criteria identified in the Subcommittee
 charge and utilized to evaluate the  quality of work performed by individual
 centers  were jointly developed by ORD and SAB staff and were accepted by
 the  Subcommittee in its charge (See  Appendix A).   These criteria formed
 the  basis  for soliciting  information and formulating opinions on the  work
 performed  at the Epidemiology Research Center, the National Center  for
 Ground Water Research (NCGWR),  the Ecosystems  Research  Center, and  the
 Advanced Environmental  Control  Technology Research Center (AECTRC),

           1-   Pesign_and  Focus  of the Research Programs at  Four  Centers

     The program at  the ICGWR focuses On the  fate  of organic  chemicals in
 the  subsurface  environment Including  both saturated  and  unsaturated Ground
 Water conditions.   It  Is  a multi-university, multi-department, multi-
 discipline  program.   The  researchers  have  training and  experience in  civil
 and  chenlcal engineering, chemistry,  physics,  mathematics and biology.   In
 developing  its  program, the  Center has aimed at  the  long  term.   Some  of
 its  best work is  fundamental  and  clearly  identifies new and important
 phenomena about  partitioning  and  transport  of  chemical  species in ground
 water.  This program has  shown the ability  to  cross departmental and
 institutional barriers  to bring a multidiseipllnary group together.   Its
 talents are broader and stronger  for  ground water research  than the current
 EPA  funds can support.

     At  the Ecosystems Research Center the  program is multifaeeted,
 involving both review and synthesis and modelling efforts.  An example of
a review and synthesis effort is  the  development of a multi-authored
 treatise on ecotoxicology, an. emerging field of toxicology-  This is of
particular value In the area of Setting water quality criteria under the
Clean Water Act.  Modelling efforts have long been used to simulate complex
physical  and biological systems.  The center has employed this tool to
pinpoint  subsystems in the aquatic ecosystems surrounding drilling platforms
for early warning monitoring.

-------
     The research program at the AECTIC focuses on air and water pollution
control.  Each of the research projects is aimed at gaining a fundamental
understanding of mechanisms to improve control processes*  The researchers
are primarily faculty members from the college of engineering who hold
appointments in the civil or chemical engineering departments*  Several of
the investigators have advanced degrees In physics, chemistry, or biology
as well as engineering.  Near-terra research addressses topics In biological
degradation of pollutants on activated carbon, regeneration of spent acti-
vated carbon, ozone and hydrogen peroxide oxidation of dissolved hydrocar-
bons, and surface properties and charge on aerosol collection efficiencies.
Longer-term research is examining fundamentals of super critial fluid
extraction.  In addition to EPA support, many of the researchers have other
government agency and/or industry research support, and thus the EPA, gains
by having a highly trained staff, not fully supported by Che Agency, working
on projects that are of interest to EPA.

     There has been extensive occupational epidemiologic research and some
environmental epidemiologie research by faculty members associated with the
Epidemiology Research Center, but most of the research has not been
identified with or funded by the center.  Only in the past nine months has
the center identified environmental research areas in which it plans to
develop research programs.  These Include development of methods for
epidemiologic study of populations Impacted by hazardous waste dump sites;
improvement of methods for surveillance of reproductive effects; volatili-
zation  of chemicals from potable water as a source of indoor pollution; and
improvement of risk assessment methodology based on epidemiologic studies.
These are appropriate topics for long-term study, given the competencies
and interests of core and resource faculty in this center.  The new director
and associated faculty members in epidemiology and biostatistics have
well-earned reputations for high quality research in occupational epidetaio-
logy,

          2.   Research Quality and_CQntrol

     A  simple means to judge research quality and control is to measure
peer acceptance of research output.  Each of  the reviewed centers uses its
Scientific Advisory Committee  (SAC) a little  differently to ensure  the quality
of its  research projects and publications.  For example, at the NCGWR there
is competition for research funds through a process of submitting internal
proposals  from each of the three universities that are part of the  center,
Quality control is maintained by both a pre-proposal and post-project
review.  Preproposals are solicited and reviewed by the  three center co^
directors as well as by scientists and engineers from the Robert S. Ksrr
Environmental Research Laboratory (RSKERL), Ada, Oklahoma,  for relevance
and to  avoid duplication of efforts.  Full proposals are prepared,  reviewed,
and ranked by the SAC,  The co-directors use  this  ranking to determine
which projects are to be supported.  The  policy board reviews these projects
and the yearly work plans.
                                        10

-------
     ACET5.C uses Its Scientific Advisory Committee to review and approve
Individual research proposals and to conduct periodic reviews of completed
research projects.  The SAC, In conjunction with the policy "board, addresses
the relevance of the research program Co EPA with the purpose of initiating
new research themes as well as deciding if certain project areas should be
phased out.  The research staff of this center "has excellent credentials,
and their reputations lend to .the scientific credibility of Che center's
activities.  For example, one assistant professor was recently recognized
with a Presidential Toung Investigator Award, and the center director is a
member of the National Academy of Engineering*

     The Ecosystems Research Center uses a Scientific Review Committee (SRC)
to evaluate the work of both the staff and the Scientific Advisory Committee
(SAC)*  This independent panel of ecologists is nominated by the center
director and approved by the policy board chairman.  The SAC provides
guidance as programs are initiated, whereas the SIC provides aid-course and
end product evaluation.

     In contrast, the SAC of Che Environmental Epidemiology Center has been
underutilized until this year*  However, the SAC now has a regular meeting
schedule*  Its principal function is to review research proposals and parti™
clpate in quality evaluation.

          3.   Quality^Assurance

     When the centers program was first Initiated, the competitive process
between universities was used by EPA to identify high quality institutions
and staff committed to its announced goals for the centers.

     The principal products of the centers are written reports, or papers
that are prepared for publication in peer reviewed journals.  Acceptance and
publication of these papers is taken as explicit recognition that the work
is acceptable to the scientific community.

     In the particular case of the Epidemiology Research Center, conventional
laboratory practices for quality control are of limited application since
laboratory or experimental science plays a small role in this center's
activities.  The data of epideaiologie studies, such as death certificates,
have well known problems of accuracy, completeness, bias, and other sources
of variability.  However, as statisticians and epidemiologists, the key
faculty are familiar with these problems and the measures to reduce their
Impact.

     At the three centers where laboratory or experimental science play a
more significant role (NCGWR, AECTR.C, and the Ecosystems Research Center),
quality assurance is adequately addressed throughout the data collection
and interpretation phases*
                                       11

-------
          4,   EPA-jCenter Communicat ions

     Interactions between EPA and the centers have taken several forms*
The Subcommittee has not attempted to characterise or offer opinions on all
means of communication; father, it has identified several areas that it
believes are representative of EPA staff's attitudes and management approaches
toward the centers.

     EPA staff from ORD headquarters and laboratories and the program offices
serve on the policy boards that provide oversight and articulate policy and
programmatic guidance and goals for each center*  In general, the Subcommittee
found that the policy boards it examined have, over time, appropriately
evolved into oversight units that have attempted to facilitate center
operations.  For example, the director of the Robert S. Kerr Environmental
Research Laboratory in Ada, Oklahama, who chairs the policy board for the
National Center for Ground Water Research, has made diligent attempts to
obtain funds for this center.

     Both Office of Exploratory Research and policy board guidance should be
consistent.  Even generally supportive policy boards, however, sometimes raise
obstacles when their relationship with OER or the Assistant Administrator's
office is unclear.   The policy board for the Advanced Environmental Control
Technology Research Center (AECTRC), for instance, at one point stated its
desire to approve the terms by which an industrial group would fund a center
project even though no conflict of interest was apparent.  This was unaccept-
able co the industrial representatives who subsequently declined to co^sponsor
the project.

     Perhaps the worst example of EPA^center comiaunlcation breakdown existed
at the Epidemiology Research Center which has not sought non-KPA external
funding in part because of difficulties in gaining policy board approval*
The policy board chair for this center has experienced a deplorably high
turnover rate; in the five years of its operation, there have been five
different chairs.   This lack of stability has contributed to the difficulties
experienced by the center in obtaining consistent EPA guidance.  In addition,
a number of the center's activities involved EPA's requests for reviews of
criteria documents, grant applications and research protocols.  The Subj
committee concludes that these technical assistance activities have been
excessive, and many are inconsistent with the stated purpose of a center*

     The most fruitful center~policy board communication patterns observed
by the Subcommittee occurred when policy board representatives were EPA
laboratory employees.   This is true for at least two reasons.   First,
ORD laboratory personnel are more likely to possess the technical expertise
necessary to understand and review center activities.   Second, laboratory
representatives,  in contrast to ORD headquarters or program office staff,
can more readily regard their center counterparts as scientific colleague.?.
Such colleague-to-colleague interactions are, in selective instances.
                                       12

-------
 beginning to bear fruit.  For example, cooperative research ventures have
 evolved between the Ecosystems Research Center and both the Gulf Breeze and
 Corvallis laboratories.   Similar endeavors are transpiring between AECTRC
 and the Municipal Environmental Research Laboratory (MERL) In Cincinnati,
 The Deputy Director of the Industrial Environmental Research Laboratory in
 Cincinnati is also the project officer and policy board chairman for the
 AECTSC and the other two technology centers.   This arrangement has promoted
 direct exchanges of information between the laboratory and the centers and
 within the centers.

      Such examples represent an Improvement In laboratory-center relations
 since the time when the  centers were first established.   At that time,
 research funds were withdrawn from the laboratories to support the centers'
 operations.   This "tapping"  of the laboratories by the Assistant Administra-
 tor produced an understandably negative reaction by laboratory directors
 and staff*   In subsequent  years the laboratories have, until the very modest
 budgetary upswing in the current fiscal year,  witnessed a considerable
 erosion.in both people and dollars for intramural and  extramural projects.
 Given this context, it Is  remarkable that laboratories and centers are
 Increasing their scientific  exchanges  on mutually beneficial projects.

      One should not become too sanguine about  these observations,  however,
 because as  already noted,  they do  not  apply to  all EPA laboratories  and
 centers.   In addition, they  may not last in cases where they currently
 apply*   The  development  of_ a useful dialogue...and joint project planning
 between^laboratories  and centers are primarily  the result of the willingness
 of  IgtQgledgeable people  in both institutions who recognize the advantages of
 cooperation,  rather than the existence of any  fomal mechanism_ that  promotes
 their interaction.  Given  the fact  that the circumstances_eneouraglng such
 exchanges  can change,  through,  forjexample, the  departure of key^personnel.
 thlg  is  an extremely  f ragile_and ad hoc .format  for plan.nlng_research and Ig
 no  substitute  for a more strtictured_ approach,

           5.    Intra  and Inter-University Linkages

      Examples  of  Intra and Inter-university contacts are  presented below for
 each  center.   During  the early  existence  of the  Epidemiology Research Center,
 efforts were made  to  establish  the  center  as a university-vide  activity
 within the University of Pittsburgh and  to build  bridges  to  neighboring
 Carnegie-Mellon University.   One mechanism for this effort was  an executive
 committee which included representatives  from a.  lumber of  faculty departments
 at both universities  to  solicit grant  proposals.  However, none  of these
 proposals was approved by the policy board, and  the collaborative effort
was eventually abandoned.  At the present  time the center  Is  located
within the School of Public Health, providing a  focus  for many  activities
 of the Departments of Biostatistics and Epidemiology, with some  Involvement
 of other departments.  The center Is collaborating with Brookhaven National
                                       13

-------
Laboratory in developing risk assessment methods for the synfuels industry*
Current attitudes at the center would promote broader inter~university and
inter-laboratory collaborations, if funding were increased and if EPA had
an epidemiologic studies program intramurally.

     The NCGWE, a consortium of three universities located in two states^
has addressed the probleo of linkages from the very beginning of its
existence*  In addition to the consortium, the center has supported research
projects with co-principal investigators at different institutions.  For
example, there is a laboratory and field project conducted at a superfund
site in Texas that is directed by one investigator from Rice University
and one from the University of Texas at Austin.

     The Ecosystems Research Center has been very successful in marshalling
a variety of talent to the center.  Various units on campus, including the
Department of Ecology and Systematics, the Department of Natural Resources,
and the Boyce-Thompson Research Institute are active participants in the
center's projects,  the center's interaction with the Boyce-Thompson Institute
has resulted in significant findings concerning acid deposition effects
on the secondary infection of plants.  Extra-university linkages facilitated
by the center include working relationships with Woods Hole Oceanographic
Institute and Oak Ridge National Laboratory as well as with EPA laboratories
at Narragansett, Gulf Breeze, and Corvallis.

     The center director of the AECTRC serves on the SAC of the Industrial
Waste Elimination Research Center, thus providing center-to-center interaction*
Researchers supported by the AECTRC are affiliated with the colleges of science
and engineering at the university; center researchers also have additional
research support from other government agencies or from industry.

          6.   Non-Center Cong_i_d_e_r_ati_on3_

     There are a number of factors which influence the conduct of center
business but which are not under the control of the center.  A good example
is the Epidemiology Research Center, which lias had significant problems in
its relationship with EPA.  One set o£ problems related to the composition
of the policy board, in particular the frequently changing chairmanship
prior to the appointment, one year ago, of the director of the Health
Effects Research Laboratory (HERL) at Research Triangle Park, N.C.  The
unstable leadership of the policy board led to changes in direction
and uncertainty about the goals of the center.  The second major problem
resulted from EPA's termination of its intramural epidemiologic ptograeu
Except for individual epidemiologists at HERL, there was no epidemlologle
research program or epidemiology "presence" within EPA to which the center
could relate.  The dissolution of epidemiology research within EPA was not
accompanied by any contractual mechanism which might have given the center
responsibility for carrying out the epidemiology mission of the Health
Effects Research Laboratory, with linkages to research planning and work
plan development In the Agency,
                                        14

-------
     There have also been a number of positive results, attributable to the
centers, within the broader scientific community outside of EPA.  For
example, the ASCTtC has been involved in other educational activities such
as a trilateral agreement to train Japanese environmental engineers under
the Engineer Exchange Provisions of the U.S.-Japan Bilateral Agreement on
the Environment,  The attendance and participation at the two International
Research Symposia on Ground Water reflect the prominent role that the NCGWE
has assumed as a mechanism for transfer of research results to the multi-
disciplinary Ground Water community*

     D.  SPA Management of the Centers Program

     The centers program Is managed by the Office of Exploratory Research
(Q1R), which Is a staff office within the Office of Research and Development
at EPA headquarters and whose director reports to the Assistant Administrator
for QRD.  OIR manages other research' programs including the Peer leview-
Investigator Initiated Competitive Grants Program and the Visiting Scientist
Program, and serves as the lead ORB liaison with the National Academy of
Sciences.  Both the grants and centers programs have headquarters staff
assigned to manage day-to-day operations, although during the current
fiscal year each program lacked a permanent director for a number of months.

     The Subcommittee's review of the management of the centers program
has focused on  several issues.  These include:  1} the nature of EPA guidance
to and interactions with the centers; 2) reporting requireaents and the
number of programmatic and scientific reviews of center operations; and
3) the budget process.

          1.  EPA guidance for the centers

     Bespits the progress noted in some areas, major problems in EFA-center
interactions persist.  These, In fact, overshadow the positive .features of
the dialogue between the centers and most EPA laboratories and  policy boards.
Many of  these problems can be traced to EPA/ORD headquarters and more
specifically to the Office of Exploratory Research.  More than  one policy
board  chairman  stated  that he did not know  what current GRD/OER policy was
towards  the centers' acceptance of non-EPA  funds because prior  guidance on
this  issue had  shifted 180 degrees.  Also,  the previous Acting  Assistant
Administrator had opposed the use of extramural laboratory funds for specific
center projects, although  the Incumbent Assistant Adminstrator  has reversed
this  decision.

      QER direction  to  the policy boards  and to  the  centers Is also Inconsistent
or nonexistent.  This  is reflected  in  the obscure guidance on preparation of an
annual center  report,  the  absence  of a mechanism  to  integrate center  research
activities into ORD's  planning  process,  the lack  of  a  formally  articulated
conceptual framework on  how center  projects are  to  be  judged relevant  to
EFA's  mission  and  how  they  should  support  regulatory programs»  confusion
over  whether  the  centers  are EPA  centers  and only to be  supported  with
Agency funds or whether  they are university centers  for which EPA  provides
                                        15

-------
core support, and lack of clearly defined criteria for replacement of
center directors or the renewal of centers*  The absence of PER leadership
to develop creative proposals to try to resolve these problems is discon-
certing and represents_an abrogation of responsibility.  Discussion between
PER personnel and_the Subcommittee also^indicated that it is not clear how
knowledgeable or concerned^ the current PER leadership is about these problems
and their effect on the program-

          2.  Reporting Requirements and Programmatic and Scientific
              Reviews of the Centers

     Individual centers are subject to at least three kinds of review.  These
include:  1) programmatic review of center goals, operations and expenditures
by the policy board; 2) peer review of project proposals and on-going
research activities by a Scientific Advisory Committee; and 3) periodic
administrative review by ORD headquarters, including OER.  For each kind of
review, center staff must prepare written position papers and/or proposals,
with an accompanying rationale to present to the reviewers.  Several center
directors estimated that about 20-25% of their FY'84 budget of approximately
$420,000 was expended on such reviews and administrative costs, while one
director judged that more than 50% of his budget was absorbed by-such
activities.  These expenditures included such items as travel and salaries
and expenses.

     The frequency of policy board meetings has varied from center to center
and from year to year.  At the AECTRC, for example, the policy board averages
two to three meetings per year, while at the Epidemiology Research Center it
has convened approximately five meetings since 1979.  Both center and most
EPA personnel were in general agreement that the policy board could meet
once per year and retain an appropriate level of oversight.  The peer
review provided by the Scientific Advisory Committee was also regarded by
most observers as adequate if carried out once per year, although the
current practice is to meet somewhat more often.  In addition to these
sets of meetings are periodic administrative gatherings called by the OER
director and staff to review such issues as the budgetary outlook and
changing EPA peer review procedures.  In addition, it is sometimes necessary
for center directors or staff to make site visits to ORD laboratories.  The
total number of such meetings and reviews more readily explain why many
center directors spend approximately 50% of their EPA funded time to manage
administrative issues associated with the center.  It also leads the Sub-
committee to conclude that, for the amount of dollars allocated to the
program, the centers are over reviewed and over managed.

          3,  The EPA Budget Proces_s for Centers

     The process of preparing, defending and allocating the budget for the
centers can be summarized in one word—byzantine.  A brief description of
how this process works will help to illustrate the ORD management attitudes
and approaches toward this program.
                                       16

-------
      The  budget  cycle  for a typical^ center begins  approximately six nonths
 (during April  or May)  before the  allocation of funds  for the next fiscal
 year. At  that time  OER provides  each center with a target allocation so
 that  the  center  can  identify which research it plans  to. carry out within
 the dollar ceiling*  The center director and staff  prepare position papers
 for review by  the policy board on what research projects should "be carried
 over  or phased out during the next fiscal year as well as identifying new
 research  initiatives that can be  funded under the target allocation.   If
 new projects are approved by the  policy board, center staff proceed to
 prepare formal research proposals for review by the Scientific Advisory
 Committee*

      Assuming  clearance through both the policy board and the SAC,  the  center
 "package"  is submitted to the project, officer*   The project officer Is  very
 Important  to the center because he/she is the legal and  institutional
 representative of record for the  EPA and Is responsible  for monitoring  the
 performance of the center and upholding EPAfs obligations under the terms  of
 the cooperative  agreement.   The policy board,  de jure,  deals with the center
 through the project  officer but,  de  facto,  the relationship is less clear
 because In some  cases  the policy  board chair and the  project officer  are
 the same  person,  while in other instances at EPA, the latter works  for  the
 policy board chair in  a supportive role.

      Following his/her review of  the  center's  budget  package,  the  project
 officer prepares  a decision memorandum which includes the comments  of ORB
 headquarters'  reviewers,  attempts to  reconcile  differing  views, and certifies
 that  the  proposed funds are  sufficient to carry out the  proposed  tasks.  The
 decision memorandum  Is forwarded  to  the director of OER.   At this juncture of
 the process It is approximately mid-August,  or nearly six weeks before  the
 start of  the new fiscal year.   The 015, task at  this point Is to actually
 locate money to  fund the center.   OKD's budget -process does not specifically
 designate  funds  for  a  specific  center,  but  rather provides  a general  account
 called the  decision  unit  for "interdisciplinary  research"  which also  Includes
 the peer review  grants program  and a  number of  smaller, miscellaneous Items.
 From  this  general  funds Category  OER  patches  together a budget  for  the
 center which is  then forwarded  to  the  Assistant  Administrator  for concurrence.
 Assuming concurrence,  the package  is  submitted  for reviews  and  concurrence
 by  the Grants  Administration Division within  the  Office of  Administration.
 This  latter office is  organizationally separate  from  ORD  and is,  In fact,
 headed by  another Assistant  Administrator.
2SInce each center Is subject to a slightly different budget cycle this
description does not conform exactly to the time frame of an individual
center's funding year.  The Subcommittee believes, however, that it has
Identified the key elements of the centers' budget process, if that pro-
cess was synchronized with EPA's fiscal year planning cycle.
                                       17

-------
      By  the  end  of  this  review cycle  several  changes are likely to have
occurred  to  the  original package  approved  by  the  policy board and the
Scientific Advisory  Committee.   If  recent  experience is a useful guide, the
center would not receive its  funding  until long after the fiscal year had
actually  begun.   Also, the  actual level  of funding  is considerably lower
than  the  original target allocation.   In its  fourth year of funding, for
example,  the Ecosystems  Research  Center  received  a  target allocation of
$680,000; its actual funding  level  was $420,000,  Similarly, the National
Center for Ground Water  Research  was  directed to  plan its fifth year program
within a  budget  ceiling  of  $650,000;  it  received  $420,000.   In FY'84 all
centers  received the same funding—$420,000.

IV.   FINDINGS AMD RECOMMENDATIONS

      A.  Issues from the Charge  to the_ Subcoaiaitt£e

          1.  New Center Themes

      The  Subcommittee was asked  by  the Assistant  Adnlnistrator for Research
and Development  to  identify new environmental areas or themes that,  in
future years, will  be of highest  priority  for the centers program.   Before
presenting specific  proposals  on  this issue t he $ubcoiaciittee _strong ly
reconraends _that_  ORD  and  the Agency  resolve the _ma|or problems that  pLague
the existing centers_j5ragram_'bgfQ^e.,gvg^ CQnsMe.rJ-_njj .fcfee establishrne..nit.. °A
new centers.^  At the present  time it  would be ,,f OQlish_to_eons_idgr new
c_e_nte_rs bejfojre^ the_current  budgetary,  leadership  and management problems
Qf_the_current progran are  corrected.

      There are several minimum  requirements that  should be  satisfied by ORD
before it considers  new  center  rhymes-   These include:

          a.  Significant nevr  funding  is available
          b«  OER should define EPA's high  priority  research  needs  for
              abating health and environmental  risk  during  the  next  five
              to ten years.  OER should  identify which  of  these needs  can
              be addressed most effectively by  university-based centers.
              Finally, OER should develop nominations for new center themes.
              For each of these three activities OER should consult  with
              the SAB Subcommittee on Strategic and  Long-Term Research
              Planning.

          c»  The solicitation process for  new  centers  should be competitive.

     Members of the Subcommittee have identified several environmental areas
which they believe warrant consideration for center  sponsorship by EPA.   In
addition, the Clean Air Scientific Advisory Committee (CASAC) of the Science
Advisory Board, in a December 1983 report to the Administrator  on Research
Needs for Setting National Ambient Air Quality  Standards, recommended  high
priority themes for new centers.  ORD should consider the recommendations
of both of these panels if the other minimum requirements are satisfied.
These recommendations are as follows:
                                       18

-------
 Extrapolation of quantitative experimental animal response data for
 the prediction of human responses.   Sponsorship of further research
 in this area through a center would contribute significantly to an
 understanding of the mechanisms of  the responses to pollutant insults
 to both^laboratory animal and hunan systems.   Results of such work
 are directly related to developing  the scientific basis  for standard
 setting under most of the statutes  that !PA is charged with implementing
 and enforcing,

 Research, relating air pollution exposures  to  doses received by target
 sites  within human populations.   Developing an enhanced  capability to
 assess pollution exposures is a critical element of risk assessment.
 Important  components of a center organized to evaluate the exposure-
 dose-risk  assessment links Include:  a) establishing relationships
 between concentrations  at air monitoring sites to human  exposures;
 and b) assessing biological  responses  to multiple pollutant exposures.
 While  directly applicable to supporting EPA's regulatory responsibilities
 under  the  Clean Air  Act,  the research  results of such a  center can
 enhance EPA's exposure  assessment for  land and water  pathways  analysis
 and can provide scientific support  for assessing the  human health
 risks  from air  and water  pollutants, hazardous chemicals  and  radiation.

 Evaluation of unused  areas for waste disposal,   EPA is currently
 surveying  the disposal  of toxic wastes  in  some relatively uninhabited
 or unused  environments.   For example.  It Is assessing  the envlroiraental
 and human  health impacts  associated with the  incineration of hazardous
 wastes  at  sea.   In addition,  it  is charged with establishing  criteria
 for the  disposal  of  high-level  radioactive wastes*  Waste  sites for
 the latter, many  of  which are located  in areas  of  low  population
 density, are  currently  under  review by  the Department  of  Energy*
 Creation of a center(s) for  desert .ecology and/or microbial ecology,
 or  expansion  of  the research  program of an existing center, will
 directly support  EPA's  responsibilities under  various  statutes and
will lead  to  a  greater  understanding of pollutant behavior and fate
in media where  the Agency's current depth of knowledge is limited*

Application of biotechnology  principles and techniques to pollution
control.  The evolving  science(s) of genetic manipulation creates
numerous opportunities  for environmental scientists to better
understand and neutralize  highly toxic pollutants.  Examples include
the use of microbiological techniques to detoxify hazardous chemicals,
such as some  pesticides, in soil systems and the development of
organisms that consume  toxic substances without themselves being
rendered coxic.  IPA could potentially utilize such state-of-the-art
practices for pollution abatement under most statutes that it
                               19

-------
        implements.   In addition, a long~terra, ample commitment of researchers
        and dollars  to support a biotechnology center will hopefully increase
        the Agency's knowledge of this important scientific innovation*

     »  Research related to monitoring of ambient levels of contaminants In
        water, soil, and air.  Accurate measurement of low levels of
        contaminants In the environment Is an essential facet of any clean—up
        effort, be It Nation-wide or at a specific location.  Problem sites
        need to be identified and progress in their remediation monitored.
        Creation of  a university center in this broad monitoring area would
        generate new basic approaches for sampling and analysis and would
        complement the more applied work of EPA's Environmental Monitoring
        and Support  Laboratories,

          2.  Sclentific Review of the Centers

     During Its review of the centers program the Subcommittee reached a
consensus that the concept of a centers program is worthwhile, that such a
program has the potential to achieve high levels of research productivity
if it is adequately supported by EPA, and that, in general, EPA should
have a centers program.  The research generated at most of the centers has
generally been of high quality.  The Scientific Advisory Committee review
process of the current centers Is appropriately focused and is carrying
out Its intended functions.  The program has proved to be a vehicle for
mobilizing talent from diverse academic sources, both intramural to the
institution housing  the center and frequently extramurally.  In addition,
the program has been partially successful in attracting funds from outside
of EPA, thus magnifying the EPA investment.  In short, despite EPA's
underfunding of the  centers, most of them have proven to be a worthwhile
research investment.

          3.  Centers Prograni_a_nd__OR_D_^_s: Mission

     Although the centers program is a part of ORD, it has not been clearly
articulated how the program carries out the mission of the ORD or of EPA.
As a result, there is confusion and reluctance on the part of ORD to encourage
funding such items as graduate student research, even though such research is
usually integral to  a centers program.  The ORD must communicate clearly
to the centers what their role should be in research, education, and service.

     Recommendations:

     a  ORD should create mechanisms to more closely integrate center activities
        whose those of its research laboratories and to encourage the labora-
        tories to use centers in the same disciplinary area for research, but
        not for technical services such as proposal reviews.  To the extent
        possible, each center should have a laboratory "constituency" with
        which to plan and/or carryout research.
                                       20

-------
      * Long-terns and  short-tern research goals and objectives need to be
        clarified.

      * ORB  should prepare  guidance which clearly states  the missions  and
        goals  of  the centers.

      » Exploratory studies should be  supported.

      • Centers should be encouraged to  exchange  staff  scientists with
        other  centers  and with EPA laboratories.

          4.  Duality  of Work

      The work  performed by  the centers is usually of high quality (see
Appendix D).  However, the  uncertainty of funding levels  undermines center
stability and hinders  fundamental long term  research.

          5.  _Rene_waI_ of Centers

      The low level  of  support  for the centers has resulted in their not
being centers.  In  drawing  this conclusion,  the Subcommittee  has examined
center sponsorship  at  other Federal  agencies, such as the National  Institute
of Environmental  Health Sciences  (NIEHS)  and the  National Science Foundation
(NSF).  Either the  amount of support for  the _centers program should be
increased to realistic levels,  or EPA should acknowledge  that it cannot
_f_und  such a  program and should terminate_it^.  The Subcommittee has  recognized
several options;

          a.  If  funding remains  at  the current level of  $ 3.4 million
              ORD should consider:

              •   reducing the  number of centers froci eight to  a number which
                 will provide more adequate funding  - probably four or five
                  centers, or

              *   distributing  funds on an unequal  basis to the eight centers,
                 or

              *   terminating the  program  entirely.

          b.  If  funding increases to a level such that each  existing  and/or
              new center would receive a mininum  support  of $800,000 to $1,000,000
              per center, OHD  should:

              *   increase funding to the  eight  centers but not necessarily on
                 an equal basis,  or

              •  add more centers
                                         21

-------
     Funding of centers should be based on a three year cooperative
agreement, followed by a second three year cooperative agreement.  Competi-
tive renewal should occur in the fifth year.  If the center Is competitively
renewed, it would receive two additional three year cooperative agreements.
If it failed to be renewed, it should receive 50% funding for an additional
year (year 7) as a transition to a total phasedown.

     A potential disadvantage of this fifth year competitive renewal is that
it does not provide for long term tenure of funding that centers such as
those funded by NIEHS have come to expect*  Consequently, ORD may also wish
to consider the renewal process followed by NIEHS or other Federal agencies
a$ an alternative to the competitive renewal process described above.

          6.   Cmauctlng Research Results
     Technical exchange between the Agency and centers is mutually
beneficial and should be encouraged.  On the issue of technical transfer of
information, the Subcommittee finds three areas of particular concern.
These include;

          a.  information exchange;

          b.  technical transfer involving personnel exchanges; and

          e-  specific response to Agency initiatives, which are related
              only in principle to the research mission of the center.

     Re_c Qmme nda_t i_o ns_ :

     *  OER should develop standard guidelines for an annual report of
        center activities.  This report should be a substantive summary of
        aJQ center activities (both EPA and non-EPA funded),  OER should
        act to coordinate the dispersal of the reports to other centers,
        EPA laboratories, program offices, and other appropriate Agency
        personnel,   OER should assume an active role in summarizing its
        supported research to the entire Agency.

     »  No more than 10% of core funding should be devoted to information
        exchange or technical transfer (items 6a  and 6b).  This should be
        a "rule-of-thurab" directive, and final decisions should be made by
        the center director.

     e  There exists a continuing need for EPA to make an internal organi-
        zational link between research and program Initiatives.  Something
        akin to the Office of Integrated Technical Analysis proposed by
        the National Acadeay of Sciences in 1977  remains a useful suggestion,
        The operation of the ORD research committees has, to date, failed
        to meet this need for Integration of research,
                                       22

-------
     B.  EPA Management of Center_s_

     In the Subcommittee's view, what is critically needed at the present
time to promote the success of the centers program as a whole Is a. quality
of leadership in ORD that understands what a centers program should provide
to IPA, what factors lead to success or failure, and what management and
budgetary resources are necessary for a successful program.

          1,  The centers are currently subject to reviews by the policy
     board for programmatic and budgetary oversight, by the Scientific
     Advisory Committee for peer review of the research program, and periodic
     reviews by ORD headquarters.  There are currently an excessive number
     of such review meetings in relation to the amount of resources allocated
     to the centers.  The net result is that administrative costs of running
     the program are too high relative to available dollars.  The program
     is being over reviewed and over managed.

     Recoroaendation;  Both the Policy Board and the Scientific Advisory
     Committee should convene only once per year, ideally at the same time.

          2.  A number of institutional relationships relating to SPA's
     management of the centers program need to be clarified.  These include
     the relationship between the project officer and the policy board; Che
     stability of the policy board and the knowledge of Its members; and
     the role of ORD headquarters and the Assistant Administrator in annual
     sign-off of support for the centers.

     Re comme ndations!

     *  At  the current level of funding the need for separate reviews by
        both a project officer and the policy board is not apparent,
        Where possible, the project officer and policy board chair should
        be  the same individual, preferably a senior laboratory official
        who understands the technical basis of a center's program and is
        knowledgeable about ORD policy issues.

     *  Members of the policy board should be appointed for fixed terms,
        staggered to promote the continuity of knowledgeable membership
        while gradually encouraging new participants.

     *  At  almost any level of funding, a review of a center by ORB head-
        quarters need only occur when a center is up for a renewal decision,
        when there is a a change in leadership, or when a special problem
        arises.  After the initial awarding of a center agreement to a
        university, ORD should delegate the sign-off authority for the
        center's annual support to the project officer*  There is no need
        for the Assistant Administrator to annually approve the funding.
        One objective of this proposed delegation of sign-off authority is
        to  ensure that budgetary authority resides at the same level of
        ORD where the responsibility for managing the program exists.
                                       23

-------
     3.  There is a need for OER to develop more explicit  procedures
concerning the replacement of the center director.

Recommendation:  In the event a center director leaves  the  institution
of is unable to carry out his/her duties» an acting director  should be
appointed on an interim basis by written mutual agreement  with OER.  A
permanent director should be selected as soon as possible  (the acting
director could be an eligible candidate) to ensure continuity in  the
direction of the centers research program.  The new director  is expected
to possess scientific and management capabilities at least  equal  to
those of the previous director and must be acceptable to EPA  staff and
the policy board, with the advice of the Scientific Advisory  Committee.
The appointment of the new director should also be confirmed  in writing
between the center and OER.

C.  The_^Budget_ Process for the Centers Prograia

     1.  The Subcoiamitteee reached several findings about  the budget
process.

         a.  The intensive preparation of the centers to develop  and
             continue a research program within a given level of  resources
             is subverted by the ORD/EPA budget process*   What, in fact,
             occurs is that after extensive contingency planning  by the
             centers and review by the policy board and the Scientific
             Advisory Committee, budgetary choices are  made in the absence
             of a clearly defined rationale*

         b«  One of the central purposes of strategic research planning
             in any organization is to inform the budgetary process, i.e.,
             to provide an analytical basis for prioritizing  alternatives
             and selecting those which are affordable.  The budgetary
             process should not subsume the planning function, for if
             this occurs Identification and discussion  of  many potentially
             creative alternatives will not take place.  The  current ORD
             practice of patching together various funding  accounts to
             fund centers, to allocate monies sustantially below  the
             original budget ceiling, and then to provide  the same amount
             of funds to each center after the fiscal year has already
             begun is counterproductive to good research management and
             is analytically bankrupt*

         e*  The budget	process in.^any__Qrganisation is  characterized^ by_
             _complexity,_but__c_omplexity	nee_d__not_^ob_scure _the  lines of
             authority and responsibility for ensuring  program success.
             The^EPA/QRD budget process ^separat^es^^e^authorlty of those
             individuals who provide funding for the centers  from the
             individuals who are responsible for the day~to~day center
             operations*  This separatlot\_gf ^authority  and^ responsibility ,
             in practice, results in a situation where  everyj)tie is to
             blame and, therefore>_flO_one_ls__to_blani6«  This  is the
             definition of bad management.

                                  24

-------
     lecotiaendatlons;

     *  Allocations of resources should be received by the centers  prior  to
        the start of the annual project period.  The project officer and
        the director of OER should work closely with the center directors
        and the EPA Grants Administration Division to assure that funds are
        transmitted expedltiously.  The director of OER should be accountable
        and responsible for ensuring that resource transmission occurs in a
        tlaely fashion.

     •  ORB should Identify ways of strearaling the multiple layer of reviews
        in the centers' budget cycle Including the delegation of more
        budgetary siga^off authority*

          2. "Core" support for centers

     Centers require a critical mass of professional staff, support staff,
equipment, facilities and space and research funds to carry out a viable
research program.  Within this context, centers receive core support froa
EPA,  Core support refers specifically to salary and service support
allocated to individuals associated with a center, not including research
support.

     Recommenda tioni

     *  OER should identify which portions of a center's budget constitute
        the core budget and those'elements that comprise funds for research.
        However, the Subcommittee does not assume that each center has the
        same core support needs.  The support needed by nost centers to
        maintain a high level of research productivity ranges froa a minimum
        of $800,000 - $1,000,000 per year to a maximum of $2,000,000 per
        year (including "both core support and research funds)*

     B.  EPA Leadership for	the Centers

          1»  Throughout the years of the centers' existence there has been
     no effective spokesperson or advocate within EPA, especially at QRD
     headquarters, to ensure the success of the program.  As a result, the
     centers continue to be treated as orphans whose existence Is tolerated,
     but they are not adequately supported or effectively utilized.

     Reconnendatioti'   The advocate for the centers program must be the
     Assistant Administrator of OSD.  Only the AA has the scope of authority
     and span of control to initiate actions that lead to more effective
     utilization of the centers as a major scientific resource within ORD»

          2.   The Subcommittee believes that the quality of leadership within
     OER has not been effective.
                                       25

-------
     Recommendations;  The following criteria should be adopted for selecting
     future directors of OEE;

     •  committed to the EPA centers concept and the operation of the
        centers;

     •  committed to support of long-term research needs of the Agency;

     *  an advocate of optimal financial support for the centers;

     •  possess scientific competence and professional experience in
        managing a long-terra research program;

     *  have the support and respect of the research community and senior
        policy officials at SPA;

     *  able to provide the intellectual leadership necessary for a centers
        program; and

     *  capable of clear communication, of policy decisions and management
        guidance.

     E.
     One point of consistent confusion encountered by the Subcommittee was
whether the centers are EPA centers and only to be supported with SPA funds
or whether they are university centers whose cote support Is funded by EPA.
For the long haul, the latter approach is more desirable.  The EPA support
should be considered as base support used to initiate Important centers at
various locations,  The SPA base support should be used as "leverage" to
enhance the success of the center.

     Recommendations;
     •  centers should be reviewed on a regular basis and be phased out/
        terminated if performance does not yield sufficiently high quality
        research that benefits EPA.

     *  centers should seek a broad  base of support and participate fully in the
        academic community without being obligated to only one sponsor.

     •  so long as both EPA and the  centers contribute and benefit from the
        cooperative agreement relationshipt there should be no tirae limit
        set for the duration of EPA support.

     •  EPA should have the management flexibility to alter the mix of
        centers to reflect its changing Information requirements over the
        long~term«  As the Agency's  role as a sponsor of research In the
        Federal government evolves (for example, it is currently a primary
        sponsor of ecological research but not a primary sponsor of health
        research)  so will its need for specific kinds of centers.
                                       26

-------
                                 Appendix A

                               Charge _.tp_the
                  Subc_oiamittee___on Strategic and Long-Tern
                             Research Planning.


     In December 1983 Dr. Bernard D. Goldstein, Assistant Administrator for
Che Office of Research and Development (OID) formally requested the Science
Advisory Board (SAB) to assist in the review of OED's Research Centers
Program.  The Subcommittee on Strategic and Long-Term Research Planning was
formed by the Executive Committee of the SAB to carry out that review.

     Specifically, the Subcommittee has been asked to review the following
issues;

     1)   Examining what environmental areas or themes will
          be of the highest priority for the centers program;

     2)   Assisting OID in conducting a scientific review
          of the centers, associated with QRD's scientific
          and management review of the programs

     3)   Reviewing the effectiveness of the centers
          program as a neatis of carrying out OfiD's mission;

     4)   Advising on the quality of the work performed by
          the centers}

     5)   Advising QRE> on the issue of options for renewal
          of centers;

     6)   Examining how research results generated by both the
          Research Centers Program and the feer Review Investigator
          Initiated Grants Program can be more effectively
          communicated to ORD's laboratories and to EPA's
          program offices•

     Six research quality review criteria have been identified for use in
evaluating the centers program:

          " The Research	Program - From a scientific point of view,
            Is the research program well designed and focused?
            Has a specific research objective been identified?
            Within the context of the centers thenie assignment»
            Is the research objective of major interest to the
            scientific community?  Will the research be useful
            to EPA or to anyone else?
                                      A-1

-------
  Research Quality and Control - Are  there  acceptable peer
  review procedures  to ensure the quality of research
  projects and publications?  Is the membership of  the center's
  Scientific Advisory Committee (SAC) well  qualified and relevant
  to  the needs of the center?  Does the SAC represent a national
  scope of interest?  How has the SAC been  utilized?  Does it
  retain objectivity?  What other measures  have been taken to
  ensure research quality?
          Assurance -  What measures are taken to define the
  quality of research data generated?  Are statistical methods
  being used in experimental design and data interpretation*

~ .Non-Center Considerations - Are there any considerations
  outside the center director's control that might impact on
  this evaluation?

~ Communications -  To what extent has there been interaction
  between SPA and the centers?  Is there communication between
  the centers and the EPA laboratories and EFA's program offices?

" Intra-University Linkages - How well do the centers act as
  magnets to draw in other researchers within the University
  to participate in research projects?
                           A-2

-------
                                   Appendix B

                     Subcommittee on Strategic and Long-Tern
                                Research Planning
Dr* John M. Seuhold (Chairman)
Department of Wildlife Sciences
College of Natural Resources
Utah State University
Logan, Utah  84322
            Mr.  A.  Robert Flaak
            Executive Secretary
            U.S.  Environmental Protection Agency
            Science Advisory Board
            401  M Street, S*W.
            Washington, D.C*  20460
Dr. Clayton Callis
Director
Environmental Operations
Monsanto Fibers and Intermediates
  Company
800 N. Lindburgh Blvd.
St. Louis, Missouri  63167
            Dr.  Keros Cartwright
            Illinois State Geological Survey
            615 East Peabody Drive
            Champaign, Illinois  61820
Mr. Richard A. Coaway
Corporate Development Fellow
Union Carbide Corporation
P.O. Box 8361 (770/342)
South Charleston, West Virginia
25303
Dr* Edward F. Feerand
Assistant Commissioner for Science
  and Technology
New fork City Department of "
  Environmental Protection
51 Astor Place
New York, New York  10003
Dr* Robert Frank
Department of Environmental and
  Health Science
The Johns Hopkins School of Hygiene
  and Public Health
615 N. Wolfe Street
Baltinore, Maryland  21205
            Dr. Leonard Greenfield
            1221 Columbus Blvd.
            Coral Gables, Florida
                       33134
Dr»' Joseph Koonce
Department of Biology
Case Western Reserve University
Cleveland, Ohio  44106
            Dr. Michael Lebowitz
            Professor of Internal Medicine
            University of Arizona
            Health Sciences Center
            Tucson, Arizona  85724
Dr. Morton Lippmann
Institute of Environmental
  Medicine
Lanaa Laboratory
New York University
Long Meadow Road
Tuxedo, New fork  10987
            Dr. Raymond Loehr
            Professor of Agriculture
              Engineering and Professor
              of Engineering
            207 Riley-Robb Hall
            Cornell University
            Ithaca, New York  14853
                                       B-l

-------
Dr. Francis McMichael
Department of Civil Engineering
Carnegie-Mellon University
Pittsburgh, Pennsylvania  15213
Dr. Daniel Menzel
Professor of Pharmacology and
  Experimental Medicine
103 Jones Building
P.O. Box 3813
Duke University Medical Center
Durham, North Carolina  27710
Dr. JaTnes Porter
President
Energy and Environmental
  Engineering, Inc.
1-8 Monsignor O'Brien Highway
P.O. Box 215
East Cambridge, Massachusetts  02141
Dr. Frank Speizer
Channing Laboratory
180 Longwood Avenue
Boston, Massachusetts
02115
Dr. James Whittenberger
Director
Southern Occupational Health Center
19722 MacArthur Blvd.
University of California
Irvine, California  92717
                                       B-2

-------
                                    Appendix C

                                ORD Research Centers
Theme

Epidemiology Research
  Center
Institution

University of Pittsburgh,
  Pittsburgh, Pennsylvania
Starting Date

October 1979
Advanced Environmental
  Control Technology
  Research Center
University of Illinois,
  Urbana, Illinois
                                                                  October 1979
National Center for
  Ground Water Research
Consortium!
  University of Oklahooa,
    Norman, Oklahoma
  Qklahooa State University,
    Stillwater, Oklahoma
  Rice Universitjj
    Houston, Texas
                                                                  October 1979
 Industrial Waste
  Elimination Research
  Center
Consortium:
   Illinois Institute of
    Technology, Chicago
    Illinois
   University of Notre Dame,
    South Bend, Indiana
                                                                   October  1980
 National  Intermedia
   Transport  Research
   Center
 University of  California
   at  "Los  Angeles,
   Los Angelesa California
                                                                   October 1980
 Icoystetns Research
   Center
 Cornell University,
   Ithaca,  New York
                                                                   October 1980
 Marine Sciences
   Research Center
 University of Rhode Island,
   Kingston, Rhode Island
                                                                   October 1980
 Hazardous Waste
   Research Center
 Louisiana State University,
   Baton Rouge, Louisiana
                                                                   October 1981
                                        c-i

-------

-------
                                    Appendix D

                                REVIEW OF THE
                         EPIDEMIOLOGY RESEARCH CENTER

                                 April 2-3, 1984
INTRODUCTION
     The Site Visit Team met at the Graduate School of Public Health, University
of Pittsburgh, on April 2 and 3, 1984,  Team members Included;  Robert Frank,
Michael Lebowitz, Morton Lippmann, Frank Speizer, James I,. Whittenberger,
Terry F. tosie (SAB Director) and Robert Flaak (Executive Secretary)*


GENERAL COMMENTS
     This was one of the first group of centers established by Cooperative
Agreements with EPA, and Is now in its fifth year.  It is the only center
prinarily concerned with health effects of environmental pollutants.  However,
there are several cogent reasons why this center has not reached the maturity
of purpose and program »hlch one would normally expect la a five-year old
center.  These factors were discussed In Subcommittee nestings in February
and March, and were known to the sitt visit team prior to the visit*  The
site visit extended our knowledge of the problems, as well as the improvements
made by the university participants in the past eight or nine months.  Soae of
the problems are common to all the centers; others are peculiar to this
center.  The problems will be listed now and some of them amplified later.

    General_Problenis;

          Budgetary uncertainty, reductions, misunderstandings.

          Lack of  policy direction and consistency in Office of Exploratory
          Research.

    Special problems _o£ Pittsburgh Center;

          Chaotic  history of  policy hoard  -  frequent changes of chairmanship,
          unstable  policy guidance.

          Absence  of  any laboratory or program activity  in  ORD  to which  the
          center could relate  (the epidemiology intraaural  program was
          abolished  by SPA in  1981),
 CONDUCT OF THS SITE_VISIT

      The center director provided a substantial set of documents in advance,
 which gave the site team a good background for the visit.   By agreement,
 Dr.  Interline also developed an agenda which would provide the site visit
                                       D-l

-------
 team opportunities to hear from and to interrogate key faculty members of  the
 center.  On the second day, the site visit team visited the facilities and
 had extensive discussion with the center director (following an executive
 session).

      Two persons important to the center were not present at the site visit:
 Professor Lewis Kuller, Chair of Epidemiology, is away on sabbatical leave,
 and Dean Raymond Seltser was out of town.  Dean Seltser called the Chair of
 Che site visit team to express his strong support of the center; this Is more
 than routine support because Dean Seltser Is himself an environmental
 epidemiologist.


 OBSERVATIONS AND CONCLUSIONS

      A.  Leadership of i the center.

      Until  September  1983,  the director of  the Pittsburgh center had been
 Professor E.P,  Radford,  Jr.,  who has resigned and left the university.
 Granting that EPA's contribution to the cooperative  agreement  was deficient
 and sometimes not  in  the best  Interests of  the center, it is also apparent
 that the quality of leadership at  the  institution was different under Radford
 than it is  under the  new director,  Philip Interline,   Since  September 1983,
 the center  has  identified four major themes  for future research,  has organized
 the role of the  advisory committee and  appointed  a chairperson  for  the  first
 tine,  has occupied  integrated  space, has  developed regular staff  meetings,
 and in oeher ways has  created  a  sense of  unity and purpose.

      8'   .Thg..m®s> or environmental  health _ research^areas  of highest^griority.

      During the  first  four  years of  the center, there  was  no focus  for
 long-term research.  The  EPA solicitation which led  to  the establishment  of
 the center  had no impact  on  the  activities of  the  center.  A large  amount  of
 technical advice was sought by various  components  of EPA,  including the
 laboratories, and this burden  of service  continues to  Che  present.  The policy
 board  set a  limit of 20%  of the  budget  for technical services,  but  this limit'
 was not  adhered  Co.  At one point, the  center  was asked to serve as "Project
 Officer"  for all SPA epidemiology contracts.

     At  another  tine the  policy board asked the center to  set up five task
 forces  to determine EPA priorities for  long-term research.   Task Force members
 thought  they were planning EPA's research program, not the center's program.

     The  planning effort  that started last fall identified three research themes
 to  which a fourth was recently added.  These are four of the topics on the
agenda—studies relating to hazardous waste sites, monitoring of adverse
reproductive outcomes, studies of certain indoor air pollutants, and methodology
of  risk assessments.  In general these  Involve critical "state-of-the-art"
reviews, coupled with possible specific  research plans.  The topics identified
are appropriate research  themes for an environmental epidemiology center.
                                      D-2

-------
     C.   Quality of the research program,

     In terns of epidemiologic studies attributable to IPA's support of the
center, there is little to evaulate.  This is not a reflection on quality of
faculty in the center, because most of those faculty have been doing good or
excellent research with sponsorship by other sources.  Many publications are
attributed to the center, but most of these are publications of proceedings of
the annual symposia sponsored by the center.  Some of the syiiposia have been
of high quality and responsive eo high priority needs; others have been
unreaarkable»

     Dr. Enterline is a widely respected and productive statistician and
epidemiologist, one of the pioneers in occupational epidemiology in this
country.  He is thoroughly aware of the needs and is a practitioner of high
standards in quality control, peer review, and uses of statistical methodology.
His associates  in  research are also of high quality, including some »ho are
not identified  with the center (and probably should be).

     The  Scientific Advisory  Committee has been  very little used until  this
year.   The SAC  now has a  chairman, a  regular meeting schedule, and a principal
function  to  review research  proposals and participate  in  quality evaluation.

     D,  Interactions  with EPA.

     EPA's  project officers  were complieented  for their understanding  and
 their  efforts  to  assist  the  center,  but nothing  else  in EPA's  role  was
 praiseworthy.   In 1980 the  center was informed by the  Office  of  Exploratory
 Research that  $500,000 was  available  for  research projects.   The center
 prepared and submitted nine research projects  for peer review, only to learn
 that'the $500,000 was supposed to be the  core  budget  for the  second year,
 Since eight of the nine projects were already  committed,  the  core  had  to be
 reduced from $346,000 in the first year  to $130,000 in the second year.  This
 was only one example of the "screw-ups"  in the core budget.

      The center felt obliged to respond to all requests froo EPA,  whether
 from a regional office, a laboratory scientist, or headquarters.  Center staff
 thus did feasibility studies (e.g., arsenic in drinking water), planned research
 protocols that were never used (tove Canal), organized workshops.reviewed
 grant  proposals, etc.  The center has not attempted to quantify this service
 to EPA, but estimates it at  30 to 40% of center effort. ,

      The policy board has been the major source of input  froa EPA, in  spite
 of the frequently changing chairmanship, lack of guidance as  to what  the
 center was  supposed to accomplish, etc.  There  was no Input from research
 committees  when  the policy board  asked the center  to  establish  the ,ive
 planning task  forces  and no  feed-back to the center as to whether the  task
 force  reports  had any impact on  EPA  research  planning.   As one  senior  staffer
 in headquarters  described the role of the  policy board,  it was  a  deplorable
 state  of affairs."
                                        D-3

-------
      EPA never communicated  clearly to the center what it's role In research,
 "education"  and "service"  should be.   The center used EPA funding for
 establishing two courses  in  Environmental Epidemiology in the School of Public
 Health,  for  supporting student  research projects, and for the annual symposia
 (which were  considered "education").   Eventually the  policy board put a
 budgetary limit of  5%  on student projects.   They also approved use of 20% of
 the  budget for "service,"  but made  no  effort to  monitor this restriction,


 CONCLUSIONS

      A,   The first  four years of this  cooperative agreement should be largely
 ignored  in evaluation  of this center for  renewal.  Most  of  the fault was
 EPA's.   Reasonably  good progress has been made since  August 1983  because  of
 new  leadership  of the  center, stabilisation  of the policy board under the
 present  chairman (Director of the Health  Effects Research Laboratory),  and
 efforts  of the  project officer.

      B.   Effective  integration  of the  center into EPA's  long  range research
 and  research planning  faces  fomidable  handicaps.  Effective  linkage with EPA
 Is impossible  until EPA re-establishes  epidemiology as  an Important  intramural
 program.

      C.   Although scientists in  the center are excellent, the  center is
 using faculty resources who are  at the  institution, but  not  identified  with
 the  center.  This is particularly true  in the areas of environmental  exposure
 measurement  and  reproductive biology,  two areas  which are very  important  in
 the  center's  program.

     B,   The center should be more aggressive in  seeking  non-EPA support  for
center-related  research and more willing co  identify extramural support they
already have, as closely related to center objectives.

     E.   Technical service activities should be curtailed in favor of greater
emphasis on  research.   Center funds, such as they are, should be conceived as
core  support  for faculty and specialized facilities,  supplemented by  project
support from  public  and private  sources.

     F,   In fairness to the University  of Pittsburgh,  another three-year
renewal period should be considered,  with competitive  renewal by the end of
the seventh year (May 1986),

-------
                                REVIEW OF THE
                        ADVANCED ENVIRONMENTAL CONTROL
                      TECHNOLOGY RESEARCH CENTER (AECTRC)

                                April 16, 1984

INTRODUCTION ,

     The site visit was held on Monday, April 16, 1984, at the Engineering
Experiment Station of the College of Engineering.  Members of the site visit
team present were: Clayton P. Callls, Francis C. McMichael, James H. Porter,
Terry F, losie (SAB Director), and Robert Flaak (Executive Secretary).

     The team was hosted by: Dr. Richard S. Engelbrecht, center director and
Dr. Willlan A. Cawley, Deputy Director, EPA IERL, Cincinnati, center project
officer.

SUMMARY

     The site visit team concluded that  the Advanced Environmental  Control
Technology Research Center  (AECTRC) is well managed; the research is appropriate
to  the center's mission; the quality of  the research Is excellent;  the research^
team demonstrates enthusiasm; a  cooperative spirit exists between the Individual
researchers  and the center; and  the productivity of the research effort has
been high.   This  general status  o£ AECTRC has been attained  in spite of program
budget cuts  and confusing signals received fro* EPA Headquarters because of:

     a) the  management  skills of the center's director;
     b) the  commitment  of the university researchers to
        the  center and  the  auxllllary  support they lend
        to  the  center through the excellence  of  their
        individual reputations;  and
     c) the  direction provided  by the  center's  EPA project
        officer in directing the center  through EPA's
        taanagement and  budget maze.

     The  review team  concludes  and  recommends that;

     a)  the research  conducted  at  this center is appropriate
        and beneficial to  the Agency;
     b)  the Agency provide  consistent guidance  and  support
         of  this center at  the headquarters level of  ORD; and
      c)  the annual funding  commitment to this center
         be  increased by at  least two-fold so that the
         center can effectively  carry out its mission*

 AECTRC RESEARCH PROGRAM

      The  center's research program and its mission is best described in "Summary
 of Research Activities, 1983" the report from the center director,   A copy of
 the director's report Is available for review in the SAB's offices.  The program
 focuses on air and water pollution control research.  Near-term research

                                        D-5

-------
 examines biological degradation of pollutants on activated carbon, activated
 carbon regeneration, preeharging aerosols, H202 and ozone oxidation of
 dissolved hydrocarbons.   Longer range research is examining super critical
 fluid extraction.   Each  of the research projects focuses on gaining a
 fundamental understanding of mechanisns so as to improve processes.

      The researchers are primarily professors from civil and chemical
 engineering and their graduate students.   The research projects often involve
 interdisciplinary  teams.   The research staff has excellent credentials and
 their reputations  lend to the credence of the center's activities.   Much of
 the work conducted by the researchers is  not funded through the EPA and the
 center.   Thus,  the Agency gains by having highly trained staff  who are not
 fully supported by the Agency working on  projects.

      The quality and productivity of  the  AECTRC research program are good.
 Forty-two (42)  peer reviewed  journal  articles have  been produced by the
 program  and 12  of  these  have  been generated  through the research effort.
 The center has  excellent  facilities and i$ well equipped to conduct research.
 Again the Agency gains since  much of  the  facility and  equipment is  supplied
 by  the university  and  not  through EPA funding.

      The center has attracted  funds frora  outside  of the EPA,,  for example:

      A project  sponsored  by the  Army  Corps of  Engineers to  examine
      biodegradatlon as a  means  of decontaminating soils.

      The  center has also  been  involved  in  other activities  such as
      entering an agreement to  train Japanese environmental  engineers
      under  the  Engineer Exchange  Provisions of  the  U.S.  - Japan
      Bilateral  Agreement on the  Environment.  Additionally, the  center
      has  sponsored  seminars to  bring  together expertise  from around  the
      country to  discuss current  research problems.

       It  is concluded that the center has productively  used its  EPA funds, and
its research is  focused and of high quality.

MANAGEMENT AND_JUDGET ISSUES

     The center receives  direction from the Agency through two mandatory
groups:

          The center's policy board, which establishes
          appropriate areas to conduct research and
          establishes guidelines for interaction with
          the Agency; and

          The center's Scientific Advisory Committee,
          which  reviews and approves individual research
          proposals, follows  ongoing research and with
          the policy board conducts periodic  program
          reviews.
                                      D-6

-------
     The activities of the policy board and Scientific Advisory Committee
are funded by the center's budget.  Thus the management of the center and
the support of the mandatory groups become a part of the fixed administrative
costs' of operating a center.  This cost amounts to $2QO»OOQ-$25Q»OGO annually.
At a one million dollar funding level this amounts to 20 to 25 percent cost
for administration which is reasonable.  At the current $400,000 total
funding level the administrative cost range from 50 to 63 percent of the
total budget which is clearly out of proportion.  The center feels that
both committees serve useful functions*  Thus, either the center funding
should be increased since the incremental funding would be applied to
direct research support, or the administrative budget be reduced primarily
by reducing the administrative burden of the policy board and Scientific
Advisory Committee*

     Currently the center has only  $150,000 to apply  to research, which
makes it difficult to support ongoing research and impossible to start new
research projects.  Most of the center's problems have been associated with
Inadequate budgets relative to the  original concepts  of the centers, not
receiving  clear  and timely  signals  from the Agency as to  the  level  of annual
funding  to be received, and not receiving  the  funds when  they were  expected.
Thus, the  center has  had  to operate on  university budgets while awaiting
EPA funding.

     This  has made  it difficult  for the center to develop adequate  research
plans and  placed significant  strains on the relationship  between  the center
and the  EPA.   It Is  clearly to  the  benefit of  the Agency  to  relieve these
 Strains.

 OTHER AGENCY ^ CENTER INTERACTION^

     A  major function of  AECTE.C is to transfer technology and information
 it has  developed to  the Agency.   The center has attempted to accomplish
 this function through seminars  and through having members of the  Agency on
 its policy board and Scientific Advisory Committee.   Additionally,  the
 center publishes its work In refereed journals.  Thug, the extent that this
 information is not being disseminated in the Agency tends to reflect on the
 lack of interest by Agency personnel in the research efforts, rather than
 on the center.

 CONCLUSIONS AND RECOMMENDATIONS

      It is clearly to the benefit  of the Agency to have outstanding university
 scientists and engineers working on long-range environmental problems.  The
 AECTRC program effort clearly demonstrates these benefits.  To the extent
 that centers can be developed which exhibit the qualities of this center, the
 program should be championed and supported by high level personnel within the
 EPA, certainly including the Assistant Administrator  for Research and
 Development.
                                        D-7

-------
™.M   /        Environmental Control Technology Research Center deserves
continued support and its annual funding level should be Increased to a level

            "   l
                                                                     committed
  fcv     1   S  3nCe        aCtUSl reCSipC 3° thaC the Center "n ffiore
affectively plan It. research program.  Finally, funding should be received
when due so as  not to place a strain on the university's cash flow.
                                    D-8

-------
                    REVIEW OF THE ECOSYSTEM RESEARCH CENTER

                                 April 23, 1984
INTRODUCTION
     The site visit team net with the staff of the Ee«7strat ^search
Center at Cornell University in Ithaca, New York on April 23  1984.  The
tel timbers include;  Uooard Greenfield, Joseph Koonce, John M. Meuhold
(Chairman) and Robert Flaak (Executive Secretary).

     The site visit was conducted in a relaxed atmosphere with the  in teat
to keep the discussions at a dialogue level.  Presentations were  i»*>"
allowing opportunity  for  the site visitors to ask questions, mke sta
for reaction and  to generally  get a "feel" for the ecosystem center s
operation, its  staff, students, past performance, future plans and  ca

     The site visit team  centered Its  discussions with the  staff  of the
Ecosystems Research Center staff about  the central  theme ^ the  purpose of
the centers  program:   1)  fundamental ecosystems  research;  2)  interaction
Sth  ift?a"  and extra-  university peers;  3)  interaction with EPA laboratories;
Ed 4)  interaction with EPA program offices.  After gtining their Action.,
 the  team mt in executive session  to  evaluate their individual °f ef^™s
 and coiae to  a  concensus with their  assessment of the center.   This report
 is a  summary of this assessment*


 ECOSYSTEM RESEARCH CEMT1R ORGANIZATION

      The center is now in its fourth year of existence having been established
 with the objectives to:  1) identify the fundamental  principles and  oncepts
                     -
             rob     if concern to EPA through a consideration of retrospects
and other case studies.
                *  i j v« n»-  vin assisted "by Dr« Barbara Bedford and
                Is led by Dr» a«A* L.evin assiai-eu. "j
                         on-^pus
           rpnt«>r  s  e        »   «*  .           -.
  Dr.  Mrt Sr»eU  with  „ on-^pus  c«,nple»ent of sclen^.ts £ro» such varied
          with the center on the Cornell campus for extended periods adds a
  dynamic sense of diversity to the group-

       The work plans of the center are reviewed by a Science Advisory Committee
            of group of eminent eulogists including:  Herbert Borman,
           lolling, Fred Mackenzie, Frank Mgler, David Schindlar, John Steele,
  and George Wood well.
                                        D-9

-------
       The work of both the staff and the Science Advisory Committee Is in turn
  evaluated by a Science  Review Committee comprised of an independent but
  equally eminent group of  ecologies Including:   Jerry Franklin,  Evlll, Corham,
  David Reichle,  Paul  Risser,  and Richard Wiegert.


  ACTIVITIES__AND ACCOMPLISHMENTS^

       During  the  three years  since  the  initiation  of  the  center,  86 reports  and
  publications  have been  produced by its  staff.   Fifty five  of  these were  as  *
  direct  result  of EPA support.   The remaining 31 were produced  by  the  staff
  during  their  tenure at  ERC even though  the work t*as  not  supported  by  EPA.   In
  addition  to  the  publication  record the  staff has  been  invited  to  present
  papers and seminars at  a variety of meetings and  institutions.  Some  56  such
  presentations have been made(

      The quality of the work appears to be high.  Most of  the  papers  are
 published in peer reviewed journals or are published as books  which also
 undergo peer review,   Several of the reports are  issued as reports of  the
 center which also undergo a peer review before publication.  The content of
 the reports and papers range from theoretical considerations of environmental
 questions to applied  resolution of various environmental problems.  One of
 the major significant efforts of the center is the conduct of workshops
 dealing with synthesis or integration sciences.   Their effort in ecotoxicology
 is * good example of  such synthesis work which will likely result in the
 definition of an emerging science of immense  importance to EPA.  Further
 details on publications, reports and presentations are available  ia the
 A r r i /*> o *                                                             v *
-------
GENERATION OF EXTRAMDRAL SUPPORT

     The center has been very successful in generating support from sources
other than EPA £or activities which ultimately will benefit EPA's mission.
For example, the staff has been successful in obtaining research and training
funds from the Andrew W. Mellon Foundation for the program in ecotoxicology.
Research support has come from NSF, EPRI, Hatch Act Funds, Mdntire-S tennis,
and the U.S* Forest Service.


DISCUSSION AHP CONCLUSIONS^

     The goals and objectives of  the  center are most  relevant to  the mission
of EPA.  The nature of many  of our environmental  problems and the intent  of
much of our  regulatory legislation requires a sound understanding of ecology,
an understanding that cannot be gained from existing  textbooks on the  subjec-
Theoretical  concepts need generation, testing and ultimately application.
The objectives of  the center and  the  center's performance on those  objectives
seem to us  to  be right  on the mark*
for
      The leadership demonstrated by Levin is unquestionably responsible
 the success of the center to date.   He has been responsible for P^ting
 together a group of scientists committed to applying theoretical knowledge to
 EPA's applied problems.   He has been successful in forging diverse groups
 into producing scientific units.  And he has been successful in opening and
 maintaining dialogues with prospective EPA user groups.   In spite of these
 accomplishments in administrative leadership, Levin's real strength lies in
 his ability to generate ideas and by so doing to stimulate others into
 following his lead.

      Much of the everyday administration - the Incessant meetings with people
 from EPA, the incessant reviews, the changes in signals from the centers
 ptoeram-all serve to detract Levin and his cohorts from the substantative
 science that needs to be accomplished and which they are capable of doing.

      We find it difficult  to imagine any laboratory group  that could be more
 productive than the  staff  of this center.  Their record of accomplishment in
 the  three  short years of their  existence is  truly remarkable.  Much of what
 has  been accomplished has  been  done with less  than optimal funding.  Worse,
 it has been done with an uncertainty of  funding that requires  too much time
 of the leadership and taakes  the planning of  a  significant  long-term program a
 frustrating experience.

      The  center staff have spertt a  lot  of  time communicating their capabilities
 to various institutions  within  SPA  often with  the help of  their  policy board
 which Is  comprised  of representatives  from the laboratories and  the program
                                       D-ll

-------


five.
                                    D-12

-------
                                REVIEW OF THE
                  NATIONAL CtNTER FOR GROUND WATER RESEARCH

                              April 24-25, 1984
INTRODUCTION
     The National Center for Ground Watar Research was one of the first
three centers established by the Environmental Protection Agency.  Tne
center is a consortium of three tiniversities:  Rice University* Oklahoma
University, aad Oklahoma State University*

     The three present co-directors .have worked together since the initation
of the center.  The center also has had the sana* project officer, there have
been only two chairmen of the Policy Board, and the members of the ^iginal
Science Advisory Committee have remitted in place.  Thus, the Center has had
constancy of management  personnel.

     The site visit team, Raymond Loehr, Keros CartwrighC, Richard Conway,
and Francis McMichael accompanied by Robert  ?laak  of  the EPA Science Advisory
Board,  reviewed  the Center  in Houston* Texas  oa April 24-25, law.   me
team met at Rice University  to  facilitate  travel and  minimise the tine
involved.  The relevant  center  activities  at  all three <»»«« s  *    at
the EPA Robert S.  Kstr Environmental Research Laboratory
Oklahoma were discussed.
      The individuals who met with the review team included:   (a) the
 center co-directors:   Dr. C. Herbert Ward,  Rice University;  Dr.  Mo man Durham,
 Oklahoma State University; and Dr. Larry Canter, Oklahoma University, (b) the
 project officer for the center, Mr. M.R. Scalf, RSKERL  (c)  the  chairman of
 the center policy board, Mr. Clint Hall, RSKERL, and 
-------
  GENERAL



  from the  sTt^vi^t^ ^^  Dbservatl*ns and conclusions that resulted
         There  continues  to  be  a  need  £or  a  cencer  devoted  to  Ground Water
         research  that  receive, EPA support.   This  need  is  p^tlcularly
         Important because of pressing RCRA  and  CERCLA issues  related to
         uncertainties  about mechanisms of transport, contaminant  trans-
         formations, and  approaches  for contaminant control.
                   *«'ctively, EPA should provide greater consistency of
        direction and constancy of support.  The centers need:  (a) /clear
        conflation of the intended role of the center, and (b) an increasing
        base of support that can be counted on.
            cur     level of EpA 8up      $420,000, is inadequate,  A mininun

        maintain    I °f abT 91'°00'000 P« y*« *hould be'provided to
        maintain a strong and relevant Ground Water research center.

      • The center should be able to acquire other EPA (non-base) funds to
        create a critical ma$s of project activities,


      * The center should be free to seek and acquire  non-SPA support for
        needed research  and  not  rely only on the  available  EPA support.

      • For the dollars  that are involved, excessive management,
        and  review now is  required  by EPA.
       such1sWif LrirKTV£qUlf S Eult"***PU«ary  talents and efforts,
       such as is available from the current consortium.
     The following indicates the que$tions used by the review team to arrive
at an understanding of the center.                                     arrive
                                          of the centsr?
        The strengths include:
           *-h.  **',.  4  *;--<*  d*Partmental  and  institutional barriers to have
           the  best  individuals  work on specific problems

        — development of  strong  researchers  now focusing on Ground Water
           research  rather than  on  other  issues
                                    D-14

-------
       — development of young faculty to be front-line researchers  in  the
          area of transformation and transport of contaminants in the soil
          and ground water

       — access to a real  field site  (CEROA. site) and joint collaboration
          of researchers and  projects  at the site

       — interaction with  1SKEEL avoids duplicating  on-going studies

       — developnent of well trained  students  (scientists and engineers)
          who have worked on  projects  and continue  related efforts  in sub-
          sequent positions

       — broad  information and knowledge exchange  through conferences  and
          workshops

       — courses related  to  Ground Water now  taught  at  the  three  universities
          that  did not  exist  before the  center was  established

       The  relative weaknesses include;

       — current  level of  funding

       —  lack of  clear direction from EPA

       —  geographical  separation

B.  Is it desirable  to have a  consortium of  universities  be a center?

      '  The  geographical separation, potentially different interests of
three universities and administrative Inefficiency and overhead can be
probleras  In having nore than one university be a center.   However,  none  o^
these items  appears  to be a real problem for this center.  The administration
of the center appears to be done well, the dollars spent  for administration
are low,  and the co-directors  appear to get along well-  In this case, a
consortium Is needed since no single university has the broad talent and
expertise that are needed.  In addition, a wider base of qualified researchers
Is available.

C.  Has the financial support received by the center been used satisfactorily?

        The center has attempted to use the available resources broadly and,
in previous years, has acquired supplemental funding  to broaden Its  research
efforts.   The center showed imagination In choosing to work with the state
of Texas on the  Conroe hazardous waste site and  to use the site as a field
laboratory.
                                      D-1S

-------
          The center directors have attempted  to  leveraee  the
  support in K appropriate manner.  However   the current  1™
  result, in . deadenlng constralat „„ ^aoL™/     center


  D'
                 n
  been  8rMter  than  that

  E.
                                   effortsjLdgntifieJLby the ct
                                                                            Ort^
                  use the SAC ranking to deter™ the projects  that
 »ere  ece                      full
  '   rs there  too close or too l~nse a relationship wjrh KSKHRL?

                          lnvolveme"t between the center and
J-1   _.
 "  ^J.^£Ster_the appropriate mechanj^jror^lgag-range  ground  water_£es

        Given the constraints on the EPA research efforts, a center  i*  an
appropriate n,«chanl.n, to assure long-range research related to Crouad
j.no current center has r~ ~    "
                                     D-16

-------
H.  	
    nature?
        In developing Its program, the center has moved In this direction.
So* of the lift wort Is fundlmental and clearly identified new,  important
phenomena about partitioning and  transport in Ground Water systems.  Thxs
and other research supported by the center is of a long-rang e na cure and
requires considerable time  and sound intellectual capability to br^ng  to
completion.   Such fundamental research is purposely and properly  balanced
with applied  research projects that also are of a long-range nature.
    Have  the  research efforts  supported b
    scientifically sound?
         The written research proposals that were provided and the discussions
 with the two principal Investigators indicate that the current projects are
 lound and are of high quality,   The internal peer review provided by the
 sic is Tf great assistance in assuring this for the future.   The ^scuss.on
 indicated that the research has become better as the center activit.es have
 matured.
         The center has held conferences in which the center research is
 presented and critiqued by knowledgeable individuals.  The results are
 published in pertinent, peer reviewed journals.  %y °^f S, uency of
 various teams helps insure that the research Is sound.  The frequency of
 such review should be reduced with the SAC reviews being the primary project
 review mechanism.
 CONCLUSIONS
      Based on the quality of  research  supported  by  this center  and  the
 successful development of a cadre of effective interdisciplinary  researchers,
 the  site  review  team feels that, to achieve  its  full  potential, the current
        should continue to be supported  by EPA and the  level  of support should
 be  Increased
                                       D-17

-------

-------