Report of the
Health Effects
Research Review
Group
U.S. Environmental Protection Agency
Science Advisory Board K
February 197i
Ul
O
-------
-------
EPA NOTICE
This report has been written as a part of the activities of
the Agency's Science Advisory Board, a public advisory group
providing extramural scientific information to the Admin--
istrator and other officials of the Environmental Protection
Agency. The Board is structured to provide a balanced expert
assessment of scientific matters related to problems facing the
Agency. This report has not been reviewed for approval by the
Agency, and hence its contents do not necessarily represent the
views and policies qf the Environmental Protection Agency.
-------
-------
TABLE OF CONTENTS
Page
I. Introduction, 4
II. Summary and Recommendations.... 6
A. Summary. 6
B. Recommendations,... — 8
III. Committee Membership, Approaches, and Procedures 14
A. Committee Membership .....»,,.........,,,. 14
B. Approach to the Assessment of RID and Procedures
Utilized , 14
IV. Research in a Regulatory Agency: The Conflict Defined.... 17
A. Present and Future Agency Needs for Data. 17
B. Investigatory Time Frames...,,,........,..,. 18
C, Investigator and Program Staff Interactions 18
D. Evaluating thet Responsiveness of ORD.,... 19
E. What is, an Investigatory Product in a Regulatory
Agency... 21
V. Observations of Current EPA Research and Development..... 22
A. Identification of Research Needs. 22
B. Planning the Research Program 24
1. Budget Formulation. 24
2. Research Program Formulation.,,,,.... 25
3, Pre-project Evaluation of Productivity and
Costs. 26
4. Good and Poor PIanning. 26
C. Performance of Research...... — 28
1. Adequacy of Facilities for Research.. *.,... 28
2. Staffing for Research.... 29
3, Accountability for Expenditures.................. 30
D. The Quality of Health Effects Research.... 31
1, Publication and Reporting of Research Results.... 32
2. Quality Assurance in Grants and Contracts,....... 33
3. Career Opportunities. .*.,....,. 36
4. Other Components of Quality Assara'nce............ 36
5, Interagency Agreements... 37
E. Other Relevant Topics. 38
1, Long-range, or Core, Research. 38
2. ORD/Congressional Staff Information Transfer..... 38
VI. Utilization of ORD Results , 39
VII. Status of Implementation of Two Sets of National Academy
of Sciences(NAS) Recommendations to EPA...,, 40
A, Recommendations from the Environmental Research
Assessment Committee of 1975. 40
B. Recommendations of the Review Committee on Management
of EPA's Research and Development Activities,....... 44
-------
page
VIII. Community Health and Environmental Surveillance System
(CHESS): An Investigative Report...... 55
A. Background of the CHESS Program..,,..... 55
B. Findings of the Subgroup. 56
C. Steps Taken by EPA to Meet Brown Committee
Recommendations. * 57
APPENDICES
A. Charge to Committee and Authorization for Charge
in Public Law 95-155................... A-l
B. Committee Membership and Consultants.... A-9
C, Chronology of Committee Visits to Facilities and
Meetings. ....,,....,,.....,.....,,...........,. A-l1
D. List of Principals Interviewed and/or Supplying
Information....,,.....,. A-l3
E. Agenda of Major Regulations Being Considered by EPA,. A-29
-------
I. INTRODUCTION
The Congress required an evaluation of the health effects
research efforts of the U.S. Environmental Protection Agency in
section 8(d) of Public Law 95-155, enacted November 8, 1977.*
Subsequent to the passage of the Act, EPA's Science
Advisory Board formed a special committee to perform the
mandated evaluation. This Committee, named the Health Effects
Research Review Group (HERRG) and composed of experienced
scientists and research managers, began their task in May 1978,
The Act stated that the evaluation include the following;
1) The health effects research authorized by this
Act and other laws;
2) The procedures generally used in the conduct of
such research;
3) The internal and external reporting of the results
of such research;
4) The review procedures for such research and
results;
5) The procedures by which such results are used in
internal and external recommendations on policy,
regulations, and legislation; and
6) The findings and recommendations of the report to
the House Committee on Science and Technology
entitled "The Environmental Protection Agency's
Research Program with Primary Emphasis on the
Community Health and Environmental Surveillance
System (CHESS): An Investigative Report."
The Act further stated that
"the review shall focus special attention on the
procedural safegards required to preserve the scien-
tific integrity of such research and to insure
reporting and use of the results of such research
in subsequent recommendations. The report shall
include specific recommendations on the results of the
review to ensure scientific integrity throughout the
Agency's health effects research, review, reporting,
and recommendation process."
The word "research" takes on a broad meaning in a regula-
tory agency. For the purpose of this evaluation, health
effects research will be defined as requested by Mr. Costle in
his letter of June 17, 1978, to the Chairman of the Science
Advisory Board. A quotation from that letter follows.
*Section 8(d) of this Act requires that a special evaluation
of EPA's health effects research be prepared by the Science
Advisory Board :{SAB) and the report be submitted to the
Administrator, the President and the Congress.
-------
"To delineate the Congress1 charge more sharply, I
urge the Study Group to define health effects research
to include all planned activities, collection and
analyses of data done within the Agency for the purpose
of adding to the scientific basis for understanding
the effects of environmental factors on human health.
This definition would include those activities within
the Agency which may be used to assess human risk, and
which support standard setting and regulatory deci-
sion and any activity which gathers new knowledge
about human health, or improves our understanding of
human health either directly or which can be used to
extrapolate to human health impacts."
In view of the limited time available to the Committee,
this study focused on the collection and analysis of data
primarily to add n_ew knowledge. The analysis of existing
information and d'ata, which already satisfies generally
acceptable criteria for scientific adequacy, was not considered
to be within the scope of the charge to the Committee. Some
requested data were unavailable or not provided to the
Committee, therefore,the evaluation is not as complete as
initially anticipated or desired.
-------
II. SUMMARY AND RECOMMENDATIONS
A. Summary
The purpose of this report is to summarize the nature of
health effects research in a regulatory agency, to describe the
current status of that function in EPA, and to present conclu-
sions and recommendations. Supporting data and reports
relating to individual ORD facilities are available but are not
included.
The Committee visited (either as a full or partial
committee) all EPA laboratories performing health effects
research. Interviews were conducted with senior laboratory
staff, managers, and bench scientists as well as with senior
managers in the Office of Research and Development (ORD) and in
the Program Offices. For the purposes of this report, a
"Program Office" refers mainly to the Offices of Water and
Waste Management; Air, Noise, and Radiation; and Toxic
Substances, as these are the offices responsible for developing
regulations and setting standards or tolerances in response to
specific legislative acts. A list of the facilities visited,
Committee members visiting each facility, and those EPA
employees interviewed or providing information can be found in
Appendices C and D.
The Committee also utilized the services of SAB members,
other scientists, and research managers on an ad hoc basis
(Appendix B).
Programs and facilities were evaluated using a number of.
criteria relating to the objectives of the research and the
quality of facilities, staff and results. Among these criteria
were responsiveness of the research function, research
influence in the decision making process, coherence of planning
and goal-setting between ORD and the Program Offices, and
quality assurance through peer review and publications.
The Committee interviewed many competent and dedicated
people with a real desire to work in a more effective,
efficient and involved way.
Research and development in a regulatory agency is a
complex task, one requiring research targeted to regulatory
requirements usually having short (six month to two year) time
frames. Research and development must be related to specific
regulatory needs. Identification of gaps in data and needed
research effort necessitates cooperative planning between
-------
program managers*, often unfamiliar with research, and research
managers, who are often insensitive to regulatory pressures and
requirements. Researchers, as professionals, may have
difficulty in identifying results which will sitisfy regulatory
needs when these results are not in their scientific
specialties. Constantly altering budgetary allocations to adapt
to rapidly changing regulatory needs aggravates research-
program staff relations. For these and other reasons, ORD has
frequently been viewed as unresponsive by many program
managers, who do not, in general, depend upon ORD to support
their regulatory efforts. The Committee concluded that it
would require far greater joint planning and coordination of
ORD and Program Office staffs if ORD outputs, useful to
regulation, were to be commensurate with the funds allocated,
At present, it is not an effective or an efficient system. The
dilemma of research in a regulatory agency is further treated
i n Chapter IV.
The most successful and useful research programs were
found where there was a close working relationship and
understanding between scientists in the laboratories and their
counterparts in the Program Offices, Such communications are
essential to an understanding of priorities, quality demands,
timing and whit was truly needed to back up the regulatory
process in the short and long terms. Poor results were seen
all too often, however, because close relationships did not
exist.
Pilot research committees have helped to establish
essential communications between those who have direct and
indirect responsibilities. Where successful, the resulting
agreements, e.g., Drinking Water and Pesticides, have helped to
make research more responsive and have cut across juris-
dictional barriers to establish objectives, goals and plans.
The pilot research committees are one means to an end, but
shorter and more direct communications lines are needed between
data generators and data users.
Beyond a committee approach, there seemed to be little
consideration of organizational structures designed to
streamline decision making. Hopelessness was expressed many
times by those concerned when faced with the seemingly obdurate
character of the civil service system and the highly placed,
inflexible, and sometimes less thin adequate individuals who
occupy unessential positions. Inflexibility makes it difficult,
indeed, to place people properly and to transfer or get rid of
people not performing up to expectations in their jobs.
*A program manager is defined as that person in the
Program Office who is responsible for developing the regulatory
or standard-setting activity for a specific program as mandated
by legislation. A research manager is that person in ORD who
is responsible for formulating, planning, and executing specific
research programs.
7
-------
Recent changes in the civil service laws were not seen is
adequate to effect much improvement. Desirable changes can
occur, but they will require enormous effort, training in, and
application of the principles of management by objective and
job performance evaluation to establish a clear understanding
of whit is ex'peeted of each employee*
B. Recommendations
The Committee recommends that:
(1) ORD and Program Office
leadership take immediate steps to
coordinate all research planning and
activities in the Agency. Joint
planning to identify information needs
must begin as soon as a decision is
reached to prepare a regulatory
proposal.
Immediately following a program decision to develop a
regulatory proposal > Program Office and ORD staff should be
assigned to review existing information needs. This group
should be given authority to organize Program Office-ORD staff
to identify regulatory needs for specific proposals and outline
the required research to fill the gaps.
(2) ORD continue to use
appropriate research committees, but
they should not be QRD's exclusive
planning mechanism.
Research committees, initiated on a pilot scale in 1978 to
help ORD plan and coordinate its research activities with the
Program Offices, should be used sparingly. These research
committees, really task forces,,will be most useful when
research needs relate to multipfe Program Offices and
laboratories.
The research committees should be used for Identification
and prioritization of needs. These committees should not be
involved with research implementation.
Key managers within ORD should devise mechanisms to
develop well understood objectives, goals, plans and measures of
performance for how research should be conducted.
The Committee does not believe that it will be possible for
ORD to fulfill its function without extensive agreement by key
personnel on objectives, goals, plans, and measures of
performance. It might be helpful for ORD to hire experienced
management specialists, as consultants, to help address some of
the difficult managerial problems which currently exist.
8
-------
The Committee feels that too many specific directions
regarding research implementation come from headquarters. This
prevents the scientists from using their talents and diminishes
the scientific climate for innovative research.
(6) After agreement on
responsibilities for research
implementation, laboratory
directors and their scientific
staff be permitted to performed
their assigned tasks. (See
recommendation 5.)
Laboratory staff need protection against unwarranted
mandates, incursions into allotted -time for research, and
reorganizations and spurious changes in policies that occur with
the all-too-frequent changes in leadership. The scientists also
need a sense of the Agency's long range commitment to its stated
goals.
(7) An expansion of the
Interagency Regulatory Group (IRLG)
activities be carried out. The
excellent planning initiatives of
IRLG should be extended to include
environmental health research.
The IRLS is seen as an excellent beginning with the
potential of reducing duplication and confusion among agencies.
This effort should be extended to strengthen coordi nation" of
research planning by all agencies conducting environmental
health research.
(8) A simple, easily under-
stood accounting system be
established for planning, assigning
and monitoring use of funds and
personnel relative to ORD's
intramural and extramural programs.
Effective use of limited funds and personnel requires that
they be carefully managed. The accounting systems now in use
are inadequate. At the present time, analyses are not performed ,
to place in perspective salaries, equipment costs, services,
etc. Those cost breakdowns are necessary to give ORD
information about responsive and nonresponsive work at the
different laboratories performing health effects research.
(9) Standard procedures for
awarding contracts, grants, and
cooperative agreements, and
monitoring extramural research be
simplified and enforced.
10
-------
Current elaborate rules for contract and grant awards
should be reviewed and revised to promote efficiency and
timeliness of extramural awards. All personnel must adhere to
these new procedures* This would end the current abuses of the
extramural award system. Procedures should be adopted to ensure
adherence to the new requirements after revisions are made.
The monitoring procedures should indicate methods for
evaluating the performance of contractors and grantees during
and after completion of their work. Furthermore, the extramural
research results should be published in peer reviewed scientific
journals. EPA-published reports are no substitute for open
literature publications.
Adequate travel funds should be allocated for proper site
visits and for monitoring of extramural work. Presently, there
is no routine, operational audit of the quality of extramural
research.
Responsibility for extramural research (planning, awards,
and monitoring) should be made according to the staff's
capabilities to effectively plan and monitor such research. This
should take into account the amount of independent in-house
research expected from the staff scientists. Extramural
monitoring assignments should only be made to scientists who
have demonstrated professional competence and are thoroughly
familiar with how research is conducted in the field being
moni tored.
(10) Scientific peer review of
proposals, programs, and intramural and
extramural research be greatly
i ntens ified.
Scientific credibility and defensibi1ity of research done
in support of regulations are key elements of the success and
acceptance of the Agency's role by the public. The Committee
feels that,, to the maximum extent practical, scientific peer
review mechanisms should be utilized to improve the quality of
final research results.
All programs and organizational units should be
periodically subjected to peer review by qualified scientists
from outside the Agency. All proposals and completed research
should be reviewed by peer scientists within the Agency, and
representative items should be reviewed by scientists outside
the Agency.
The quality of research in EPA is important not only
because any research should meet standards acceptable to the
scientific community but also for reasons derived from the
regulatory nature of the Agency.
11
-------
To ensure acceptability of research results, the studies
must be reviewed by one's scientific peers and published in
reputable scientific journals. Failure to so treat results of
research investigations, involves the risk that review will occur
at a later date, in a adversary situation, with possible
refutation of results and embarrassment to the Agency.
(11) A dual-ladder promotional
system be implemented for qualified
scientists to advance in grade and
salary without having to undertake
• supervisory or managerial
responsibilities.
Presently EPA has a promotion ladder inadequate to allow
scientists to remain in the laboratory and be promoted strictly
on the basis of their scientific excellence. EPA suffers from a
poor reputation as far as the scientific quality of its health
effects research is concerned. This reputation is not totally
deserved. There does need to be a system whereby both qualified
scientists and qualified managers can each advance and be
rewarded in their own fields.
Well qualified personnel are the key ingredient to the
conduct of a scientifically sound research program. At the
present time, there are both formal and informal procedures that
encourage scientists seeking promotions to accept supervisory
and administrative responsibilities, thereby reducing the amount
of time they have to spend on laboratory research.
When personnel are assigned to senior management positions,
primary consideration should be given to individuals who have
demonstrated scientific and managerial capabilities; an
understanding of how research is planned, conducted and
reported; and the ability to communicate research information
and needs to both scientists and non-scientists.
(12) Research management give
immediate attention to instituting,
in the laboratories, a variety of
procedures to create an atmosphere
conducive to scientific excellence.
Even though the laboratories are located on or near
university campuses or other research institutions, EPA
scientists were somewhat outside the mainstream of scientific
events. The Committee, therefore, urges management to regularly
schedule seminars in which both outside scientists and Agency
scientists participate, invite outside scientists to spend time
in EPA laboratories (in addition to use of the Interagency
Personnel Agreement—IPAs), encourage EPA scientists to spend
time in outside laboratories (an exchange program), sponsor
workshops and symposia, and generally institute a closer
interaction with geographically close institutions.
12
-------
(13) ORD and senior Program Office staff
rotate assignments, preferably on the basis of
those ORD and Progaiti organizational units
which consistently interact.
It is essential for effective performance that Program
Office and ORD managers understand the problems and capabilities
in each organization. Program managers are often unfamiliar with
research planning, laboratory work and the inherent time
constraints. Likewise, research managers are often unaware and
insensitive to regulatory pressures and requirements and with
the dilemma of how to present data in a form useful to the
Programs.
{14} The research program
using the clinical inhalation
exposure facility at Chapel Hill,
North Carolina, be fully staffed and
a sound research program implemented
as soon as possible»
The clinical inhalation facility at Chapel Hill is a unique
facility, engineered to deliver the desired exposure levels;
however, the scientific program, staffing, and plans to utilize
the facility are totally inadequate--a very conspicuous waste,
as it now stands,
ORD should immediately assess the future need for and use
of this facility, establish goals and support for the facility,
and assure that the facility is not wasted~-even if EPA has to
make it available to outside groups. This facility was designed
for long range studies to accurately assess and predict the
potential adverse effects of selected environmental chemical
agents.
The inhalation program, once developed, should be
scientifically peer reviewed and approved.
13
-------
III. COMMITTEE MEMBERSHIP., APPROACHES AND PROCEDURES
A. Committee Membership
The Health Effects Research Review Group (HERR6) consisted
of core members and consultants selected for their scientific
expertise and research management skills. The consultants
supplemented core members and were used to provide specific
expertise for the evaluation of individual laboratory programs
or special topics of research. A list of Committee members and
consultants is Appendix B.
B. Approach to the Assessment of R&D and Procedures
Used "__^_.
It was apparent from the outset that the Committee needed a
clear understanding of the mission of health effects research as
seen from the viewpoints of the personnel in both the various
Program Offices and ORD. Responsiveness of the research function
to the pressing, .(often mandated) needs of the Program Offices
has been inadequate in the past; this problem has been clearly
described in a report by a committee of the National Academy of
Sciences, Analytical Studies of the U.S. Environmental
ProtejCtjon Agency, Volume III: "Research and Development in the
EnVTFonmental Protection Agency," 1977,
Of necessity, the Committee had to subdivide much of its
investigation into small study group activities. A common
approach was taken to make it easier to analyze and assemble'the
findings of the various study groups into an integrated final
report. Thus, the research function of the Agency was to be
analysed in'the context of the regulatory_ responsibilities of
the Agency, which in turn requires a reliable and d¥fen"sible
data Base for decision making. The Committee agreed that
research can only be understood if the reciprocal relationship
between the users of the information (the Program Offices) and
the generators of the information (ORD) was examined. The
perceptions of both,the generators and the users were,
therefore, to be probed to determine if thene were s_hare_d goals
and a shared understanding of what is known, what is unknown,
and what needs to be known. It was also necessary to determine
whether there was a shared understanding of the time frame
necessary to generate or assemble the needed data. These
perceptions were to be examined at several .hierarchical levels
to determine if the intentions of the supervisors were accepted
in a way that motivated the respective organizational units
regardless of location or attitudinal preferences.
While conducting .interviews and fact-finding sessions,
Committee members tried to use some of the following checkpoints
as they were appropriate for the various situa-tions. These
points were the basis for the formulation of this report.
14
-------
a. Checkpoints relating to the mission of health-
related research as it supports short-term and long-term
Agency needs:
I, Responsiveness of the research function (as
defined at the outset)
2. Sense of urgency and commitment of the
research function
3. Research influence on judgments made on the
decision making process (level of influence
and dependence by the program offices)
4. Coherence of planning and goal setting be-
tween the Program Offices and ORD (Are
budgets really reconciled and supported
by both the Program Offices and ORD?)
5. Examples of good and poor responses by ORD
6. How and by whom is the decision made to
initiate and conduct specific research
investigations?
7, How are information gaps identified? How
are 1-ong-term trends with potential
environmental impacts identified? How are
long-term research needs defined and planned
to assure budgetary support?
8. Beyond the Program Offices and the ORD
functional organizations, what other factors
help influence what research is to be done?
b. Checkpoints relating to the quaj ity of health
effects research as it supports short-term and long-term
Agency needs;
1* Quality assurance:
a) Good laboratory practices
b) How is quality assurance implemented to
improve the defensibi1ity of results?
c) Evidence of attention to detail and
carefulness (facilities, work flow,
housekeeping, attitude, safety .program)
d) Persona-1 scientific integrity, '
including quality of planning and
experimental design, rigor of analysis,
courage to disprove one's hypotheses
(or hypotheses of a superior), and
acceptance of opinions of qualified
peers
e) Can the most qualified people be
quickly identified?
f) Is the civil service system seen as a
positive factor in the encouragement of
a good research program within EPA?
15
-------
2. Publication of results (reporting)
a) In journals requiring scientific peer
review, internal government
publications, journals or meetings not
requiring scientific peer review
b) Methods for approving manuscripts
before release or publication
c) Is publication seen as helpful to
career development?
With these checkpoints in mind, the Committee conducted
its assessment through a series of fact-finding sessions and
public meetings in Washington and in various EPA laboratories
(see Appendix C). The Committee chairman and co-chairman first
discussed the charge and the plans for accomplishing the
evaluation with the appropriate Congressmen and their staffs.
Subsequently, the Committee met with the Administrator, the
Assistant Administrators and other senior EPA policy and
management staff in various Program Offices, and with
representatives from the regions, laboratory directors, senior
science managers, and individual laboratory scientists
.{Appendix 0). The Committee members reviewed legislative
mandates, various EPA documents, and other papers and memoranda
relating to the Committee's charge.
16
-------
IV. RESEARCH IN A REGULATORY AGENCY: THE CONFLICT DEFINED
A, Present and Future Agency Need_s_ f or Data
Volumes have been written on regulatory agency
research needs in general and on EPA research needs in
particulir. Therefore, the Committee approached the subject of
the research and development needs of EPA with trepidation and
elected initially to describe the pressures and constraints
imposed generally upon a research and development group in a
regulatory agency and those imposed upon EPA in particular.
Program administrators in regulatory agencies are captives
of the calendar deadlines imposed for regulation by the specific
statutes they enforce. These agencies routinely deal with
Congress, irate constituents, citizen groups, the media, and
others. The professional skills which contribute to their
success and/or survival are all devoted to integrating immediate
pressures and existing knowledge into a set of regulations
acceptable to all. This is a difficult situation, one requiring
sensitivity to human behavior and appreciation for the relevant
available data base. Regulations are usually compromises, their
political socio-economic impact and whether they can be
enforced. The scientific and technical bases for a regulation
will be put to rigorous test if, and only if, the regulation is
challenged. Judicial review will incorporate and consider all
relevant data; an administrative "gamble" made in the absence of
sufficient data to support regulation will very likely lead to
remanding the rule to the Agency, Development, promulgation and
enforcement of regulations, particularly in an area as
underdeveloped and evolutionary as environment, is a difficult
exercise.
The formal challenges to regulation are cyclical. Because
of inflationary pressures on regulatees since 1976, there has
been an increasing trend toward challenging environmental
regulatory promulgations. The courts have been sympathetic to
the innovative promulgations of EPA, but the economic impacts of
EPA administrative interpretations of enabling statutes have led
to regulatee demands for more complete substantiating data for '
promulgated rules; those demands will increase in the future.
Even those sympathetic to prudent Federal environmental
regulations are demanding higher standards of proof during this
highly inflationary period of increasingly demanding and varied
Federal regulation. Because environmental rules are still
perceived by many as a luxury affordable only by a prosperous
private sector, EPA must anticipate continuous, more
sophisticated private sector challenges .because of inflationary
pressures.
17
-------
These challenges will be overcome only by convincing
arguments for regulation, arguments drawing upon defensible
data. These data will have to relate specifically to
improvements in human health if EPA is to fulfill its mandate as
an Agency. In the future EPA will increasingly have to document
health gains anticipated from allocation and expenditure of
large sums of money for regulation and control of environmental
pollution.
B. Investigatory. Time Frames
Specific statutes include timetables for regulation
assigned by Congress. The Agency has formulated a table of
regulations scheduled or In progress (Appendix E). Program
administrators will formulate these regulations with whatever
data are available prior to and until the scheduled completion
date. In general, schedules for EPA to write regulations are
short; 6-12 months 1s normal, while 18 months is considered
long. These are short time frames for generation of new
information in the laboratory or in the field. EPA Research and
Development Office (QRD) personnel have had enormous difficulty
responding within the time allotted. It is essential that ORD
and Program Office personnel carefully evaluate information
needs critical to implementation of scheduled regulations. This
must be done- as soon as a statute is assigned to EPA for
enforcement. In this way, ORD will be able to utilize the
maximum available time to generate needed data for regulation.
We did not perceive that research needs are routinely approached
in this manner.
C. Investigator and_Progratn Staff Interactions
The perceived needs of program managers are usually
very specific and often conflict with needs perceived by
researchers. For example, researchers may regard experiments
requiring toxicity data from animal exposure to pollutant agents
at concentrations far in excess of those likely to occur under
normal exposure as of little relevance to scientific
understanding. Program personnel, however, may regard
demonstrated toxi.city data, even at unrealistically high
exposure levels, as a rationale for regulation. Sorting out
these differing perceptions requires personal interchange if ORD
is to respond in a timely and meaningful manner. Too often in
the past the Program Offices have perceived ORD as unresponsive
because results were of a kind different from what had been
anticipated and because research time frames were too long to
allow the Program Offices to use the data produced. Under these
circumstances, program administrators did not look to ORD for
solutions to their problems.
18
-------
Principal Program Office and ORD administrators are located
in Washington, D.C. ORD investigators are located in laboratory
facilities throughout the nation. Specific administrative
mechanisms are required to ensure that communications occur
between Program Office administrators and ORD investigators as
research in support of specific regulations progresses. In 1978
five research committees were initiated on a pilot basis to help
ORD plan and coordinate its research activities and become more
responsive to the needs of designated Program Offices. These
pilot research committees have helped to provide an essential
communication function; furthermore, they have helped to
establish understanding and commitment to objectives, goals, and
plans. Carefully selected research committees are seen as a
means to an end, although a cumbersome one, because their
meetings help to educate those who need to know. In the long
run, however, the functions served by the pilot research
committees need to be institutionalized so that laboratory
directors are not excluded from key roles in leadership or from
maintaining a high level of competence in their respective
laboratories.
Program administrators frequently have their primary
training, in the legal or engineering professions; they are often
not familiar with the state-of-the-art of ORD scientific
research* ORD utilizes scientifically trained personnel at all
levels of the organization, those working at science on a daily
basis* One can draw flow diagrams of the decision making
processes in a regulatory agency, diagrams illustrating ORD and
Program Office personnel interactions. However, in the final
analysis, exchange of information and resolution of issues is
required of persons with essentially different bases of
understanding. There will be a major built-in obstacle to
communications between ORD and Programs Offices as long as ORD
relies entirely on scientific managers and the Program Offices
on managers who pride themselves on their pragmatic approach,
managers grounded in law and/or engineering sciences. By one
mechanism or another (rotation of assignments, creation of new
positions for complementary professionals in each Program Office
and ORD), there must be promotion of ORD-Program Office
communication by ensuring that senior managers have a common
language(s).
D, Evaluating the Responsiveness of ORD
The responsiveness of ORD is judged by a variety of groups
and individuals» including EPA program managers, Congress,
citizen groups, and the media, to name a few. The Committee
probed primarily EPA program managers' perceptions of ORD's
responsiveness to their needs. Senior program managers have
indicated that there have been recent improvements, but much
19
-------
remains to be done. In the past, many Program Offices did not
participate in ORD planning. Recent joint ORD-Program Office
research planning exercises, such as the pilot research
committees, have caused Program Offices to be more favorably
disposed toward ORD activities.
Ultimately, ORD's response to the Program Offices will be
more stringently judged by how effectively the research results
meet the specific needs of the regulators in a timely and
scientifically rigorous fashion. The current auspicious climate
for ORD pilot research committee planning must not be confused
with future ORD outputs necessary to satisfy hard-pressed Agency
program managers. For this reason, the major ingredients of ORD
research that would allow ORD to be considered, "responsive" to
regulatory .program needs will be briefly discussed. Following
this discussion will be comments on the current EPA research
process from the planning stages to the final utilization of
results by Agency Program Office staffs.
The timing of the delivery of research results to a Program
Office is a major factor contributing to the perception of ORD's
responsiveness to Agency needs. Regardless of the quality of
research results, they are viewed as only marginally useful if
available after statutory deadlines have passed. One can argue
that in the long run "late" results will be integrated into
environmental programs, but this does not engender Program Office
staff confidence in or support for ORD,
The icigntifjc and technical soundness of ORD results is
crucial if"~EPA Program 'Of f fee's ar¥ ~t~6~ sustain their regulatory
positions. Transfer of weak results by ORD will lead either to
rejection of these results by administrators or to utilization
with subsequent public embarrassment upon disclosure of a weakly
supported position and/or reversal of the Agency position by the
Courts.
In addition to being scientifically defensible, research
results mus_t_be targeted to meg,t Program Office needs* Needs
must be commonly perceived and agreed upon "by researchers and
program administrators. Dictation of needs by regulatory staff
to researchers can result in untimely and fruitless
investigations; likewise, researchers with inadequate
understanding of program needs may pursue scientifically sound
studies which are irrelevant to the Programs.
The understanding of ORD results by potential users is
probably a major ingredient of the perception of responsiveness.
ORD must not only deliver sound results in a timely manner, but
must also translate these results into terms and concepts
understandable to the users, i.e., the Program Offices. ORD has
a responsibility to assist its users in understanding the
strengths, weaknesses and full significance of those research
results transmitted for Agency use.
20
-------
The above ingredients of "responsiveness" relate to the
research function as it serves regulatory needs. Each ingredient
must be carefully developed and nurtured, literally on a project
basis, if expectations of ORD efforts are to be fulfilled.
With this brief introduction to the demands placed upon
ORDf specific aspects of performance of health effects research
and development in the Agency will now be discussed.
E. What is j>n Investigatory Product in a
R e g u 1 a t o ry Agency?
The investigatory product in a regulatory agency is that
body of scientific information and data base which is either
available to or resides with the scientific staff. The product
must be provided to the Program Office in a form that is useful,
understandable, and defensible in setting reasonable standards
and for writing regulations.
This scientific information can be provided to the Program
Offices in many ways. The best way would undoubtedly be to have
the research described and published in professionally peer
reviewed journals§ but information can also be provided through
monographs, letters and verbal presentations. The key to the
desired investigatory product is for the Agency to have an in-
house core of capable scientists who understand the regulatory
and standard setting requirements, who can perform the necessary
literature searches, can perform their own research and evaluation,
and can freely attend professional scientific meetings where
discussions and information exchanges occur.
21
-------
V. OBSERVATIONS.OF CURRENT EPA RESEARCH AND DEVELOPMENT
A. Identification of Research Needs
ORD can be viewed as a large multifunction apparatus
capable of responding in a variety of modes if appropriate
planning of the necessary dynamics and a complete "tune-up"
occur prior to "start-up." The initial step is to identify the
required outputs. ORD outputs should be responsive to regulatory
needs, in the short or long term* At present and, indeed, during
the entire history of EPA, short term R&D needs have been
stressed. We do not see any conflict between simultaneously
sustaining research programs with long (years) and short term
(months to years) goals, prov_1_d_ed__Program. Office-ORD concurrence
Is reached ;.as' to these. goaTsT " "
Historically, Program Offices outlined needs according to
their perceptions of the problem. It was a hierarchical
planning process which gave the scientists at the laboratory
little understanding of what was needed or why. Laboratory
scientists often communicated with lower level Program Office
staff who did not fully understand the needs and priorities of
.their program.
There seems to be no systematic identification of
information gaps (research needs) in the Agency. This
identification should take place as soon as EPA receives
legislation on which it must act; it requires close cooperation
between the appropriate Program Office and ORD scientists,
especially those in the laboratories. These staff members
should carefully analyze the Act to assess what the Agency must
do to gather the needed information and to fulfill the
requirements of the Act, Additional research needs come from the
process of drafting regulations and from writing the criteria
documents when perceived needs for information are recognized.
Better identification of needs takes place when there is a close
association between ORD and the Program Office, but this must be
directed throughout the Agency in a systematic way.
Long-term (anticipatory) research in subject areas central
to Agency responsibilities should be planned as a natural
extension of the identification of gaps in the data base. It
cannot be designed in a vacuum, as an activity to be initiated
or terminated at will. When effective cooperation occurs
between ORD laboratory and Program Office personnel and when
effort is expended to define common objectives, goals, and
plans, opportunities are likely to arise for defining relevant,
long-term research programs.
The perception of needs for longer term research arises
from the interaction of key regulatory people and creative
researchers who are in touch with the issues and the scientific
literature. People who do research, read scientific literature,
22
-------
attend meetings and work cooperatively with the Program Offices
are those with the best resources to define needs. The
Committee believes that the stress on identifying long-term
research needs must come from ORD and that more attention must
be devoted to identifying these needs and pursuing the
associated research studies.
The pilot research committees have helped to identify gaps
deserving further research effort, to date only short term; but
even this has helped to gain better insight into Agency
priorities. Because of the large number of people involved,
these pilot research committees are cumbersome, but they have
forced a meeting of minds among key people in the Program
Offices and ORD. In fact, the identification of research needs
by individuals with diverse backgrounds and responsibilities is
a very strong feature of the pilot research committee effort and
should be retained regardless of the ultimate fate of the
activities of these committees. This should be expanded to
include identification of long term needs.
Several efforts at identifying research gaps and
implementing research should be highlighted. The Drinking Water
Program has been an example of effective cooperation in
identifying and implementing research needs, whereas the Human
Inhalation Exposure program at HERL, RTP (Chapel Hill) and the
Animal Exposure Program at HERL, Cincinnati are examples of very
poor coordination* In the area of pollutant inhalation studies
on human subjects, the scientists of the Chapel Hill facility
have attempted to implement longer range studies to predict and
assess more accurately the potential adverse health effects of
selected chemical agents. In general, ORD administrators tiave
been sympathetic to funding short-term inhalation projects, but
have not been supportive of longer term inhalation research
programs. The Inhalation Toxicology (animal model) Program at
HERL, RTP, on the other hand, was enthusiastic about its
relationship with the Program Office. This group is well
supported, largely as a result of a sustained effort by the
section leader to keep close contact with ORD and Program Office
personnel in Washington. Development of new methodologies was
considered to be a major responsibility of the group working on
animal inhalation toxicology; they expressed the desire to be
involved in toxic substances support as well. This group also
supervised contracts and grants. Management of both grants and
contracts in addition to the "in-house" responsibility was seen
as a desirable component of the total job done by the Inhalation
Toxicology Section, A key element of this program seemed to be
the desire on the parts of the Program Office and the laboratory
to engage in cooperative planning and goal setting. The result
is a very spirited and productive group of researchers.
23
-------
Scientists in the Diesel Exhaust Program at Center Hill
(Cincinnati) clearly foresaw the emerging importance of diesel
engines and attempted to start long-range research several years
ago. These projects were turned down by ORD staff members in
Washington, who have recently recognized the need for such
studies. Work is now frantically underway to obtain needed
results to meet the statutory deadline for establishment of
diesel emissions criteria,
B, PIann 1.n:g Research Projects
1. Budget Formulation
During the period of our Committee review, the Agency was
in the second year of zero based budgeting {ZBB}§ i.e., fiscal
years 1979 and 1980 budgets were in progress. Funds are
authorized and appropriated directly to ORD in categories
related to enabling legislation or special projects.
Prior to the introduction of the ZBB process, senior ORD
personnel often established project allocations without
communicating with Program Office managers. The zero based
budgeting process has been an exasperating (but probably
desirable) experience for all concerned—Program Offices^ ORD,
and laboratories alike. It has forced a certain amount of
communication and has led to some good, though tortured,
outcomes, especially in the pilot research committees. However,
communications are still occurring only between ORD and Program
Office personnel of relative seniority. We perceive that many
bench scientists in ORD do not understand the relationship of
their work to overall ORD and Agency goals. If communication
involved the laboratory investigators doing the work, even more
effective decisions could be reached, while simultaneously
gaining the commitment of the researchers to the work.
An additional budgeting problem is the mismatching of
personnel ceilings and funding for specific programs and
laboratories. Numerous examples were found in which program
areas in specific laboratories had very few or no people
assigned and relatively large amounts of funds available. In a
few instances, relatively large numbers of personnel were
assigned with limited funds available. At the headquarters
level, the view was frequently expressed that OMB had minimized
management's latitude for shifting personnel between programs to
better match program needs and fund allocations. Laboratory
personnel expressed a feeling of hopelessness in dealing with
the problem and were, on occasion, forced into the unrealistic
posture of showing, for the record, programs with substantial
funding managed with zero personnel; obviously this does not
happen. The people who are assigned to manage the program
simply charge their time to some other program that has a more
adequate manpower ceiling. The result is manpower accounting by
progam that is suspect, at best, and probably of limited value.
24
-------
Clearly, if laboratory directors are to be effective research
managers, they must be given the latitude to utilize assigned
personnel without rigid program area constraints, A change in
approach should allow laboratory directors to place increased
emphasis on developing the appropriate mix of disciplinary
skills of their staffs to better serve current and future
program needs.
Allocation of travel funds is another budget problem. When
travel funds are allocated to the laboratories, consideration
should be given not only to the number of scientists in the
laboratories, the degree of participation in extra laboratory
Washington mandated activities^ and the required extramural
program monitoring required, but also to the geographic location
of the laboratories with respect to these activities and to the
location of national scientific meetings. Furthermore^
increased flexibility should be given to the laboratory
directors for control and utilization of travel funds. For
example, the laboratory director at the ERL in Duluth should be
authorized to approve travel for his staff to go to Canada. One
of the major functions of this laboratory is scientific
cooperation with their counterparts in Canada. Yet this
collaboration is minimal because travel to the Canadian
laboratory in Thunder Bay is considered foreign travel and must
be approved each time, well in advance, by ORD headquarters in
Wash!ngton,
2. Research Program Formulation
The Committee senses that the major contribution of the
pilot research committees in program formulation has been to
overcome previous inadequacies in planning and to initiate
discussions of research by the many individuals with an Interest
in the outcome and utilization of the work* The previous "old
system" of hierarchical planning failed to establish
understanding and commitment by those who should have been
involved. The pilot research committee approach to planning has
been warmly endorsed by laboratory staffs because they,
personally, provided inputs and gained familiarity with and
perspective of the entire program and an awareness of their
projected contributions to the entire program. This type of
"grass-roots" motivation must be retained, but the leadership
must also be involved in the process. Methods need to be
established to institutionalize the involvement and commitment
of the staff through proper involvement of laboratory directors,
as well. Pilot research committees are a useful means to an
end, but they are no substitute for accountable leadership,
which must be responsible for the integrity and quality of the
final product.
When laboratory personnel did feel that they had an
influence in setting priorities, they became i nvolved with input
to the Program Offices, became involved In the objective
25
-------
setting, and became involved in the design of protocols to meet
objectives. The drinking water projects are outstanding
examples and illustrate many of the elements of success that
need to be emulated by others. The reputation of the people,
their professional standing, and the history of performance
stemming from the Cincinnati laboratory and its predecessor, the
Taft Center, are influential factors which command the respect
and attention of the Program Office, A critical factor in
responsive and quality programs is the need to maintain a
continuum of qualified, knowledgeable personnel. Also, it is
important to recognize that, in the drinking water program
office, there are counterparts to ORD staff who understand the
scientific and technical issues.
3. Pre-project Evaluation of Productivity and
Costs
The laboratories in ORD are mostly media oriented, and
scientific program projects and resources are assigned
accordingly without assessment of the cost-effectiveness of
performing research in each specific laboratory.
ORD, or an outside agency, should perform a yearly
assessment of each laboratory's past performance with respect
to the quality of the research information produced, the
timeliness of delivery of research results, the cost-
effectiveness of the laboratory, and other factors which deal
w,ith a laboratory'.s performance and productivity. Only after,
such assessment has been performed and deficiences corrected
should the scientific work (decision units) and resources be
assigned to a specific laboratory.
4. Good and Poor Planning
a. Some examples of good responses by ORD
-The drinking water program at Cincinnati
-The animal inhalation toxicology program at
RTF "* * -,-
-The pesticide pilot research program
involving program and laboratory personnel
-The Wenatchee Laboratory studies of field
exposure of applicator to pesticides
(relevant work goes back in history and
should be better utilized)
These good responses all have a very important common
element; namely, the participants work at good communication.
Objectives, goals and plans are understood by the affected
parties. Solid scientific approaches are being utilized and
researchers in the laboratory are involved with personnel in
the Program Offices.
26
-------
Figure
Diagrammatic Representation of Old and New Systems to Develop
Experimental Protocols at Bench Level of Investigation
Old System
New System
Assessment of Research
Needs by ORD Headqtrs
(from internal ORD and
external sources)
Lists of In-
dividual Pro-
gram Research
Priorities
priorit-
ization
Development of Objective
Statement by Hdqtrs ORD
(includes statements of
scientific objective plus
dollars and manpower)
ORD Lists of
Research Pro-
grim Priori-
ties
Senior Managers
Intermedia Ranking of
Research Priorities
To Laboratories for
Development of Accom-
plishment Plans
Decision Units for Research
funding of
decision
unit
Laboratory Development
of Work Unit Document
for Each Accomplishment
Plan (retained at Lab-
oratory)
To Laboratories to Develop
Work Unit Documents
Lib Development Protocols
for Individual Project Con-
tracts or Grants
Lab Develop Protocols
Individual in-house
Projects, Contracts,
or Grants
27
-------
b. Some examples of poor responses by ORD
The Human Inhalation Facility at Chapel Hill is an unusual
facility, engineered to deliver the desired exposure levels,
but the scientific program or plan to utilize it is totally
inadequate-- a very conspicuous waste*
The Diesel Exhaust Program at Center Hill was prevented
from doing adequate dosage response tests because of directives
from Washington, The Epidemiology Program associated with the
Diesel Emissions Program lacked adequate and mature direction.
C. Performance of Research
EPA's intramural health effects research is conducted in
two major laboratories and in portions of three other
laboratories, which were established primarily for other
purposes. The major laboratories are Research Triangle Park,
North Carolina, and Cincinnati, Ohio. Small programs are in
effect at the environmental biology laboratories at Duluth, Gulf
Breeze, Narragansett, and the Environmental Monitoring and
Support Laboratory, Las Vegas. There are also health-related
field laboratories in Wenatchee, Washington and W. Kingston,
Rhode Island.
All of the laboratories have close relationships with
neighboring universities; in some cases the laboratories are
located on university campuses (the main Cincinnati Laboratory,
the W. Kingston Laboratory, and the Human Inhalation Facility at
the University of North Carolina, Chapel Hill).
1. Adequacy of Facilities for Researcjt
The facilities of the health effects laboratories are
generally excellent. The major exceptions are the RTP
laboratory and the W. Kingston facility, neither of which was
built for biomedical research purposes. Some laboratory
buildings, on the other hand, were constructed for biomedical
research within the past five yfears (e.g., Cincinnati). In
spite of limitations of physical plant, such as the absence of
modern animal care facilities at Research Triangle Park, EPA
laboratory staff have improvised and created the physical
conditions necessary for good research. The laboratories are,
in general, notably well-equipped for physical and chemical
analysis and modern biologic research; they also appear to have
adequate library, data processing and statistical services on
the premises or conveniently accessible.
The Committee did not conduct a formal audit of good
laboratory practice at any laboratory visited. However, the
Committee did consider as part of their general review many of
the items that would be considered in such an audit. It was the
28
-------
Committee's perception that additional attention is needed in
this area if EPA laboratories are to achieve the same standards
that EPA expects from research conducted outside the Agency and
submitted to the Agency.
Some of the specialized physical facilities are unique
in the cabability of their chambers to provide accurate
concentrations of gasses and aerosols at very low
concentrations for human exposure. The inhalation facilities
at Cincinnati fop experimental animal exposures and the
Inhalation Exposure Facility at Chapel Hill for controlled
human exposures are good examples.
Housekeeping and safety programs were generally quite
satisfactory. Animal facilities in only two laboratories were
examined (Cincinnati and RTP). The facilities at Cincinnati
have been approved by a national animal facility accreditation
committee, while no such accreditation has been attempted at RTP
due to its many deficiencies. Our Committee agrees with the
findings of the accreditation committee and suggests that EPA
devote the necessary resources to bring the RTP animal facility
into similar compliance.
2. Staffing for Research
The Committee recognizes the role of history in present
EPA staffing, not only the legacies of personnel from the
predecessor agencies and programs that were coalesced into EPA
in 1970 but also the effects of legislative actions, OMB
decisions, and civil service regulations. The Committee,
therefore, addressed only limited aspects of the total problem,
including the effects of imbalance between funds available for
extramural research and professional staff available to monitor
the research, the availability of research staff to make
effective use of special facilities, and the utilization of
scientists from academic institutions to supplement EPA
research staff.
Over the past three years,'there have been several
increases in research appropriations, without proportional
increases in personnel (Energy-Environment Act, TOSCA, CAA
amendments, etc.). One result is an increase in the burden of
monitoring extramural grants and contracts. We found great
variability from one research program to another in the
distribution and intensity of the monitoring load. There was
also much variability in attitudes toward an extramural
program. Ideally an extramural project should complement and
enrich the intramural scientific endeavor. The individual
research worker may or may not wish to expand his (her) own
research effort through an extramural grant or two.
29
-------
The Committee found that some EPA scientists were
attempting to monitor six or more extramural projects and had
no time f§r their own research. In one Instance, every member
of a laboratory division was fully occupied monitoring grants
or contracts; there was no intramural research. This is an
unsatisfactory method for establishing and maintaining a
program of high quality; it is made even worse when
appropriations are increased without additional staff
increases, as frequently happens.
EPA's special inhalation facilities were costly to build
and are expensive to maintain (over $1 million annually for one
facility). It is important that such facilities be competently
and fully staffed to be effectivly used. In fact, these
facilities are seriously underutilized, due both to lack of
skilled personnel and to lack of funds for research projects.
At the same time multi-billion dollar decisions are being made
which would benefit greatly from the kind of information these
laboratories could provide (for example, the standard setting
for ozone and NC^).
One practice which increases available manpower and
promotes intellectual quality is the exchange of staff between
universities, industry, and the Agency (Interagency Personnel
Agreement-IPA}. The exchange is largely from academic
institution to research laboratory, and we found universal
enthusiasm for this arrangement within the laboratories.
However, there seems to be little systematic effort to recruit
IPAs; most of the arrangements develop out of personal
acquaintances. While these arrangements are mutually benefical
and should be encouraged, EPA has recently adopted a policy
which will make university recruitment much more diffieult--an
academic institution must guarantee a position for a returning"
IPA. This would severely limit opportunities for young
scientists in the early post doctorate period of their careers*
3. Accountability for Expenditures
The Committee did not discover any managerial accounting
and auditing efforts within ORD to (a) analyze-the success or
failure of research projects after their conclusion or (b)
apply accounting methods to individual projects to determine
dollar allocations to equipment, salaries, travel, and
services. There is a remarkable and conspicuous lack of
managerial auditing procedures in the ORD operation. After
initial formulation of the decision units and their overall
budgets, the laboratories are assigned the implementation of
projects. In general, it Is at the laboratory level that work
unit productivity and costs must be tracked on a continuing
basis and evaluated for effectiveness and adherence to or
departure from categorical costs of ORD operations. The
insensitivity to project evaluation after completion of effort
was reflected by attitudes of managers and bench scientists.
The unawareness of costs was also widespread.
30
-------
D. The Quality of Health Effects Research
The quality of research in EPA is important not only
because any worthwhile research should meet standards acceptable
to the scientific community but also for reasons derived from
the regulatory nature of the Agency. Presumably all research
supported by EPA should be related in the short or long term to
the development of a regulation or standard. In this context
scientific information is likely to be examined critically in an
adversary relationship. Any sloppiness in conduct or
interpretation of the work is likely to weaken or destroy EPA's
position.
Another characteristic of a regulatory agency is the
importance of the credibility of research supported by the
Agency. Just as research supported by industry is often
suspected of bias, whether justified or not, so research
supported by EPA is often alleged to be biased toward the
overzealous protection of public health. This question of
credibility is a difficult one and is never easily solved. For
EPA it implies a great need not only for the highest standards
of quality in scientific work but also for active and constant
efforts o-f EPA scientists to participate in and have the support
of the scientific community.
It was our experience in visiting the health effects
research laboratories and Program Offices that EPA has many
scientists who would be welcome in the nation's universities and
private research institutions* Many of the scientists we talked
to were clearly dedicated to the best traditions of public
service in carrying out the missions of EPA, The Committee
found areas of high morale and sense of accomplishment, but was
disturbed to find areas of low morale and frustration from
frequent changes of research direction or even the absence of a
sense of direction, often stemming -from frequent changes in
1eadership.
In trying to assess quality, the Committee used what it
could of the usual criteria for evaluation. The legal
counsel's interpretation of the Privacy Act did not permit the
Committee to request a curricul urn vita_e of any scientist, but
many offered them voluntarily. The T6T1 owing information was
usually obtained from each research unit: the number of staff
with research doctorates; the scientific publication record of
the unit, in peer reviewed journals and others; the statistical
and computational resources of the unit; the procedures used for
peer review; and a sense of the intellectual climate of the
unit.
The Committee al so* exami ned the procedures used in conduct
of "extramural" research through grants and contracts.
Consultants were added as necessary to evaluate specific
programs and special facilities such as animal housing and care.
These and other aspects of quality assurance are described under
the headings that follow.
31
-------
1. Publication and Reporting of Research
Results
Scientific investigators aria part of a tradition
which places great importance on scientific peer review of
results prepared for publication in professional journals. As
with other characteristics, there was high variability of
attitudes and procedures among the different laboratories and
divisions of laboratories. Some resembled university
laboratories in their emphasis on scientific peer review of
research plans and peer review of manuscripts before submission
to high quality journals. In these cases publication was seen
as an incentive for promotion and professional advancement.
Publication"in peer reviewed journals enhances the probability
that a product of research will "stand up in court." These
research units usually had strong interactions with local
universities and promoted attendance at scientific meetings,
development of symposia and workshops, and participation by
IPAs.
At the other extreme were units that appeared to put no
emphasis on publication in the scientific literature and who
sensed that there was no incentive in EPA for such publication.
Others recognized the desirability of such publication but felt
so overwhelmed by other responsibilities that they could not
find time to publish. Some felt that internal reports were all
that the Agency expected.
The policy on review of manuscripts varied from in-house
review only to submission of the document to up to five
external reviewers. Some scientists not only met the formal
requirements but also sent their manuscripts to one or two
personal acquaintances whose opinions they particularly valued.
To ensure acceptability of research results, the studies
must be reviewed by one's scientific peers and published in a
reputable journal. Failure to so treat results of research
investigations involves the risk-that review wi]l occur at a
later date, with possible refutation of resufts'and
embarrassment to the Agency. Specific mechanisms must be
established to require peer review of QRD results and to
encourage prompt publication in peer reviewed journals.
Attendance at professional scientific meetings to present
research results is not consistently encouraged.
It has been argued by some laboratory staff that peer
review and publication are not necessary for mission-oriented
research, the EPA focus. The Committee rejects this viewpoint;
applied research, often with short-term goals, must be reviewed
and published as surely as that related to more fundamental
investigations. Applied research is the final product of years
of bas.ic research and should receive even greater review.
32
-------
2. Quality Assurance in Grants and Contracts
Examinations of this important component of the
health effects research program revealed serious problems,
which affect 1n-house performance as well as the quality and
relevance of extramural research. One aspect is wide
variability in funding from year to year and the assignment of
funds without any addition of personnel (this happens with the
Energy-Environment "pass-through" appropriation, for example).
Another serious problem is the uneven distribution of
monitoring responsibility among scientists in a laboratory
unit; some are overloaded to the extent they cannot possibly do
a satisfactory job.
Both the old and new planning systems give authority to
laboratory directors to obtain extramural services through
award of contract or grant funds. Laboratory directors rely
upon their managers to allocate resources under their juris-
diction to complete work unit tasks. Thus there is local or
section management of contractors performing services for ORD.
In depth examination of several of the laboratory, sub-unit
extramural program procedures for contractor selection,
monitoring and evaluation revealed good examples of contractor
or grantee selection based on submissions and competitive
selection. There were also examples of selection of weak or
incompetent applicants, failure of laboratory staff to monitor
performance, and almost a total absence of evaluation of the
final submission and its relevance to the ORD program and EPA
i n general.
Some scientists see grants and contracts as a desirable
extention of the scope of their personal efforts and en-
hancement of their contacts with the scientific community.
Indeed,, a healthy balance between intramural and extramural
work can benefit both EPA and the universities. These kinds of
relationships do not currently appear to be the norm.
Three kinds of arrangements are used for support of the
extramural research program; contracts, grants, and
cooperative agreements. Increasingly, contracts have also been
used to provide operations and maintenance services directly
supporting in-house efforts. The Committee did not
systematically examine the quality of contract research and did
not look at all of the cooperative agreements, a recent
development which has been little used so far.
EPA has more specific requirements for the award of
contracts than for grants. The Committee was told repeatedly
that grants are being used increasingly, because processing
them is easier and takes less time"(three or four months,
instead of six months to a year for a contract).
Examination of selected files indicated that the review
procedures for grants were being abused in at least one
33
-------
laboratory. There were examples of critical reviewers recom-
mending that the work not be funded or stating that the
proposed project was only marginally acceptable. Yet the
project officer proceeded to rationalize the reviewer's comments
ind indicated alterations in the study protocol of the grant
applicants which would overcome the objections of the reviewers.
Because the proposed project review and the project officer's
revisions were performed near the end of the Federal fiscal
year, the funds were awarded without either further submissions
or a modified submission by the applicant. In one example,
inquiry revealed that one year later the project monitor still
did not know if the grantee had modified the protocol, added
additional personnel, etc., as was recommended by reviewers and
as was rationalized by the project officer in justification of
awarding the grant.
In other examples the Committee found that external reviews
were not obtained before award of grants. (Some EPA staff
informed the Committee that soliciting external reviews of
contract proposals was illegal, except with permission of the
applicants *}
Scientists were encountered who had difficulty keeping
track,of the number of awards they were assigned to monitor;
they were not familiar with the details of extramural contract
or grant work as it progressed. The quality of investigatory
work external to EPA laboratories and supported by ORD funds was
highly variable and of great concern, mainly because ORD
oversight was usually lacking. It requires project monitoring
effort to ensure that contractors or grantees perform responsive
work on a timely basis. There is an efficient "mix" of one's
own research and that of others that can be effectively
monitored. Conversations with ORD laboratory staff suggested
that monitoring one or two contracts or grants totalling perhaps
$100-150,000 per year would be a stimulus to a senior ORD
scientist. More extensive monitoring responsibility is a burden
to the ORD scientist and, even more important, he/she cannot
efficiently discharge the monitoring responsibilities. Some
research units are so heavily committed to monitoring grants and
contracts that no scientist in the unit has any time for his/her
own research. We lesson is a clear one; Congress should not
increase R&D funding without concomitantly increasing ORD
staffing or without identifying alternative approaches.
A frequent complaint was that monitoring was handicapped by
the absence of travel funds for the project officer to visit the
institution where the research was being done.
Grant applications are of two types--solicited and
unsolicited. The latter presumably represents the spontaneous
interest of university scientists to do research on
environmental problems in which EPA might be interested. The
34
-------
common response to the Committee's inquiry was that unsolicited
grant proposals have almost no chance of being funded, primarily
because they are judged "not relevant." It seems clear that EPA
scientists are using grants in lieu of contracts,, that they
monitor them like contracts, and that there is little
opportunity for "investigator initiative."
The mechanisms for soliciting grant proposals vary from one
unit to another. We found little evidence that EPA has found
effective ways to interest university scientists in its problems
on a sustained basis.
Another practice, employed to extend the time for longer-
term research but with the potential for abuse, is the "front-
end loading" of a newly awarded grant. In this practice the
amount of the award may be as much as twice the amount of the
first year's budget. The investigator can then request an
extension for a second year without additional funds, an action
routinely granted without a critical review of research
progress. The Committee does recognize the need for assured
funding of projects that may require more than one year to
complete. However, if funds required for more than the first
year's operation must be obligated, the project must be
carefully monitored to assure that funds for the second year are
required and appropriately used.
Another shortcoming of the present EPA system is the
absence of a routine operational audit of the quality of
extramural research. Individual scientists and laboratory
directors told us that a contractor or grantee who performed
poorly was not likely to obtain another grant or contract*. This
informal and spottily used system is not adequate to assure the
high quality of extramural performance.
ORD's entire program to make extramural awards of funds
under contracts, grants or cooperative agreements requires a
thorough overhauling. Extensive standard operating procedures
for awarding grants and contracts exist in the Agency; they are
voluminous, difficult to comprehend, and are avoided by
laboratory staff. It is necessary to establish simple, explicit
procedures to be followed by laboratory directors and scientists
throughout the life of an extramural award. At present,
laboratory directors are expected to satisfactorily complete
work unit tasks; extramural projects are their choice and
responsibility. The Committee recognizes the need for
extramural assistance, particularly if the trend continues to
increase ORD dollars without increasing the number of positions
for investigators, but the procedures for extramural programs
must be placed on a more defensible basis throughout ORD.
35
-------
3. Career Opportunities
The civil service system was examined as an Influence on
the quality of research programs and on career opportunities for
EPA scientists. There were several examples of negative effects
of the civil service system; for example, it does not permit the
flexibility to hire new people or to move people as program
orientation shifts. Consequently, there are cases in which
excellent scientists are placed on projects where their
expertise is not needed and where they have to be "re-tooled".
Although the Committee talked to people who had been
promoted because of the quality of their research, more
frequently promotion related to the assumption or increase of
administrative responsibility. Many times a good scientist
makes a poor administrator, but the scientist takes the
administrative position for the higher salary, not because he or
she has management skills. Talented researchers must be
encouraged to continue as investigators. Mechanisms must be
instituted to further their professional development and their
allegiance to the Agency.
It appears that the policies and procedures for
advancement do not encourage the emergence of either top
scientific or managerial performance. The system does encourage
job-hopping by bright people, particularly those in Program
Offices. A promotion ladder based on scientific achievement
rather than administrative responsibility would help to solve
this problem. Many industrial research laboratories use dual
ladders for advancement-- administrative and research. Senior
research personnel are rewarded with remuneration and privileges,
comparable to those of a senior manager. ORD is experiencing
difficulty in retaining research physicians, epidemiologists,
and toxicologists, among others. At the time of this writing,
the Human Inhalation Laboratory in Chapel Hill, N.C., a unique
facility, is virtually without physicians to perform the
research vital to scheduled regulations in the air media.
Administrative mechanisms should be developed to offer a
challenging career ladder to these professionals if first rate
health effects research is to be performed in ORD. The
Committee recognizes that many of the reforms addressed
elsewhere in this report will improve conditions for these
professionals, but an explicit analysis of conditions and
incentives related to a research career in ORD must be performed
and improvements implemented where necessary.
4, Other Components of Quality Assurance
Performance evaluations of individuals and laboratories
are often perfunctory. Many individual scientists were unclear
about the criteria applied to their evaluations and advancement.
36
-------
Evaluation of laboratories is not being done in terms of good
laboratory practices, rewards and incentives, budget and
resource allocations* and accountability.
Personal scientific integrity is difficult or impossible to
determine in a study of this kind. To the extent that personal
conversations, attitudes expressed, and measures taken to assure
the quality of research, design, and analysis can be used to
assess scientific integrity, the Committee was favorably
impressed. If there were subtle biases in the interpretation of
research results, they were not detected in this study.
There are periodic "program reviews" in which head-
quarters' staff members visit the laboratories. These are
described by the laboratory scientists as superficial "show and
tell" sessions. There is limited scientific feedback from
headquarters' staff, and the only benefit to the laboratory is
the stimulus to prepare material for presentation.
By contrast, it was noted that when N!H is involved in a
jointly sponsored project, there is a visit by NIH staff
members, who conduct an intensive critical analysis of the
proposed research project. EPA staff who have thus been "nailed
to the wall" to defend their projects say they would welcome
this kind of evaluation of EPA projects.
There appears to be a general lack of understanding of the
Science Advisory Board and its constituent committees by
laboratory staff. In view of this, it was not surprising that
the Science Advisory Board was criticized for its lack of
scientific interaction, failures in communication, and lack of
subsequent feedback.
5. Interagency Agreements
The Interagency Regulatory Liaison Group (IRLG) is a
new activity which seems to be off to a promising start. Since
it is a developing program, no attempt was made to evaluate it.
Other programs involving interagency agreements have had
mixed success, at best. EPA has substantially supported the
National Center for Toxicologic Research since its inception,
with little evidence of any product benefiting EPA. Disap-
pointment was also expressed about, interagency agreements with
Los Alamos and Oak Ridge National Laboratories and three of the
National Institutes of Health.
A significant portion of EPA's health effects research is
supported by interagency^agreement for the special Energy-
Environment appropriation. No attempt was made to examine this
program i n detai1 *
37
-------
E. Other Relevant Topics
1. Long Range, or Core, Program Research
There are subjects for research which are important
to several of the media programs. Examples are the properties
of particle dispersions, be they in air or water, because of
their relevance to eollaction of the disperse phase prior to
effluent discharge, to particle deposition in the human
respiratory tract and to particle retention or solubilization
in the human gastrointestinal tractj epidemiological
methodology because it is a major tool for relating exposures
to pollutants to potential effects in the exposed population;
and techniques of risk assessment and presentation of the
implications prior, to judging acceptability of risk. The.re
should be a long term ORD investment in researchers and
facilities to develop highly active and productive groups in
those areas of research which are central to large segments of
Agency regulatory activity. This investment is currently being
augmented by initiation of extramural university centers. It is
planned to shuttle ORD staff between their resident
laboratories and the centers for "leaves of absence" during
which they can pursue studies in core areas while upgrading
their capabilities on a university campus. We applaud this
plan, but also see the need for small, active core research
groups in ORD laboratories. Allocation of a specific
percentage, at least 10%, of the ORD budget for relevant
research in core subject areas, but not on projects
specifically traceable to immediate program needs (6 months-2*
years), is a reasonable assignment of funds. There is no
obstacle to this programming of funds under the present
procedures for funds authorization. They are part of the funds
assigned to research for the specific statutes, because results
will be applicable to those statutes, as well as to others.
2. ORD/Congressional Staff Information Transfer
The relationship and relevance of ORD projects to
regulatory needs is not always obnnous, particularly to non-
scientists. It is essential that members of Congress and their
staffs understand the efforts of ORD. Such understanding does
not develop accidentally. ORD should develop a plan to
regularly inform interested members of Congress and their
staffs of the results of ORD efforts and the manner in which
they further the goals of statutes administered by the Agency.
ORD's investment in what is essentially an educational program
for legislators should involve ORD's roost senior scientific
staff. It is critical that this communication effort include
laboratory personnel who are directly involved in the conduct
of research. We note the 1978 and 1979 Research Outlook efforts
by ORD, but believe efforts must go far beyond this and must
incorporate personal communications, as well as transfer of
printed information. The concepts of chronic disease, multiple
etiologies of disease, host factors, and cumulative effects, to
name only a few, are complex and crucial to understanding the
underlying approaches to research in ORD.
38
-------
VI. UTILIZATION OF ORD RESULTS
Different Program Offices utilize ORD research results to
different extents. Senior program managers indicated that they
did not look to ORD for results; rather, they sought capable
laboratories and investigators related to their needs, be they
within or outside the Agency, A Radiation Program manager
indicated that ORD has little capability to assist them; ORD
has no capabilities in the area of biological effects of noise.
ORD appears to have little involvement with the Toxic Substances
Office. The Water Program draws heavily on ORD at the present
time, and recently ORD had a major involvement in the
formulation of criteria documents for 65 water pollutants.
The input of research to the screening test and risk
assessment process was clearly evident from the Drinking Water
Research Program in Cincinnati and the Pesticide Programs at the
Gulf Breeze and Wenatchee Laboratories. Their scientific
standing is recognized. The respective leadership has
maintained the kinds of communication necessary (with the help
of pilot research committees) to keep the personnel in
Washington knowledgeable and involved.
It is not surprising to find that the utilization of
results from ORD projects is not carefully tracked when the
joint planning of research by Program Offices and ORD is in its
infancy with the pilot research committee program. Program
managers elaborated on many needs not being met by ORD; there
were few illustrations of ORD responsiveness to programs and
subsequent incorporation of results into regulatory programs. On
the other hand, ORD staff were often praised for their responses
to requests for preliminary review of regulatory documents,
consultation on imminent regulatory submissions to the courts
and, in general, what can be characterized as technical support
to the Program Offices, The Committee was not able to estimate
the average percentage of ORD professional staff time devoted to
technical support; it varied with individual research sections.
It was clear that in some instances it represented a significant
portion of some individuals' time. This technical support has
on some occasions played a critical role in the Agency's
formulation and defense of regulations.
The ORD function in the Agency is defensible mainly on the
basis of program utilization of insights and results developed
intramurally or extramurally under its auspices and guidance.
The Committee found that ORD did not fully recognize or accept
this criterion for judging its efficacy, had not developed
mechanisms for efficient utilization of research results by
Program Offices, and did not maintain records of results which
had been incorporated into regulations.
39
-------
VII. STATUS OF IMPLEMENTATION OF TWO SETS OF NATIONAL ACADEMY
OF SCIENCES (NAS) RECOMMENDATIONS TO EPA
The analytical study of Research and Development in the
Environmental Protection Agency conducted by the Environmental
Research Assessment Committee (John M. Neuhold, Chairman), of
the National Academy of Sciences, National Research Counei1tin
1974 and 1975 set forth a number of useful recommendations.*
Before that, a Review Committee on the Management of EPA's
Research and Development Activities (Robert W, Berliner,
Chairman) had developed recommendations submitted to the Agency
on August 27, 1974. Our Committee (HERRG), therefore, in its
collective judgment, has attempted to evaluate the extent to
which former.recommendations have or have not been implemented.
This final exercise was undertaken at the end of our study when
all visits had been completed. It was possible by this means to
add a different, but closely related, viewpoint against which to
compare our own observations of performance and changes during
the past four years.
Although there has been significant improvement in
selected aspects of EPA research planning and management, most
notably the development of pilot research committees with
representatives from across the Agency, the overall planning and
management system is still unsatisfactory. Many of the reasons
for inadequacies in the system in 1974 still exist today and
will be enumerated in the following*
A. Recommendations from t_he |ny1ronmental Researc^h_
Assessment Committee of T'97'5 ** —— — r.
(1) "EPA's research and development should
concentrate primarily on support of the Agency's
decision making and anticipation of future
problems."
There are improvements arising from better communications
between research workers in the laboratories and the Program
Offices. The pilot research committees have helped establish
communications and understanding.
*AnaTyticalStudies for the U.S. Environment _a1.. Protection
Agency, Volume III, "Research and DeveTopmentin the
Environmental Protection Agency/1 Environmental Research
Assessment Committee, Commission on Natural Resources, The
National Research Council, National Academy of Sciences,
Washington, D.C. 1977.
**Ib_td. page 2.
40
-------
{2} "EPA should supplement its primary research
responsibilities with some fundamental research
to help advance understanding in environmental
sciences and technology."
Planning for fundamental or longer term research is still
inadequate. However, to achieve the right kind of balance there
first needs to be a close and direct relationship between
researchers and program managers. Both must understand the
research process and information needs of the regulatory
process.
(3) "A new legislative mandate will be required if
EPA is to conduct effective anticipatory and
fundamental research."
The HERRG Committee does not agree that additional
legislation is needed to fund and conduct "anticipatory and
fundamental research."
(4) "We recommend that the Office of Science and
Technology Policy (OSTP) develop a federal
environmental research, development, and
demonstration strategy that includes designation
of the appropriate roles of all participating
federal agencies and existing interagency
coordinating committees, and delineation of the
relationships between federal and nonfederal
research and development* The OSTP should
coordinate the implementation of the strategy .-
through its mandated consultations with the
Office of Management and Budget (OMB) about the
scientific programs of federal agencies."
This recommendation has not been followed, per se.
However, the Interagency Regulatory Liaison Group is seen is an
excellent initiative which has the potential of reducing
duplication and confusion among agencies. Better coordination
of research efforts and better agreement on the methodologies
applicable to hazard assessment are encouraged by this
Committee,
(5) "We recommend that the management of all research
and development in EPA be centralized in the
Office of Research and Development {OR&D}."
There seems to be progress in centralizing the management
of R&D within OR0, but a number of Program Offices administer
R&O contracts and grants directly. The Committee urges that
this Academy recommendation be implemented to assure that proper
oversight and scientific peer review be applied whe.never
research is conducted by the Agency,
41
-------
(6) "EPA's research program needs to be better
organized for balance and continuity, through
planning developed around a logical conceptual
framework of environmental protection..."
A number of areas within the present EPA research and
development program are still not aligned within a logical
conceptual framework of environmental protection and thus are
not as effective as they could be. The conceptual framework
proposed in the earlier NAS/NRC report (1977) still appears to
offer a sound framework for the assessment of research needs,
the planning and conduct of research, and the utilization of
research results. The framework is shown below:
Framework for Environmental Protection
Natural,
Economic, and
Human Resources
Production Processes
(including control technology),
Transportation, and Use
Other Environmental
impacts Including
Unintentional Release
of Pollutants
Environmental
Quality (air, water,
soil, biota)
Decisions to Control
Effects
42
-------
(7) "A central function of scientific support to
decision making should be to provide integrated
assessments of available scientific, technical,
and economic data pertinent to pending decisions
in forms suitable for use by Agency decision
makers. We recommend that the importance of this
function be recognized by giving it formal status
arid organization in QR&D."
The importance of integrated assessments continues to be
recognized, and the Agency is moving toward establishing the
formal organization required to make such assessments. When
such an organization is fully operational, it should be of major
assistance in providing information that is useful to the
regulatory decision makers; but of equal importance is inform-
•ation that is crucial for the planning of a responsive research
program. Carefully conducted assessments can identify gaps in
research information or parameters that have the greatest
influence on the,effects of emissions. In the absence of such
assessments there is a risk that research efforts may be
directed to developing information that may have limited value
in establishing or reassessing standards or in guiding their
enforcement.
(8) "The research planning system now in use in OR&D,
characterized as "top-down" in structure, should
be retained for research in support of decision
making. For anticipatory and fundamental
research, however, we recommend a "bottom-up"
scheme that relies on the scientific community to
identify research needs."
Except for the pilot research committees, the planning
process remains "top-down." Substantial improvements are needed
to achieve involvement of those generating and using the data.
(9) "We recommend that block funding of extramural
grants, contracts, and-interagency-agreements be
considered as a mechanism to establish centers of
excellence, federally funded contract research
and development laboratories, and umbrella
interagency agreements to supplement the intra-
mural research and development program."
To date, block funding mechanisms have not been extensively
used by QRD, although legislation has provided the opportunity
for use of cooperative agreements that may very well match ORD
needs. ORD has made preliminary plans for using such agreements
and should proceed expeditiously to implement their use. Such
agreements offer an opportunity for a complementary approach to
the present system of grants and contracts for extramural
performance.
43
-------
(10) "All proposals and completed research should be
subjected to review on their technical merits by
scientific and technical peers."
Peer review of proposals and completed research was
inconsistent and, in many cases, inadequate.
(11) "We recommend the use of a parallel grade
advancement system, based on performance of
research, that does not require researchers to
assume administrative or managerial tasks to
attain promotions."
There was little evidence of implementation of a parallel
grade system. In some cases, individuals have accepted
administrative or managerial assignments based on the perception
that such assignments are critical to obtaining promotions,
B. Recommendationsof the Review Committee on Management
of EPA's Research and DevelopmentActivTties *
The Review Committee report noted that the present (1974)
"Office of Research and Development planning and management
system fails to meet the needs of the Agency" and proceeded to
identify two main categories of failure; (1) the nature of the
system itself and (2) external constraints as perceived by the
Office of Research and Development and communicated to the
Review Committee.
1. The nature of the system itself.
a. "Planning is separated from
responsibility for execution, leading'
to severe resentment among performing
researchers. The assignment of
responsibility for specific actions and
decisions is difficult.11
There is still an inadequate linkage between planning and
responsibility for execution that is apparent, in varying
degrees, at all levels of the organization below the Assistant
Administrator for Research and Development. An individual
researcher charged with responsibility for performing a task may
have no input to the planning of that task,
b. "Priorities do not reflect the needs of
regulatory offices and regional offices
because of the 'vacuum cleaner1
approach to soliciting ideas,, and the
system-induced barriers to using common
sense in the selection process."
*Ibiq. page 96.
44
-------
There has been Improvement in the establishment of
priorities in selected areas, most notably those for which pilot
research committees have been established, to yield a research
program potentially more responsive to the needs of the Program
OfficeSt In other areas, the research program is less clearly
defined and priorities have not been established. Faced with
necessarily limited resources, the responsible individuals have
frequently elected to continue work in all areas at a reduced
level of effort rather than electing to eliminate or defer the
lowest priority projects* The result is a reduced potential for
success in the highest priority areas because of lack of funds.
c. "Inadequate attention has been paid to
the possibility for trade-offs, or
modificat ions in budgeted costs, among
various projects. This has aided in
the development of a situation where
there is only a series of discrete
projects and no Agency program. This
situation is further aggravated by the
absence of long-term {3-5 year)
planning,"
Long-range planning within the Agency remains inadequate.
The large portion of the planning within ORD is necessarily
dependent upon the needs identified by the Program Offices,
These long-term needs have often been inadequately stated, if
at all, thereby handicapping the development of a responsive
long-term research plan. It was originally anticipated that
the pilot research committees would develop a strategic plan
for their areas of responsibility. However, this was not done,
in part because of the timing and pressure of the ZBB process '
which forced the pilot research committees to take a shorter-
term outlook. An additional factor which should also be
recognized is the reluctance of some individuals to engage in
defining a strategic plan until they are certain that managers
are serious about the effort.
d. " The complexity of the system makes it
counter-productive. The large amount
of paperwork and excessive bureaucratic
review is a wasteful consumption of
time and energy. The needs of the
Agency are complex; however,"this does
not change — but rather heightens — the
need for a simple and understandable
planning and reporting system clearly
directed by the Assistant Administrator
and in which field personnel have a
real participatory input,-"
45
-------
The planning and management system is still extremely
complex. Involves a large amount of paperwork, and is often a
waste of valuable time and energy. An inadequate amount of
authority has been delegated downward to the laboratory
directors and lower echelons of the Agency. In those cases
where authority has been delegated, there appear to be excessive
requirements for keeping all upper levels of the Agency
informed. One example is the use of the highly structured
quarterly "Project Status Reports," which include detail at the
task level (tasks ranging in expenditures of less than $10,000
to over $200,000' per year); the volume of material developed at.
the laboratory scientist's level is passed successively to the
Division Director, the Accomplishment Plan Manager, and the
Office of Health and Ecological Effects and its various staff
units.
e. "Accountability is made impossible by
the parallel but separate management
systems — some for housekeeping and the
others for program content--and by the
hopelessly complex Program Area
Manager-Program Element Director-
Program Assessment Group-Strategies
system which obfuscates management
responsibi1ity."
The chain of accountability is extremely difficult to trace
from the laboratory scientist (either in-house or engaged as a
contractor or grantee) to the Assistant Administrator for
Research and Development. The "chain of command" is excessive
with numerous intermediate steps that serve only to delay or, in
some cases, reprocess information without serving.any clear
management functions to enhance research productivity,
efficiency* or responsiveness. Indeed, in many areas the number
of information reprocessors and/or relayers makes it difficult
to identify the laboratory scientist.
f. "Excessive requirements for detail at
all planning levels lead to an
oversized headquarters staff and to the
stifling of innovation in the
laboratory."
The level of detail required at all levels and the transfer
of materials with limited informational or management value
continues to contribute to the maintenance of an overly large
Washington staff. In what appears to be a contradiction, the
Washington staff is understaffed in relation to the amount of
material being transferred and processed. Unfortunately, much
of this effort is misdirected. Because of the attempts to
maintain detailed accountability of even extremely small
projects, the innovative responsibilities of the laboratory
scientists continue to be unfulfilled,
46
-------
g. "The existing management structure does
not allow for the corrective feedback
and flexibility which are essential to
any successful research and development
program."
Because the "chain of command" is so long and the
communication pathways are jammed with trivia, corrective feed-
back does not occur at the level required for effective manage-
ment. The rigid system of accountability to the laboratory
directors diminishes the flexibility needed for operation of a
responsive and innovative research program.
h. "A long-term program designed to meet
stated goals is missing and this is
vital for any scientific venture."
The ORD program has few clearly stated long-term
strategies, specific to each Program Office, with easily
identifiable objectives and goals. In the absence of long-term
objectives and goals, the Agency's research and development
resources seem excessively preoccupied with meeting short-term
goals, some of which are restatements of goals not previously
attained,
i. "A false sense of control is generated
by the highly structured mechanism for
planning,"
The highly structured planning and control system,-'which
generates considerable activity, has promoted the feeling
that something is happening that is of a positive nature. The
widespread lack of clearly stated and agreed upon long-term
objectives and goals, however§ makes it difficult to determine
whether the movement is positive, negative, or random in nature.
j. "Relationships between the headquarters
and field are strained at best; a state
of frustration in the field staff is
apparent."
Considerable frustration is apparent in many of
the organizational units below the Assistant Administrator's
office. In many cases, the individuals have resigned themselves
to tolerating a work environment that is constantly, changing,
but rarely for the better,
2. External constraints as perceived by the Office of
Research and Development.
a. "Enabling legislation is noncoherent
and mandates a set of unbalanced and
uncoordinated research objectives and
timetables."
4?
-------
The enabling legislation for the Agency has been
and continues to be viewed as noncoherent, mandating a set of
unbalanced and uncoordinated research objectives and timetables.
Since the enabling legislation has not and may not be changed in
the near future, ORD has no real choice but to accept the
situation that exists and strive to adjust its planning and
operations accordingly.
b. "The lack of an integrated approach to
environmental pollution control in the
Agency as a whole makes an integrated
research and development program very
difficult to form."
Although some individuals view the Agency as not
having an integrated approach to environmental pollution
control, some progress has been made, and the use of approaches
such as the pilot research committees offers the opportunity for
developing an integrated research program with long-range
objectives and goals as recommended in 1974.
c. "Civil Service rules, parochial
political pressures, and human nature
combine as barriers to the
simplification, assembly into 'critical
masses,' and logical organization of
the research units which were inherited
by EPA when it was created."
Civil service rules, parochial political pressures, and
human nature continue to be barriers to simplification, assembly
into "critical masses," and logical organization, of the research
units. Of perhaps equal importance has been the failure to
recognize that in the absence of a clearly recognizable research
and development strategies specific for the Program Offices, the
constraints of civil service rules, the influence of political
pressures, and human nature will have substantial adverse
impacts on the research program. An identifiable strategy with
well thought out objectives and goals will go a long way toward
minimizing the impact of factors that can push a reaction-
oriented program, with ill-defined objectives and goals, off
course. As addressed elsewhere in this report, civil service
rules do adversely impact the research program, and suggestions
for change are offered. However, in the absence of changes in
the rules, the situation roust be accepted and plans developed
within the constraints of the rules. Parochial political
pressures have been, and probably will continue to be, brought
to bear. However, it should be recognized that the Agency has
strong political supporters, who can counter parochialism if
they know that the Agency has a research program that is
scientifically and managerially sound and programatically
responsible with a plan for the future. Without question human
nature may at times offer constraints, but, if properly
directed, can also provide forward momentum.
48
-------
d. "A level budget (except for the energy
'roller coaster1 of FY 74,75,76)
prevents transitions which would be
possible in a steady growth situation.
An internal 'roller coaster' budget
appears to be particularly disruptive
to individual projects."
The level budgets of fiscal years 1974, 1975, and 1976 were
given as the reason for the failure of the ORD planning and
management progression. The level budget was said to prevent
transitions that would be possible in a steady growth situation.
Recent budgets have shown an increase; however, transitions do
not appear to have occurred any more smoothly. A concern raised
even more frequently than the shortage of funds is the
restriction on the number of full-time employees. Although the
impacts of the restriction are real, little has been or is
likely to be accomplished by merely accepting the OMB mandated
personnel ceilings until they can be changed. Until changes are
made,, it would seem prudent to exercise greater care in the use
of available personnel and to have a strategic plan for addition
of personnel when vacancies do occur. Such a strategy for the
management of personnel resources is an essential part of the
total Agency research and development plan and is the only way
the personnel resources (as to number of individuals with
specific types and levels of disciplinary training) can be
matched to the long-term needs of the Agency.
The 1974 letter report of the NAS/NRC Review Committee listed
four major recommendations.* The recommendations have been
implemented to varying degrees and, even where not fully
implemented, still seem appropriate. Because they are still
germane, each is reviewed below.
1. "The Environmental Research Objective Statement-
Research Objective Achievement Plan-Program Area
Manager-Program Element Direction-Program
Assessment Group-system should be abolished.
Responsibility for carrying out a program
designed to meet the goals of the Office of
Research and Development should be delegated
directly to the National Environmental Research
Center directors. Resources of manpower and
money should be allocated directly to each
National Environmental Research Center."
* I b i d. page 98.
49
-------
-- The planning and management system referred to has
been largely abolished. It has not been replaced by a system
that is understandable to all parties involved; thus vestiges of
the old system remain. The five Pilot Research Committees cover
a portion of the ORD program and partially meet the planning
function requirement. The National Environmental Research
Centers and related field stations in existence in 1974 have
since been separated into 15 individual laboratories, which
report through four deputy assistant administrators to the
Assistant Administrator for Research and Development. Although
allocations of resources are made directly to the individual
laboratories, there appear to be numerous strings attached which
severely restrict the authority of the laboratory directors.
2. "The line reporting within the Office of Research
and Development should be from the National
Environmental Research Center directors to the
Assistant Administrator. The Assistant
Administrator should have a small staff to
perform only staff functions and not to serve as
a filter or layer through which the National
Environmental Research Center directors report.
This should develop into a simple pyramidal
management system through which all direction,
supervision, and evaluation is accomplished.
This would, in effect, eliminate all layers or
parallel management plans and result in a clear
chain of authority from the individual
researchers to the Assistant Administrator for
Research and Development. The pyramid should
decentralize quickly from Washington Headquarters"
to major field urtits. The Headquarter1s staff
should be trimmed appropriately and those
necessary for "Washington liaison" activities
clearly labeled. We did not have sufficient time
to evaluate the role and position of the
Washington Environmental Research Center. Such an
evaluation should be made.
"Because of the recent formation of the
Agency by coalescence of disparate portions
of other agencies, a particular need for
intra-agency communication exists. To this
end, a planned continuing rotation of field
personnel into and back from a small
Headquarters staff unit and between other
units should be carried out. Short term,
non-government talent should also be worked
into this rotation system."
50
-------
--The Washington staff of ORD is still quite large with
a relatively large number of individuals serving in special
staff roles and on numerous ajl hoc committees. Clear chains of
authority do not exist between individual researchers and the
Assistant Administrator for Research and Development; rather
there are numerous filters through which information exchange
must take place. Despite the largeness of the Washington staff,
many appear overwhelmed by their work load, while others
apparently fill slots for which there are no longer meaningful
work assignments. Approximately 90% of the work load seems to
be carried by one-half of the staff.
Communication between Program Offices and the Office
of Research and Development has been virtually non-existent in
some areas. The five recently organized pilot research
committees appear to have helped improve intra-agency
communication and offer considerable promise for further
improvement.
Rotation of field personnel into and back from head-
quarters has occurred to a limited extent, but more exchanges
are needed. A limited number of short-term, nongovernment
individuals have rotated through the system, however more
exchanges of this type are also needed,
3. "The function of the Assistant Administrator
for Research and Development should be to
assemble, analyze, and clearly define Agency
research and development needs and objectives
with the participation of the other Assistant -*
Administrators and the National Environmental
Research Center directors as the mechanism to
develop goals, programs, and priorities. He
should allocate objectives and the resources for
their accomplishment to the National
Environmental Research Centers. Once allocation
is decided upon, the performer of the research
or development should be linked directly to the
user of the projected output for information
exchange.
"A performance evaluation should be set up to
include continued inputs from users, and outside
visiting committees reporting at a high level
should be regularly employed* The system of
visiting committees employed by the National
Bureau of Standards should be studied for
applicability.
"A plan for a 3-5 year period to be revised at
least annually should be developed."
51
-------
--The Assistant Administrator for Research and
Development has not systematically assembled, analyzed or
clearly defined research and development needs and objectives.
"The Research Outlook: 1978-1983", which has been published, and
"The Research Outlook: 1979-1984", currently nearing completion,
are perhaps the most definitive statements of research and
development needs and objectives. However, neither document is
an adequate statement of near-term, mid-term or long-term plans
and objectives. Participatory discussions have apparently
occurred with laboratories. Until initiation of the pilot
research committees, most planning activities were carried out
in headquarters with only limited and late stage input from the
laboratories. With the advent of the pilot research committees,
laboratory and Program Office input to near-term research
planning has occurred in those research areas for which
committees have been developed. This has had a positive impact
on planning; however, in most cases where the laboratory
director was not involved in the committee's activities, it has
minimized the ro,le of the laboratory director in the planning
process. For a majority of the research programs, the
laboratory directors and staff have been involved primarily in
near-term planning and then most frequently at late stages of
the budget cycle. In many cases the input has been fragmentary
and spurious, i.e., "What would you and your people like to do
next year?"
Resource allocations {personnel and finances) are in a
continuous state of flux. As expected in relation to the
Federal budget system, changes are made up to the beginning of
the current fiscal year, but frequently continue on throughout
the year. The major certainty appears to be that change will
take place. The laboratory directors apparently are given
little authority for shifting resources within program areas and
even less authority for shifting resources between program
areas. This lack of flexibility, with continuous management
from headquarters, appears to have had a negative impact on the
productivity of the programs. EPA scientists, in many cases,
are confronted with changes in program direction and level of
effort with very short notice. Extramural projects have, in many
cases, been treated as the most flexible portion of the system.
Contracts that have been expanded or shifted in direction on
very short notice have served to alienate substantial portions
of the research community. Precipitous actions, discontinuation
of programs, or shifting of program direction raises legitimate
questions concerning the adequacy of Agency research and
development planning. Precipitous increases of funds, although
having associated moments of elation, are usually followed by a
recognition that the time and personnel resources available do
not allow careful selection of new contractors, resulting in
projects that are less successful than they should b$.
52
-------
4. "Not only the changing nature of
environmental problems but also the
exigencies of the economy, suggest that It
would be inadvisable to build up a large
permanent staff. Rather, maintaining the
necessary competence to monitor grant and
contract work as needed would appear to be
a prudent course.
"A careful review of the contract and grant
procedures should be undertaken."
--The Agency has not given adequate attention to
developing a strategy for the implementation of its research
program, i.e., balance among intramural research, contracts,
grants and interagency agreements. Although the mandated
ceiling on numbers of personnel is recognized, the Agency has
not made adequate plans for living within that ceiling. To
circumvent the personnel ceiling, contracted personnel are used
on site at many laboratories to perform maintenance operations,
thereby extending the work force. There are numerous
individuals who are faced with a multitude of competing
responsibilities: performing hands-on research; supervising
technicians who directly assist them; preparing orders and
monitoring the efforts of on-site contract personnel; soliciting
and reviewing research grants and proposals; monitoring research
being performed by contractors and grantees, either by personal
visit or review of innumerable reports expected of the
contractors and grantees; and participating in the preparation
and review of criteria documents and related material. In some
instances, there are experienced scientists and managers
available who do an excellent job of balancing and meeting these
competing demands. In a few instances, individuals, who have been
unwilling to accept the demands placed on them, have retreated
into their corners to do "their thing," i.e., perform specific
research in line with their interests, and are content to let
the system go on its own merry way. Although this has solved
their immediate problem, it has increased the, workload and
demand placed on their colleagues* In many cases, the demands
are excessive in relation to the experience and training of the
staff member, and one or more of the aspects of the job are
performed poorly.
The impact on both intramural and extramural research is
apparent. The impact on the intramural program is discernible
by the fact that many EPA scientists do not publish because they
have performed relatively little research. A review of how
selected grants and contracts were initiated and monitored
suggests that, in some cases, the individuals involved did not
have adequate experience or time to perform their assignments.
A related and contributing factor has been the development of an
"unwritten" set of procedures for promoting the use of grants
rather than contracts because of the more cumbersome nature of
the contract award process.
63
-------
In summary,, a careful review of contract and grant
procedures is as much needed now as it was at the time of the
NAS/NRC report. A key aspect of such a review should be the
development of a strategy dealing with how much research can be
appropriately performed in the Agency and how extramural work
can best be performed.
54
-------
VIII. COMMUNITY HEALTH AND ENVIRONMENTAL SURVEILLANCE SYSTEM
(CHESS):. AN INVESTIGATIVE REPORT
A, Background erf the CHESS Program
The Community Health and Environmental Surveillance System
(CHESS) was initiated about 1970 and involved collection of data
during the period 1970 to 1975. This research and surveillance
program was designed to investigate the relationship, if any,
between air pollution and health in human populations (up to a
few thousand persons), studied at single contacts or followed
for short periods of time (up to two years), for
characterization of health status. These observations were
coordinated with observations on air pollution in the
environments of the study populations. The populations and
areas included for study were selected to represent pairs or
larger sets of contrasting exposures, for example, a "clean" and
a "dirty" town or a series of several communities with a known
or suspected substantial range of air pollution conditions. Most
populations consisted of persons not previously known to have
any special health problems, although some studies within CHESS
were directed at groups defined by disease conditions, for
example, known asthma patients.
The program operated from 1970 to 1975 and resulted in a
major publication in May 1974 (Health Consequences of Sulfur
Oxide: A Report from CHESS. |97q-T9'71T.That publication
included analysis and interpretation of the first two data
collection years. Other smaller papers and presentations
involved these and some later years' data. The major review in
1974 implicated sulfates, sulfuric acid, and sulfur dioxide as
causing health effects, chiefly respiratory tract disease or
disturbance of pulmonary function, at or near levels of these
pollutants comrnon.ly considered "safe." That report was
extensively reviewed by a number of individuals and groups and
received both praise and criticism. In part because of some of
the criticism, CHESS, in its original form, was discontinued.
It was recommended, however that additional substantial efforts
be made to optimally use the collected data beyond those uses
reported in 1974. Special features to be considered in further
work were to include: (1) analysis of extensive data collected
from 1973 to 1975 and not included in the 1974 report; (2)
improvements of statistical data and analytic techniques; (3)
assessment of validity of coded data and of extent of coding
errors or other correctable problems in the data set; (4)
increased objectivity i'n interpretation of findings; and (5)
assessment of confidence range of estimates of pollution.
55
-------
B. FJndingl of the Subgroup
During the site visit in September 1978, the status of the
CHESS program was reviewed and a summary follows. The mechanism
for continuing work on CHESS is a contract from the Environ-
mental Protection Agency to the University of North Carolina,
Chapel Hill, principal investigator Dr. Carl Shy. This contract
work is closely followed by members of the epidemiology division
and the statistics unit of the Health Effects Research
Laboratory, Environmental Protection Agency. Dr. Shy was
formerly extensively involved with the CHESS project as a member
of the fepidemiology unit; he is now a member of the faculty.
University of North Carolina. The plan is to review all of the
CHESS data collected for 1970 to 1975. The contract to the
University was let in September 1977.
To date there has. been a major effort to validate the CHESS
data sets. This was projected to require two years but is now
expected to be completed about eight months ahead of schedule
because special priority was given to the validation project.
This has been accomplished in spite of a budget deletion of the
funds planned for this purpose, thereby making it necessary to
discontinue other work to meet this mandated task. The
validation project is designed to identify discordances between
manually recorded original data and tape recordings on exposure
(pollution), outcome (health measures), and control demographic
and confounding variables. It is being done very effectively
under'the direction of Mr. 'Gerald Nehls, Director of the Data
Management Unit in the Health Effects Research Laboratory. It
must be noted that any validation of these old data is now
limited to validation of the previous coding and automating and
not to any review of the correctness of initial observations of
symptoms and other health effects.
A standing committee has been created, reporting to Dr. Shy
and supported under the research contract, to review all planned
publications of the CHESS data. The committee presently
consists of Dr. Warren Winkelstein (University of California),
Dr. James Grizzle (University of -North Carolina), and Dr.
Michael Lebowitz (University of Arizona). This 'committee has
just been funded, and its effectiveness cannot yet be judged.
The membership seems appropriate, and the plan for a standing
procedure for outside review is a useful move in response to
criticism regarding objectivity of reporting.
A report of a current analysis of a portion of the CHESS
data from the Southeast region (Charlotte, North Carolina and
Birmingham., Alabama) was presented to the site group by Ms. Shi-
Ping Lan. The analysis and presentation indicated a high degree
of statistical competence and good collaboration among Dr. Shy,
Ms, Lan, and Dr. Hasselblad of the Health Effects Research
Laboratory. The material presented will presumably be in a form
for publication soon. A principal feature of the new analysis
is more adequate use of the symptoms data from the health
survey, employing a 5-level symptom scale rather than the
dichotomy used in earlier analyses.
56
-------
The Information that can optimally be obtained from this
Southeastern study is limited, however, because any possible
effect of air pollution on the measured heilth indices is lower
by factors of 10 to 100 than effects of smoking or job
exposure. Even though a pollution (intercity) association is
found, it remains possible that this association is not causal
but is due to a variable related to the stronger effects of
smoking or job exposure or to other confounding variables for
which no observations are available.
While the acronym CHESS is understood to apply to the 1970
to 1975 group of studies, certain new work in progress
follows the general outline of that program. The study most
clearly conforming to that design is in four Utah communities,
in which 1976 observations are being compared with former 1970
CHESS observations of chronic respiratory disease and of acute
lower respiratory tract disease, as related to increasing S02
pollution in the region.
A substantial change in the operation of CHESS and related
studies has been wade in the past three years with a change in
emphasis from in-house research to research grants and
contracts. This appears to be a result, in part» of the
extensive criticism of the previous CHESS program and is
reflected in the entire activity of the Epidemiology Division,
Only four professional researchers from a previous epidemiology
staff of 15 remain in that division. Three new, young junior
investigators have recently joined the division. The reduced
staff is essentially completely occupied with their duties as
project officers on contracts and grants. The result of this
change from intramural to extramural with regard to CHESS
appears not to be obstructive and may offer certain advantages.
C. Steps Taken by EPA toMeet Brown Committee
Recommendatioris" —
Public Law 95-155, passed by the 95th Congress, mandated a
review of and a report on "the findings and Recommendations of
the report to the House Committee on Science and Technology
entitled "The Environmental Protection Agency's Research
Program with Primary Emphasis on the Community Health and
Environmental Surveillance System (CHESS); An Investigative
Report.'" It was further specified that special attention be
focused on "procedural safeguards required to preserve
scientific integrity of such research and to insure the
reporting and use of such research in subsequent recom-
mendations."
57
-------
Although Chairman Brown emphasized the desirability of a
positive attitude in the letter of transmittal of the Committee
Report, the document impressed some members of the subgroup as
often being hypercritical and demanding an approximation to
perfection that is not obtainable in studies of human
populations. The EPA his published a response to the recom-
mendations of the Investigative Committee in the EPA Research
Outlook of Mirch 1978. The report pf this subgroup will
address only those recommendations that deal with on-going
activities related to CHESS or other epidemiologies! and bio-
statistical work at HERL/RTP. Recommendations will be ident-
ified by the numbers used in the Investigative Report and in
the Agency's response.
3{a): EPA should publish an announcement regarding the
limitations of the CHESS Monograph.
3{c): EPA should publish an addendum to the CHESS
Monograph including most of the Investigative Report.
Subgroup findings: It is believed that the EPA response
covers these recommendations sitisfactorily, although it is
difficult to see how the response can be delivered to all
holders of the CHESS Monograph. Most scientists, however, will
be aware of the limitations of the data in this Monograph.
4(a): Legislation should be reexamined regarding
unrealistic procedures and schedules.
Subgroup findings; The legislative mandate for a study of
air pollution and its effects on the Gulf Coast (Houston)-area
appears to require an unreasonably rapid approach to a very
complex problem. The epidemiology group expressed an interest
in investigating this situation in a systematic, planned
fashion. They doubted that the mandated crash approach would
be maximally productive but stated their intent to obtain as
much valid data as possible. It is not known to what extent
this legislative mandate was reexamined. No evidence was found
at this level to indicate that reexaminatiort was effective in
producing any important changes. Current procedures referred
to in the Agency's response in the EPA Research Outlook do not
appear to be adequate to solve problems caused by unrealistic
legislative mandates.
4{d): EPA should advise Congress if budgetary restric-
tions will impact completion of major projects.
Subgroup findings: Budget restrictions forced the
statistical unit at HERL to discontinue 'Other work to "clean"
the data tapes for continued CHESS analyses. The response of
the Administration and of Congress to this restriction is not
known. While it did not affect CHESS, it must have had an
adverse effect on other programs.
58
-------
5: OMB should be asked to develop procedures for prompt
review of questionnaires.
Subgroup findings: The Population Studies Division has
found OMB responsive to their need for quick approval of
questionnaires. The subgroup supports the EPA position that
its questionnaires for volunteers in research projects should
not require submission to OMB.
6(a): CHESS data analyses should be carried out only on
data with high validity potential.
Subgroup findings: Dr. Shy's group at the University of
North Carolina and the epidemiologists and statisticians at
HERL have reviewed the CHESS data and have decided which data
sets warrant analysis for publication,
6(b): EPA should publish research in refereed journals in
a timely fashion.
6(c): EPA should not publish large projects solely in
monograph form.
6(d): EPA should not initiate projects for policy
consideration unless they can be completed in a realistic time
frame.
Subgroup findings; Staff indicated their desire to see
results published in scientific peer reviewed journals but
emphasized their lack of time to do or report their research or
the findings of contractors. It is reasonable to assume,
however, that most grant recipients and contractors will •'
publish their findings in appropriate journals. It should be
noted, however, that a document entitled "CHESS Bibliography,
December 1, 1977" lists, for the period 1/75 to 12/77, only one
journal article, seven government publications, and ten EPA in-
house publications, plus three more in-house publications that
are undated but whose authors or titles suggest that they
belong in this time period. For 1977,, the bibliography lists
only one government publication, which must have been planned
well in advance of the Brown Committee report.
It seems unlikely that the EPA responses to this
recommendation can be properly assessed until the epi derniol ogic
staff is increased to a size more commensurate with its duties.
7(a); EPA should strengthen the CHAMP aerometric and
quality control programs.
7{b); EPA should shorten the time between data acquisi-
tion and quality assurance analysis of data.
7(c): EPA should stop employing development stage
instruments before qualification testing,
7(d): EPA should not use laboratory models of instruments
in the field until they have been field checked and operating
personnel trained.
59
-------
7(e): EPA should reevaluate the opening of the CHAMP
operations contract to competition.
Subgroup findings:. CHAMP is no longer at HERL. We were
informed that it no longer exists as an identifiable unit
separate from other monitoring activities*
?(f): EPA research and monitoring personnel should
closely coordinate regarding chemical species.
Subgroup findings: Coordination of CHAMP with health
effects personnel is now potentially more difficult because of
the transfer of the responsibilities of CHAMP to another
laboratory. It is still too early to tell whether the transfer
will help by strengthening this type of monitoring activity or
will hinder the accomplishment of the Agency's mission by
impeding coordination.
10{a): An interdisciplinary task force should draw up an
integrated air epidemiology exposure assessment program plan
for EPA.
Subgroup findings: There is a desire for an advisory
group not only to meet this recommendation for assessing health
effects of air pollution but also to provide consultation for
other epidemiologic studies, both intra- and extramural,
10(c): EPA should have epidemiological questionnaires and
panel selection criteria approved by peer groups.
Subgroup findings: Aside from a comparison of self-
administered versus intervlewer-administered questionnaires,
the wo.rk related to this recommendation is limited to the
information that can be gathered from the extensive analyses of
CHESS data being carried out by Dr. Shy. The panel data are not
scheduled for analysis.
Planning for a second round of CHESS or for investigation
of air pollution "episodes" was not mentioned. It is difficult
to see how very much can be done along this line with the
limited staff. It seems reasonable to delay planning for a
second round of CHESS until the current analyses are completed.
60
-------
-------
APPENDIX A
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
JUL121978
THE ADMINISTRATOR
TO:
THRU:
StB JECT i
Dr. Endl M. Mrak
Chairman
Executive committee, Science Advisory Board
Dr. Richard M. Dowd
Charge to the Science Advisory Board's Health 'Effects
Research Heview Group
The Authorization Act of 1§78 for Research and Development, PL 95-155,
requires that a special evaluation report on the Agency's health effects
research efforts be prepared by the Science, Advisory Board (SAB)* The Act
specifically outlines what'is expected to be included in the report regarding
your assessment of our health effects research programs, and the procedures
for the conduct, .review, reporting and, use of such research.
To delineate the .Congress's charge more sharply, 'I urge the Study Group
to define health effects research to include all planned activities, col-
lection and analyses of data done within _the agency for the purpose of
adding- to the scientific basis for understanding the effects of environmental
factors on human health. This definition would include those activities
within the Agency which may be used to assess human riskf and which support
standard setting and regulatory decisions, and any activity which gathers
new knowledge about human health, or improves our understanding of human
health either directly or which can be used to extrapolate to hwnan health
impacts. I am happy to hear that Dr. James Whlttenberf^r and Dr.
McClellan will chair and co-chair this review groupf ',
I can assure you that your assessment of the Agency's
within the scope of this definition will be appreciated and that
have our full cooperation in this endeavor.
Doug
A-l
>u will
-------
_1«_
PUBLIC LAW 95-15S—NOV. 8, 197T 91 STAT. 125?
Public Law 95-155
95th Congress
An Act
To authorise ai>i>K>iirlutltfi)8 for m'tlvlllM of UK; KnvlrotimcMlul I'ratiicUoii .. rov- ff» 197?
As«ngy, mid for other purpoatB, (H.ft, S101J
Re «'£ unacted by the K&nats end House o/ IteprtiwntatiiHig of the
ed States of Aiwricu. in Cenyreim a,w«i&/?8", Pfjelopme
SRC;. 2. (a) Then; aro authorized to he appropriated to the Environ- 1"
mental Protection Agency for environmental research, development, AuihortwrtioiTAet
and demonstration n,H ivitn-s for fiscal war 1078—- Of 1973.
(1) $9^,500,000 for water f[iiaTity activities authorised wider
the Federal Water Pollution Cond-u) Act of which— 33 USC 1251
(A) $£1,200,000 is for th« Health and Ecological Effects note-
(B) $0,300,000 is for the Industrial Processes program;
(C) $6j()0!>,00y Is for thfcMoniloringand Tci'hnfoa! Support
(D) $^2,300,000 is for the Public Sector Activities pro-
gram ; and
(K) $ti,f»,G31,000 is for the Energy program.
(2) $10,800,000 for activities authorixcd under the Fcdoral
Insecticide, Fvmgiuide, and Kodvntiddc Act, in the Health and ? USC 136 note.
Ecological KlfcvtK program.
(3) $lfi,0f>0j000 for water supply activities authorized under
" Sa" ' " •"
safe Drinking Water Act, in the Public Sector program. *2 USC 300f
(4) $8,200,000 for toxic sulxitancc control activities anthorixinl Bot*-
under the Toxic Substances Control Act, in the Health and Eeo- 1S USC 2W1
logicftl Ejects progrmiL n |