UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
November 9, 1993
SCIENCE ADVISORY BOARD
EPA-SAB-EEAC-CQM-94-003
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, S.W.
Washington, DC 20460
Subject: Commentary on Peer Review of Research Used fn Support of
Environmental Policy
Dear Ms. Browner:
Since the Environmental Economics Advisory Committee (EEAC) is a relatively
new Committee within the Agency's Science Advisory Board, we felt that this was an
appropriate time to reflect on our activities during our first two years; as a result, we
have developed some general recommendations regarding the role of peer review of
scientific research conducted in support of environmental policy-making.
Based on our Members' experience both prior to joining this Committee and in
the tasks we have undertaken as part of its activities, it is clear to us that the Agency's
staff is attempting to develop and use research methods and findings at the frontiers of
their disciplines in the areas relevant to EPA's regulatory mission. These efforts are to
be commended and encouraged. They reflect well on the professional training of the
staff and the incentives provided to them. Although our direct experience has been
primarily related to the research in environmental economics, we believe the same
experience is shared in a number of other areas where EPA staff plays a significant
role in the definition and development of research.
Because the research is often directed at the frontiers of each discipline's
scientific understanding, it is particularly important to incorporate peer review as an
integral part of the development of the research design. In these cases, it is essential
that the design, implementation, and final results of significant research initiatives be
subjected to external peer review. Clearly, the need for such comprehensive involve-
ment of peer review at all stages in research design and execution will depend upon
the scale of the research undertaken. For modest efforts, review at the outset and then
Prbud M pip* INH eanaira
M Mat 75% "*5><**J a**
-------
-------
again prior to the use of final results would offer a prudent approach that recognizes the
needs for evaluation and the scarcity of resources available for reviewing. As the size
and complexity of the effort (as well as Its importance for policy) increases, so also
should the resources devoted to peer review.
Our experience with the economics components of the research efforts we have
reviewed has found only limited evidence of systematic external peer review conducted
prior to the time when documents were presented to the Committee for evaluation (This
is apparently not a recent problem - the Environmental Engineering Committee
commented on the need to organize peer review efforts in a 1989 report on mathe-
matical models [EPA-SAB-EEC-89-012]). We believe that this lack of peer review is a
mistake, and understand that it is not consistent with Agency policy on peer review
which calls for such review as an integral part of the research activities associated with
EPA's regulatory policy. The resulting rfe facfo assignment of the primary peer review
to the SAB's Committees typically imposes that review at the wrong time in the process
— when the research is largely done. Equally important, the areas of research involved
can relate to very specific "niches" within a discipline or sub-discipline where Committee
members may not have the required expertise,
ideally, external peer reviews (other than those performed by the SAB) would be
conducted on an ongoing basis as research for large projects is underway, and the
results of all such reviews made available to the relevant SAB Committee at the time a
more comprehensive review of a research program or policy evaluation was undertak-
en. EPA staff should develop a network of external peer reviewers in topic areas where
there will be continuing research interests, This could enable these reviews to be
conducted prior to submitting materials to SAB Committees, In cases where SAB
involvement at other than the final stage is desirable, Agency program officials should
seek Consultations, through which the SAB can provide discussion and advice from
various individual Members directly to the appropriate staff, outside the formal frame-
work of a full review. The use of a Consultation in no way precludes a full review and a
formal SAB report at a later stage of the effort.
A number of Members of the Committee have had past experience in designing
and implementing peer review programs for proposed and on-going research activities.
The Committee would be happy to advise Agency staff in developing an appropriate.
external peer review procedure for EPA research activities that is consistent with
-------
-------
completing these reviews prior to submitting materials to the EEAC for its evaluation
and input
We look forward to receiving your response to our comments.
Sincerely,
Div Raymond C. Loehr, Chair
Science Advisory Board
Dr. Allen Kneese, Co-Chair Dr. V, Kterr^rSmitn, Co-Chair
Environmental Economics Environmental Economics
Advisory Committee Advisory Committee
ENCLOSURES
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOARD
ENVIRONMENTAL ECONOMICS ADVISORY COMMITTEE
CO-CHAIRS
Dr. Alton V. Kncese, Resources for the Future, Washington DC
Dr. V. Kerry Smith
Department of Economics, North Carolina State University, Raleigh NC
MEMBERS
Dr. Nancy E. Bockstael, Department of Agricultural and Resource Economics, University of
Maryland, College Park, MD
Dr. A, Myrick Freeman, Department of Economics, Bowdoin College, Brunswick, ME
Dr. Charles D. Kolstad, University of Illinois, Urbana, It
Dr. William Nordhaus, Dept, of Economics, Yale University, New Haven CT
Dr. Bryan Norton. School of Public Policy, Georgia Institute of Technology, Atlanta GA
Dr. Wallace E. Gates, Department of Economics, University of Maryland, College Park, MD
Dr. Paul R. Portney, Resources for the Future, Washington, DC
Dr. Robert Repetto, World Resources Institute, Washington, DC
Dr. Richard Schmatensee, Massachusetts Institute of Technology, Cambridge MA
Dr. Robert N, StavJns, Harvard University, Cambridge, MA
Dr. Thomas H. Tfetenberg, Dept. of Economics, Colby College, Waterville, ME
Dr. W. Kip Viscusi, Department of Economics, Duke University, Durham, NC
SAB COMMITTEE UAISONS
Dr. William Cooper (EPEC), University of Michigan, Anne Arbor, Ml
Mr. Richard Conway (EEC), Union Carbide Corporation, Charleston, WV
Dr. Morton Lippmann (1AQC), N.Y, University, New York City, NY
Or, Roger McClellan (CASAC), Chemical Industry Institute of Toxicology, Research Triangle
Park, NC
DESIGNATED FEDERAL OFFICER
Mr. Samuel Rondberg, Science Advisory Board (1400F), U.S. Environmental Protection Agency,
Washington, DC 20460
STAFF SECRETARY
Ms. Mary L Winston, Environmental Protection Agency, Science Advisory Board (1400F),
Washington, DC 20460
-------
-------
NOTICE
This report has been written as a part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related
to problems facing the Agency, This report has not been reviewed for approval by the
Agency and,-hence, the contents of this report do not necessarily represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or
commercial products constitute a recommendation for use.
-------
-------
Distribution List
Administrator
Deputy Administrator
Assistant Administrators
Deputy Assistant Administrator for Research and Development
Deputy Assistant Administrator for Water
EPA Regional Administrators
EPA Laboratory Directors
EPA Headquarters Library
EPA Regional Libraries
EPA Laboratory Libraries
-------
------- |