f/EPA
United States
Environmental Protection
Agency
Office of Exploratory
Research
Washington DC 20460
Research and Development
EPA/600/9-85/004 Jan 1985
Orientation
Handbook for
Members of
Research Grants
Science Review
Panels
HA NDBOOK
FOR
r E D E R
ASSISTANC E
APPLICATION
REVIEW
-------
Contents
Introduction
Science Review Panels
Structure/Membership ........ . ................. 1
Function .................................... 2
Review Process ............................... 2
Assignments ........................ 2
Primary Reviewers' Written Evaluation ............................... 2
Meetings ..................................... 3
Recommendations by Reviewers and Panel ........................... 3
Site Visits ................................................. ... 4
Research Material ................................... 4
Other Panel Duties ............................... 4
Review-Related Issues .............................................. 4
Conflict of Interest ....................................... 4
Confidentiality ........................................ 5
Privacy Act [[[ 5
Panelist Expenses ..... ............................................ 5
Appendixes
Guidelines for Reviewers' Written Evaluation ........................... 6
Research Materials .................................. ,7
No Conflict of Interest Form .......................................... 10
-------
introduction
The mission of the U.S. Environmental Protection Agency (EPA) is to
protect the Nation's land, air, and water systems. The Office of Research and
Development {ORD) is the principal scientific component of the Agency with
the primary function to obtain credible scientific information on which to
base sound national policy for pollution control.
The Research Grants Program of the Office of Exploratory Research is the
unit within ORD that has the primary responsibility for all research grant
applications received by EPA, including their scientific evaluation and the
subsequent administration and monitoring of those awarded grants. The
research grants supported are investigator-initiated and deal with basic or
fundamental research projects.
This Handbook describes the structure, functions, and operational
procedures of the Science Review Panels used by the Agency for the peer
review of research grant applications.
Science Review Panels
Structure/Membership
Science Review Panels are established for each of the following principal
areas of interest to the research program of the Agency:
• Environmental Biology
• Environmental Chemistry, Physics, and Measurements—Air
• Environmental Chemistry, Physics, and Measurements—Water
• Environmental Engineering
• Environmental Health.
Each Science Review Panel is operated through a cooperative agreement
between the Agency and a non-EPA employee who serves as the Chair of
the Panel. The Chair is selected for a period of three years by the Assistant
Administrator for Research and Development. The Chair is responsible for
conducting the scientific review of applications assigned to the Panel, and
for providing the Agency with a Summary Statement of the review and
recommendations made by the Panel for each application.
Each Panel is composed primarily of non-EPA scientists who are
acknowledged experts in their respective disciplines. The invitation to
become a member of a Panel is extended by the Panel Chair with the
approval of the EPA. Additionally, every effort is made to have representative
geographic distribution and minority group representation on the Panels'
memberships.
A Science Review Administrator (SRA), who is a Research Grants
Program staff person, is an ex-officro member of a Panel. The SRA provides
-------
policy and procedural guidance, and assists the Chair in the preparation and
conduct of the Panel meetings. Because of the diversity of the applications
received and the disciplines required for their scientific review, in some
cases a Panel may establish sub-Panels. The sub-Panels are organized
around categories of identifiable areas of research.
Function
It is the responsibility of each Panel to review and evaluate the scientific
quality of the applications assigned to it, and to provide the Agency with a
critique regarding the scientific merit along with a recommendation of an
appropriate funding level for each application reviewed. The Panels focus
attention on the scientific merits of proposals.
Review Process
Assignments
Grant applications are received by the Agency's Grants Administration
Division where they are reviewed to determine their legal and administrative
acceptability. The applications are forwarded to the Research Grants
Program for appropriate Panel assignment.
The Panel Chair, who may be assisted by the SRA, makes primary review
assignments to appropriate Panel members. Each application is assigned to
at least three primary reviewers who independently are responsible for
providing a scientific review and evaluation. If a reviewer believes an
application has been inappropriately assigned the Chair should be
immediately notified. Each primary reviewer receives from the SRA a full
copy of each application assigned, along with any appropriate background
material. Every effort is made to limit the number of primary assignments to
5 per Panelist, with an interval of no less than 5 weeks between the time of
assignment and the Panel meeting. In addition, copies of the cover sheet
and abstracts for alt applications to be considered by the Panel are provided
to each member before the Panel meeting. A complete copy of any
application not assigned to a Panelist may be requested from the SRA.
Complete copies of all applications to be reviewed are available at the Panel
meeting.
Before making the primary review assignments, the Panel Chair or SRA
will contact each Panelist regarding their availability at the forthcoming
scheduled review meeting. It is imperative for the Panelists to respond
promptly so that assignments can be properly and efficiently made. A
Panelist may not be assigned applications for a given meeting because
either their services are not required due to the mix and/or number of
applications to be reviewed, or the Panelist has an application scheduled for
review at that meeting. In the case of the latter, the Research Grants
Program policy prohibits the participation of a Panelist at a meeting at which
their application is to be reviewed.
Primary Reviewers' Written Evaluation
The primary reviewers' evaluations are to be prepared in writing before
the scheduled Panel meeting. These evaluations are to follow the Guidelines
-------
for Reviewers' Written Evaluation (Appendix A) and include the following
criteria:
• Quality of research plan (including theoretical and/or experimental
design, originatity, and creativity)
• Qualifications of principal investigator and staff including knowledge of
subject area
• Potential contribution to scientific knowledge
• Availability and adequacy of facilities and equipment
• Budgetary justification.
The written reviews along with discussion notes from the Panel meeting
serve as the basis for the preparation of the Summary Statement.
Meetings
Panels ordinarily meet for application reviews two or three times each
fiscal year, usually for 2 days at a time. The frequency depends upon the
number of applications and availability of grant funds. Meeting locations are
selected by the Panel Chair with the approval of the SRA and are generally
determined during a Panel meeting executive/administrative session. At
least one meeting each fiscal year is at the location of an EPA research
laboratory considered appropriate to a given Panel.
The SRA prepares a list of the applications to be reviewed for each Panel
meeting. The review of each application begins with the evaluations and
recommendations of the primary reviewers. This is followed by Panel
discussion until, by consensus, there is sufficient information from which a
considered scientific appraisal and recommendation for funding can be
made.
Recommendations by Reviewers and Panel
The reviewers, and ultimately the Panel, will recommend a research grant
application for approval, disapproval, or deferral based on the scientific merit
of the proposal.
Approval: Based on the review criteria, the proposal has sufficient merit to
be worthy of support. (A recommendation of approval indicates that an
award be made provided that sufficient funds are available.) The
recommendation can be for the time and amount requested or for an
adjusted time and/or amount. A numerical priority score is required.
Disapproval: The application does not have sufficient merit to be worthy of
support. No priority score is required.
Deferral: This recommendation may be made only if sufficient information
is not available from the review to arrive at a satisfactory consensus of
approval or disapproval. Deferral applications are reviewed again at the next
Panel meeting after the additional information is obtained either by
submission from the applicant or by a site visit. A deferral recommendation
will not be used as a mechanism to allow the applicant to improve or
redirect the proposal.
When a consensus is reached with two or more Panelists not in
agreement, a "minority report" must be prepared, identified, and included in
-------
the Summary Statement. After a consensus has been reached on an
application, each panelist independently and privately records a numerical
priority score that reflects the level of scientific merit of the application. The
Panels use either a 100 point or 10 point system, with 0 the least
meritorious and 100, or 10 the most meritorious.
Site Visits
A project site visit is recommended only when additional information is
needed that cannot be communicated satisfactorily through correspondence
or by telephone. Usually such information relates to specific details about
the scientific aspects of the proposal; the arrangements for collaboration-
the progress during a previous project period; or the departmental or
institutional interest, commitment, support, or plans in relation to the
personnel and the project. In some cases, the Panel may need to observe an
experimental technique or methodology or assess the appropriateness of
special equipment and procedures that are essential to the conduct of the
proposed research. The site visit team customarily includes a small number
of Panelists and an SRA. In some cases, the team may include ad hoc
consultants with special competence in some unique, critical aspects of the
proposed investigation. A site visit report is prepared by the site team Chair
and submitted to the SRA and Panel's Chair for discussion at the next
review meeting.
Research Material
inV°IVeS the USe °f human subjects or material or data
anS' laboratorV animals- a^/or the use of hazardous
« m tCh fl recombinant DNA> ^quires special attention. Such
conformTt th8 **> °M in comPliance wi'h EPA policy which in general
conforms to the policy and guidelines established by NIH. (See Appendix B.)
Other Panel Duties
reviewir)9 9rant ^Plications, Panels may be asked to survey
MSK3tUS °f reSearch in their fields and to ^entify research
H ? b6 initiated' ^^thened, or expanded. Panel
«h J°nS TV be S0ught to identifV neglected areas of research,
research needs such as critical shortages of research resources, and ways
to improve the quality of review.
Review-Related Issues
Conflict of Interest
a Plnel.0r a Cl°Se relative' friend- or close Professional
P T™*" '* n*m** M 8n instigator in a grant
ofhe aat Pane' me,mber wi» n^ be involved in or privy to the review
Pane! mpmh?«l ^ belfXCU8ed from ^ review of such applications.
submitted hvth ° Wl" beabsent from ^e meeting when applications
submitted by their own institution are being discussed.
-------
It is often difficult for the Chair or SRA to determine whether there is a
conflict of interest that requires a Panel member to be excused from a
particular review. Therefore, it is the responsibility of the member to notify
the Chair or the SRA in advance. Each Panelist will be required to submit a
signed No Conflict of Interest Form (Appendix C) at each meeting,
Confidentiality
Grant applications, review materials, and proceedings of Panel meetings
are considered confidential by Panelists and EPA staff. All applications and
review materials will be left with the SRA at the conclusion of a review
meeting or a site visit.
It is particularly important that applicants receive no premature
information regarding the status of their proposals. Such disclosures can
result in misinterpretation and represent a serious abuse of the privileged
nature of the peer review procedure. The protection of the confidentiality of
review proceedings is essential to maintaining a peer review system that
will enable EPA to allocate public funds for research effectively and
impartially.
THERE SHOULD BE NO DIRECT COMMUNICATIONS CONCERNING AN
APPLICATION BETWEEN MEMBERS OF THE PANEL AND APPLICANTS. ALL
COMMUNICATIONS OR INQUIRIES REGARDING THE STATUS OF AN
APPLICATION SHOULD BE REFERRED TO THE SRA.
Panel members are also bound by the confidentiality of all proprietary
parts of the proposals they read under this guidance; no Panelists will
disclose or use to their own advantage any data, concept, research protocol,
or any other idea included in the applications.
Privacy Act
Under the Privacy Act, applicants may have access, upon request, to
documents generated during the review of their grant applications. Such
documents include site visit reports. Summary Statements, and reviewers
written comments, if available. Reviewers' written comments and scores,
however, are not retained after their substance has been incorporated into
Summary Statements or site visit reports.
Panelist Expenses
The Chair's cooperative agreement for the research grant application
review provides for a budget from which Panelists are reimbursed for
expenses and receive a professional fee. Allowable expenses are
professional fees round trip air or rail transportation, ground transportation,
and per diem costs, which can vary according to location. A daily consultant
fee of $200 is paid to each non-Government Panelist for each day or fraction
of a day spent on a site visit or at a meeting of the Panel.
Panelists make their own transportation arrangements and purchase their
own tickets. Travel should be by the most direct route. When air
transportation is used, travelers are expected to use less than first-class
accommodations, although use of first-class accommodations may be
necessary under certain circumstances. In cases where reimbursement is
claimed for first-class flights, written justification will be required.
-------
Reimbursement may not be paid for additional costs resulting from indirect
routing or stopovers if made for reasons not related to the Panel review trip.
Such additional costs should be waived or explained if reimbursement is
claimed. Dual compensation from the Government cannot be paid for a trip.
If a Panelist provides service for some other activity during the same trip, the
charges should be prorated between the two organizations in an appropriate
manner, giving the name of the other agency in cases of co-sponsored trips.
Reimbursement forms (Appendix D) are distributed by the SRA during the
Panel meetings or site visits. Social Security numbers and home addresses
should be included in the spaces provided to ensure prompt reimbursement.
The completed reimbursement forms should be mailed to the Chair, not the
SRA.
Appendix A
Guidelines for Reviewers' Written Evaluation
Description of Proposed Research
Give a brief statement of objectives, rationale, experimental methods, and
expected results.
Critique
Identify strengths and weaknesses in the experimental design. Comment
on the originality and creativity of the proposed research, appropriateness
and adequacy of the research materials, and the methods to be used. Be as
specific and candid as necessary in your assessment.
Investigator(s)
Comment on principal investigator(s) research training, knowledge of
pertinent literature, experience, and competence; the probability of
completing the work in a competent and timely fashion; the potential of
younger scientists to develop an adequate understanding of the problem
area in which the work is directed.
Facilities/Equipment
Comment on the availability and/or adequacy of the facilities and
equipment proposed for the project. Identify any deficiencies that may
interfere with the successful completion of the research.
Budget
Comment on the appropriateness of the budget relative to particular items
(e.g., personnel, equipment, travel, etc.). Data on recommended budget
adjustments should be enumerated by year and should be as detailed as is
practicable.
Recommendation
Make recommendation on the merit of the proposal for EPA funding
support (approval, disapproval, or deferral).
-------
Appendix B
Research Materials
The use of certain research materials is subject to satisfactory compliance
with relevant Federal regulations, policies, or guidelines. The research
materials of concern are: (1) Human Subjects, Derived Material and Data; (2)
Laboratory Animals; and (3) Hazardous Research Material and Methods
(including recombinant DNA). EPA generally conforms to the regulations,
policies and/or guidelines established by the Department of Health and
Human Services (DHHS) for the proposed use of these research materials.
Although the grant recipient institution and investigator bear the primary
responsibility for the proper use of these research materials, EPA staff and
Science Review Panel members must share this responsibility. Proposals
should be reviewed as to the adequacy of the applicants' capabilities,
resources, and proposed methods for use of these materials and
conformance with applicable law and policy. Any concerns or questions of
Panel members regarding the proposed use of these research materials
must be provided in the "critique" of the Summary Statement. No awards
may be made unless EPA has reasonable assurance that the applicant will
be in compliance with EPA policy and guidelines and all concerns or
questions raised by the Panel have been resolved to the satisfaction of the
Research Grants Staff (RGS).
As noted above, EPA conforms to DHHS guidelines for the proposed use of
these research materials. Some paraphrased excerpts pertaining to DHHS
policy regarding the use of these materials are provided below.
1. Human Subjects, Derived Material, and Data
Applicants planning or contemplating studies that involve human
subjects or derived materials or data that contain personal identifiers
or can be linked to personal identifiers are required to submit Form
HSS 596 to ensure appropriate institutional review, certification, and
compliance with DHHS policy. This policy is based on DHHS
Regulation 45 CFR 46, which is available from the Office of Protection
from Research Risks, National Institutes of Health, Bethesda, Maryland
20205.
This.regulation provides a systematic means, based on established
ethical principles for protecting the rights and welfare of individuals
who may be exposed to the possibility of physical, psychological, or
social injury while they are participating as subjects in research,
development, or related activities. The regulation extends to the
human fetus (either in utero or ex utero), the dead, organs, tissues,
and body fluids, and graphic, written, or recorded information derived
from human sources. It covers activities that present no physical risk
to the subjects, but which may create legal risks or exposed subjects to
public embarrassment or humiliation through breach of confidentiality
or invasion of privacy. The major focus of a project (for example, on
-------
medical procedure) may not be the sole determinant of the types of
risks involved or the need for additional protection.
The safeguarding and confidentiality of medical records and other
forms of data collected on individuals and groups, the use of such data
either by the principal investigator/program director conducting the
original research, concurrent uses of the data by other investigators,
and the use of the data for research purposes at a later time are
considered within the scope of this policy. The regulation requires
institutional assurances, including the implementation of procedures
for review, and the assignment of responsibilities for adequately
protecting the rights and welfare of human subjects. Safeguarding
these rights and welfare is, by DHHS policy, primarily the responsibility
of the applicant organization. However, institutional review boards,
EPA staff, and Science Review Panels share the responsibility for
determining the adequacy of the proposed procedures. In particular,
the application organization is responsible for ensuring that the activity
described in the application and additional information relating to
human subject's derived material or data are reviewed and approved
by an institutional review board.
2. Laboratory Animals
EPA policy requires that laboratory animals not suffer unnecessary
discomfort, pain, or injury. Applicant organizations using animals in
project supported with funds from EPA grants are responsible for the
proper care and humane treatment of animals. Each organization must
assure the EPA in writing that it is committed to following the
standards established by the animal welfare acts and by the docuents
entitled "Principles for Use of Animals and Guide for the Care" and
"Use of Laboratory Animals," which are available from the Office of
Protection from Research Risks (OPRR), National Institutes of Health,
Bethesda, Maryland 20205.
The general intent of these principles can be summarized as follows
in two broad rules:
• The project should be worthwhile and justified on the basis of
anticipated results for the good of society and the contribution to
knowledge, and the work should be planned and performed by
qualified scientists.
• Animals should be confined, restrained, transported, cared for, and
used in experimental procedures in a manner that avoids any
unnecessary discomfort, pain, or injury.
3. Hazardous Research Materials and Methods
The investigator and applicant institution bear primary responsibility
for protecting the environment and research personnel from potential
damage associated with the use of hazardous research materials such
as carcinogenic and infectious agents, radioactive and explosive or
flammable materials, and recombinant DNA molecules. When such
hazardous materials are to be used in a proposed study, the panel
should evaluate the applicant's capability for the safe handling of such
materials and their compliance with Federal policy and guidelines.
8
-------
In recent years there has been particular concern about the safe
research use of recombinant DNA molecules. Proposed studies that
involve recombinant DNA must comply with the current NIH
Guidelines for Research Involving Recombinant DNA Molecules (The
Administrative Practices Supplement and announcements of
modifications and changes to the Guidelines are available from the
Office of Recombinant DNA Activities (ORDA). National Institutes of
Health, Bethesda, Maryland 20205). All research involving
recombinant DNA techniques that is supported by EPA must meet the
requirements of these NIH Guidelines, which were developed in
response to the concerns of the scientific and lay communities about
the possible effects of recombinant DNA research. Their purpose is to
specify practices for the construction and handling of recombinant
DNA molecules and organisms or viruses containing recombinant
DNA. As defined by the Guidelines, "recombinant DNA" corresponds
to: (1) molecules that are constructed outside living cells by joining
natural or synthetic DNA segments to DNA molecules that can
replicate in a living cell, or (2) DNA molecules that result from the
replication of a molecule described in (1).
Several types of studies involving recombinant DNA are exempt from
the Guidelines; others are prohibited by the Guidelines. For the
remainder, the applicant organization must establish and implement
policies that provide for the safe conduct of the research in full
conformity with the Guidelines. This responsibility includes
establishing an institutional biosafety committee to review all
recombinant DNA research to be conducted at or sponsored by the
applicant organization and to approve those projects that are in
conformity with the Guidelines.
-------
Appendix C
No Conflict of Interest Form
Reviewers must be excused from participation in the review of a proposal
or a review session when there is any potential or possibility of a conflict of
interest. A conflict of interest exists when the reviewer has any financial or
personal interest in a proposal being considered by his/her review panel.
Situations that may constitute a conflict of interest include, but are not
limited to, the following:
• The application being considered is from the reviewer's parent institution
or one legally a part of that institution.
• The reviewer has some interest, proprietary or otherwise, in the
application.
• The reviewer and the application have been related as student or
thesis/postdoctoral advisor.
• The reviewer and applicant are known to be close friends or are related.
• The reviewer and applicant have collaborated recently on a research
project.
• The reviewer would benefit directly from the project, e.g., as a consultant
or paid collaborator.
When, in the opinion of the reviewer, the Science Review Administrator,
or the Panel Chair, the appearance or actuality of a conflict of interest
emerges, reviewers must be excused from the review panel meeting for as
long as is necessary to avoid such circumstances.
I certify that I have read the above statement and that no conflict of
interest exists with regard to my review and discussion of EPA assistance
applications at this review panel meeting. In the event that a conflict
situation should arise, I will notify the Panel Chair accordingly.
Signature:
Date:
10
-------
Appendix D
Panelist Reimbursement Form
Statement of Travel
T/0 #: Date:
Itinerary
Date/Time Residential:
Departure:
Return:
Name
SSN
Date
(Day)
Mailing Address
Transportation
Location
LV
AR
LV
AR
LV
AR
LV
AR
LV
AR
LV
AR
LV
AR
LV
AR
LV
AR
Time
Flight #or
Odometer Reading
Form Continued on Following Page
-------
Food and 1
Breakfast
Lunch
Dinner
Lodging
Breakfast
Lunch
Dinner
Lodging
Breakfast
Lunch
Dinner
Lodging
.odgings
Remarks:
Other Expenses
Taxi, Limo, Tolls, Parking, Rental Car,
etc. Receipts required for each entry
over $25. ITEMIZE
Other— (Explain)
Subtotal
Travel Advance
o t • ,f i r-. Total Claim
Professional F«fi fnr Days
Totals
*USGPO: 1985 — 55 9U11/107 79
12
------- |