Office of Research and Development's (ORD)
May 2007 Response to the Board of Scientific Counselors (BOSC)
October 2006 Final Report that Reviews ORD's Science to
Achieve Results (STAR) and Greater Research Opportunities (GRO)
Fellowship Programs
BOSC STAR/GRO Fellowship Program Subcommittee:
Clifford Duke (Chair), Ecological Society of America
Claudia Sturges Scott (Vice Chair), Consultant
John Adgate, University of Minnesota
Gary Barrett, University of Georgia
Richard Carson, University of California, San Diego
Costel Denson, Costech Technologies, L.L.C.
Nikola Garber, National Oceanic and Atmospheric Administration
Donald Scavia, University of Michigan
Consultant: John Hall, Morehouse College
Submitted By:
Stephanie Willett, Team Leader, STAR/GRO Fellowship Programs
National Center for Environmental Research
Office of Research and Development
-0-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
ORD Response to the Board of Scientific Counselors (BOSC)
Review of the STAR and GRO Fellowship Programs
The Board of Scientific Counselors (BOSC) Science to Achieve Results (STAR)/Greater
Research Opportunities (GRO) Fellowship Subcommittee performed a review of the Office of
Research and Development's (ORD) STAR and GRO Fellowship Programs in 2006. As part of
the review, the subcommittee conducted three conference calls prior to convening a face-to-face
meeting, which was held on March 2-3, 2006, in Washington, D.C. After one subsequent
conference call, the subcommittee finalized their draft report. It was vetted by the BOSC
Executive Committee and submitted as a final report to ORD by the Executive Committee Chair
in October 2006.
Following is a narrative response to the recommendations provided by the BOSC in their
October 2006 report. The BOSC recommendations are listed below in italics under the most
relevant charge question. The ORD response follows each recommendation.
Charge Question 1; The stated purpose of the STAR Graduate Fellowship Program is to
encourage promising students to obtain advanced degrees and pursue careers in an
environmental field, a benefit to both the public and private sectors. Has the STAR Graduate
Fellowship Program produced the desired outcome? Please make recommendations for
enhancing the potential for future positive outcomes.
"Overall, the Subcommittee finds that the fellow s funded by the STAR and GRO programs have made
excellent contributions in environmental science and engineering, and a number of them continue to
be employed in the environmental field in academia, consulting, and government (EPA and other
agencies). Although other federal agencies fund a number of fellow ship programs, none are
dedicated exclusively to the environmental sciences and engineering. Therefore, the EPA programs
are clearly of value to the Agency and the nation in helping to educate the next generation of
environmental scientists and engineers. This conclusion is based on extensive, but primarily
anecdotal, data regarding program outcomes that EPA provided to the Subcommittee. " The
Subcommittee recommends that EPA take several steps to improve its program data collection and
hence its ability to evaluate outcomes:
1A) Develop an overall information collection strategy, which includes design of an appropriate
database.
Response: ORD will adopt recommendation 1A.
The stated goals of the STAR Fellowship Program are: (1) to encourage promising students to
obtain advanced degrees and (2) to pursue careers in an environmental field. With the
introduction of the Fellowship Information Inventory (FII), which was discussed in our March 2,
2006, meeting with the BOSC, ORD has already begun collecting critical pre- and post-
fellowship information on fellowship recipients. This database will be modified to make it more
- 1 -
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
user friendly or will be merged into a larger ORD database. Additional personnel/resources will
be dedicated to data collection, input, and analyses. To measure attainment of the first goal,
ORD considers the number/percentage of fellowship recipients who earn a master's or doctorate
degree to be an appropriate metric. To measure attainment of the second goal, ORD considers
working in an environmentally related career to be an appropriate metric. We will report on
these outcomes every three to five years. Since a significant portion of fellowship recipients may
not receive their degree while they are in the fellowship program, a valid approach to data
collection, analyses, and reporting will need to be adopted. To the extent applicable, ORD will
follow program evaluation guidance produced by the Academic Competitiveness Council to
assess program effectiveness. For additional information on this interagency work group chaired
by the U.S. Department of Education, see
http://www.whitehouse.gov/OMB/memoranda/fy2006/m06-24.pdf
IB) Require fellows to submit an up-to-date resume annually for at least 5 years from the
conclusion of the fellowship.
Response: ORD will adopt recommendation IB, providing it is consistent with all EPA policies
and procedures, and assuming that collecting these data is not cost prohibitive. ORD will either
require that fellowship recipients provide an up-to-date resume or obtain this information another
way.
1C) Require universities with fellowship recipients to provide the basic initial information for
the database at the conclusion of the 2- or 3-year fellow ship period.
Response: ORD cannot adopt recommendation 1C. Since funds are awarded directly to
students, the Universities cannot be obligated to provide this information.
ID) Provide greater resources for the STAR Fellowship Program, both to address the increased
number of applications that this change would produce and to fund a greater percentage of those
applications rated excellent.
Response: In developing its annual budget, EPA seeks to balance its resource needs with those
included in the overall Federal budget and the need to be a good steward of taxpayers' funds.
Given changing economic conditions, growing entitlement spending, and financial commitments
to the President's priorities, the current budget climate is challenging. EPA continues to believe
the STAR Fellowship Program is an important part of its extramural research program.
IE) Consider increasing the graduate fellowship expense budget.
Response: ORD will consider adopting this recommendation, although increasing the fellowship
expense budget, while assuming a fixed budget, will mean fewer fellowships.
-2-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
IF) Consider seeking additional funding within the next few years to maintain the value of the
individual fellowships without resorting to funding fewer fellows.
Response: ORD will devise a Fellowship Program Strategic Plan that will include the feasibility
of collaborative efforts with other federal agencies and other organizations to increase the
resources available for funding fellows. See also ID above.
1G) Adda mentoring component to the program so that there is a specific individual within
EPA who would serve as a resource for the fellow. The mentor could complement the function of
the student's academic advisor by, for example, helping the fellow find opportunities that expand
his or her graduate school experiences and serving as a professional resource regarding career
opportunities in federal agencies. The Subcommittee recommends that ORD solicit mentor
volunteers from throughout EPA, matching professional staff with the fellows' fields of
investigation. This strategy would bring increased visibility to the STAR and GRO Fellowship
programs within the Agency.
Response: ORD will adopt recommendation 1G on a pilot basis. Scientists and managers in the
Research Triangle Park, NC, office have agreed to pilot the development of a mentoring
component, initially with a small group of graduate fellowship recipients. If successful, the
mentoring component will be expanded.
Charge Question 2; The GRO Graduate Fellowship Program, like its predecessor, the MAI
Graduate Fellowship Program, is intended to strengthen the graduate environmental research
capacity of institutions of higher education that receive limited funding to build such capacity,
especially institutions with substantial minority enrollment. Has the GRO Graduate
Fellowship Program fulfilled its purpose? Are there barriers or obstacles that prevent EPA
from fully reaching this goal? Please make recommendations for enhancing the potential for
future positive outcomes.
"Overall, the Subcommittee finds that the fellow s funded by the STAR and GRO programs have made
excellent contributions in environmental science and engineering, and a number of them continue to
be employed in the environmental field in academia, consulting, and government (EPA and other
agencies). Although other federal agencies fund a number of fellow ship programs, none are
dedicated exclusively to the environmental sciences and engineering. Therefore, the EPA programs
are clearly of value to the Agency and the nation in helping to educate the next generation of
environmental scientists and engineers. This conclusion is based on extensive, but primarily
anecdotal, data regarding program outcomes that EPA provided to the Subcommittee. "
2A) Consider eliminating both GRO Fellowship programs, while at the same time improving
marketing of the STAR Fellowship Program to minority-serving institutions to encourage
applications for graduate support from under represented groups.
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Response: ORD will consider recommendation 2A. The GRO Fellowship programs serve a
small but important role in the development of future environmental researchers and practitioners
and collaborative communities, and they contribute to increasing the national research capacity
among institutions that may not otherwise have access to such opportunities and resources.
ORD will explore all policy options currently available to us to help achieve these objectives,
including collaborating with other federal agencies and organizations to maximize outreach to
underrepresented groups.
2B) If the GRO programs are maintained, review the impacts of its 2003 determination that
competition for program awards cannot legally be limited to minority serving institutions.
Response: ORD will initiate a re-assessment of the 2003 determination.
Charge Question 3; The GRO Undergraduate Fellowship Program, like its predecessor the
MAI Undergraduate Program, is intended to strengthen the undergraduate environmental
research capacity of institutions of higher education that receive limited funding to build such
capacity, especially those with substantial minority enrollment. By providing quality
environmental education support for undergraduate students, it was hoped that recipients
would pursue careers in environmentally related fields beyond the baccalaureate level. Has the
GRO Undergraduate Fellowship Program fulfilled its purpose? Are there barriers or obstacles
that prevent EPA from fully reaching its goal? Please make recommendations for enhancing
the potential for future positive outcomes.
3) To provide sufficient resources to make a real impact on capacity building for undergraduate
education, consider devoting the combined resources from both GRO Fellow ship programs to
fund competitively selected regional consortia, which would be designed to focus on
environmental science opportunities for undergraduates.
Response: While ORD will consider recommendation 3, we are concerned that a consortium
could pose a high administrative burden and provide no assurance that our program goals will be
reached. ORD prefers instead to increase its outreach and communication about the GRO
fellowships within regions that have institutions that are most likely to benefit from the
undergraduate program. ORD will also explore collaboration with other federal agencies and
organizations.
Charge Question 4; Please review the fellowship recipient selection process and current
decision criteria. Do they assure selection of high-quality fellowship recipients performing
scientific research in areas that best support the Agency's mission? Please make
recommendations for improvements.
4A) Update the descriptions of the evaluation and selection criteria in the funding
announcements to clearly describe the criteria employed in each step of the review process, to
ensure transparency and fairness for all applicants.
-4-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Response: ORD has adopted recommendation 4A. The 2007 RFA included a more detailed
description of the evaluation and selection process, including more specifics about the EPA
programmatic (internal) review. This information will continue to be communicated and refined
in future RFAs.
4B) Broaden the categories used to sort applications to encompass emerging interdisciplinary
fields of study, and consider making the categories used to sort the STAR Fellowship Program
and the GRO Graduate Fellowship Program applications identical. Such consistency would
allow for continuity and overlap of reviewers in the non-EPA expert panels as well as a clearer
comparison of the two programs.
Response: The STAR and GRO research categories have been identical since 2005, with
reviews for the STAR and GRO fellowship applications being conducted at the same peer review
meeting. ORD reviews the fellowship solicitation research categories yearly. The research
categories are chosen not only to reflect shifting Agency priorities but also to facilitate the
external review process. The research categories in the upcoming 2008 RFA reflect our latest
revisions. The current categories do not preclude emerging interdisciplinary fields; they are
designed to accommodate the review of possible interdisciplinary research fields while also
maintaining a manageable number of review panels. Applicants are asked to identify a single
appropriate category for their proposed research, whether it be single or multidisciplinary,
thereby ensuring an appropriate review.
Charge Question 5; Part of ORD's mission is to perform research and development to identify,
understand, and solve current and future environmental problems. As such, an inherent
outcome of the STAR and GRO Graduate Fellowship Programs is to produce basic and
applied research results usable by EPA and others for decision making and/or policy
development. Has this outcome been obtained? How can the programs be enhanced to
produce results relevant to environmental protection?
5A) Review and compile the publication records of fellows. Although a comprehensive review
may be a daunting task, such information should be readily available from resumes, which the
Subcommittee recommends requiring fellows to periodically update annually for at least a 5-
year period follow ing the end of the fellowship. Other potential sources of this information
include the Thomson ISI Web of Science, which provides citation records; Google Scholar,
which provides some similar information without charge and also links to citations in the gray
literature; and EPA itself, through searching EPA reports for citations of fellows' work.
Response: ORD will adopt recommendation 5 A. We have performed a preliminary assessment
of publication rates of former fellows, which indicates the importance of a retrospective analysis
of this nature.
5B) Require that fellows provide links to their professional Web pages in the information that
they submit to EPA. Such pages, particularly for those in academia, often provide a wealth of
information on publications and other accomplishments of the pages' owners. EPA could add
links to these homepages on an Agency Web site.
-5-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Response: ORD will adopt recommendation 5B. This information has been requested for the
most recent fellows. ORD will also increase its efforts to acquire this information from fellows
in the future.
Charge Question 6; Since the inception of the fellowship programs, assuring diversity among
fellowship recipients has been a goal of ORD. Please comment on the practices, resources, and
effectiveness of the Center's outreach efforts and make recommendations for enhancing the
potential for positive outcomes in the future.
6) Consider the following as potential metrics as data become available: (1) The number of
minority students who obtain advanced degrees in environmental disciplines; (2) the distribution
or dispersion of students across eligible institutions, i.e., the concentration of fellowship
recipients among colleges and universities; and (3) the number of awards to students pursuing
master's degrees relative to the number of awards to students pursuing doctoral degrees.
Response: ORD will adopt recommendation 6. ORD already collects and analyzes data on
metrics 2 and 3 above. ORD will report on metric 1 above in the future. See also the responses
to recommendation 1A above.
Charge Question 7; Please review the fellowship programs' resources, information
management, and communication processes and procedures. Are there any recommendations
for program improvements or sustainability?
7A) To strengthen communication efforts, develop a user-friendly Web site for fellows and a
listserv that will encourage easy communication among them and between EPA and each fellow.
In addition, the upgraded database will allow fellows and alumni to track each other; to reach
colleagues who can collaborate with them on projects, grants, and speaking opportunities; and
to link with EPA after their fellowship is completed.
Response: ORD will adopt this recommendation. The Fellows Forum, a virtual community for
current fellows, is under development and will be launched in the very near future. This virtual
community will be managed and maintained with significant involvement of the fellows.
7B) Market the STAR Fellowship Program strategically to minority communities and
institutions of higher education. If EPA acts on the Subcommittee recommendation to eliminate
the GRO Graduate Fellowship Program, it will be even more vital to increase the marketing of
the STAR Fellowship Program to insure diverse representation in that program. There remains
a lack of data verifying that the STAR Fellowship Program has successfully reached under-
represented minority populations.
Response: ORD will adopt the recommendation to market its fellowship programs more
strategically to minority communities and institutions of higher education. Future
communication and outreach strategies will be a part of the Fellowship Program Strategic Plan.
7C) Enhance efforts to increase the number of applicants to the GRO Undergraduate
Fellowship Program. If the current program is replaced by the consortium approach
recommended by the Subcommittee, expanded outreach would be a key function of the consortia.
-6-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Response: ORD will adopt the recommendation to expand outreach but will not seek to form a
consortium at this time. See also recommendations 2A, 2B, and 3 above. Future communication
and outreach strategies will be a part of the Fellowship Program Strategic Plan.
7D) Work more effectively to communicate awards, results, and successes to a variety of
audiences, including Congress and sponsoring institutions.
Response: ORD will adopt recommendation 7D. Significant progress has been made in this
area in recent years. Current practices include yearly notification of fellowship awards to
appropriate Congressional members and systematic communication to the local press where
fellowship recipients live. Future communication and outreach strategies will be a part of the
Fellowship Program Strategic Plan.
7E) Continue the biennial conference ofawardees to recognize and celebrate the outstanding
recipients of fellowships and provide networking opportunities for fellows.
Response: We concur that a conference of this nature is beneficial to the fellows, to EPA, and
the scientific community. ORD plans to continue to periodically hold a conference for
fellowship recipients.
-7-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
STAR/GRO Fellowship Programs
Summary of BOSC Recommendations From October 2006 Final Report and
Proposed ORD Actions and Timelines
Recommendation
ORD Action
Timeline for
Action
1A) Develop an overall information
collection strategy, which includes design of
an appropriate database.
Recommendation adopted
Target completion
date June 2008
IB) Require fellows to submit an up-to-date
resume annually for at least 5 years from the
conclusion of the fellowship.
Recommendation adopted
Target completion
date June 2008
1C) Require universities with fellowship
recipients to provide the basic initial
information for the database at the conclusion
of the 2- or 3- year fellowship period.
Recommendation not
adopted
ID) Provide greater resources for the STAR
Fellowship Program, both to address the
increased number of applicants that this
change would produce, and to fund a greater
percentage of those applications rated
excellent.
Recommendation not
adopted
IE) Consider increasing the graduate
fellowship expense budget.
Recommendation under
consideration
Report on progress
June 2008
IF) Consider seeking additional funding
within the next few years to maintain the
value of the individual fellowships without
resorting to funding fewer fellows.
Recommendation adopted.
A Fellowship Program
Strategic Plan will be
developed.
Fellowship
Program Strategic
Plan target
completion date
December 2007
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Recommendation
ORD Action
Timeline for
Action
1G) Add a mentoring component to the
program so that there is a specific individual
within EPA who would serve as a resource
for the fellow. The mentor could
complement the function of the students
academic advisor by, for example, helping
the fellow find opportunities that expand his
or her graduate school experiences and
serving as a professional resource regarding
career opportunities in federal agencies. The
Subcommittee recommends that ORD solicit
mentor volunteers from throughout EPA,
matching professional staff with the fellows'
fields of investigation. This strategy would
bring increased visibility to the STAR and
GRO Fellowship Programs within the
Agency.
Recommendation adopted
on a pilot basis
Report on progress
in June 2008
2A) Consider eliminating both GRO
Fellowship programs, while at the same time
improving marketing of the STAR
Fellowship Program to minority-serving
institutions to encourage applications for
graduate support from underrepresented
groups.
Recommendation under
consideration
Report on progress
in June 2008
2B) If the GRO programs are maintained,
review the impacts of its 2003 determination
that competition for program awards cannot
legally be limited to minority-serving
institutions.
Recommendation adopted
Report on progress
in June 2008
-9-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Recommendation
ORD Action
Timeline for
Action
3) To provide sufficient resources to make a
real impact on capacity building for
undergraduate education, consider devoting
the combined resources from both GRO
Fellowship Programs to competitively
selected regional consortia, which would be
designed to focus on environmental
opportunities for undergraduates.
Recommendation under
consideration
Report on progress
in June 2008
4 A) Update the descriptions of the
evaluation and selection criteria in the
funding announcements to clearly describe
the criteria employed in each step of the
review process, to ensure transparency and
fairness for all applicants.
Recommendation adopted
Completed
August 2006
4B) Broaden the categories used to sort
applications to encompass emerging
interdisciplinary fields of study, and consider
making the categories used to sort the STAR
Fellowship Program and the GRO Graduate
Fellowship Program applications identical.
Such consistency would allow for continuity
and overlap of reviewers in the non-EPA
expert panels as well as a clearer comparison
of the two programs.
Recommendation adopted
Completed
January 2007
5 A) Review and compile the publication
records of fellows, etc.
Recommendation adopted
Target completion
date December
2008
5B) Require that fellows provide links to
their professional Web pages in the
information that they submit to EPA.
Recommendation adopted
Target completion
date December
2008
- 10-
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Recommendation
ORD Action
Timeline for
Action
6) Consider the following as potential metrics
as data become available:
(1) The number of minority students who
obtain advanced degrees in environmental
disciplines
(2) The distribution or dispersion of students
across eligible institutions, i.e., the
concentration of fellowship recipients among
colleges and universities
(3) The number of awards to students
pursuing master's degrees relative to the
number of awards to students pursuing
doctoral degrees.
Recommendation adopted
Report on progress
by December 2008
7A) To strengthen communication efforts and
to develop a user-friendly Web site for
fellows and a listserv that will encourage easy
communication among them and between
EPA and each fellow. In addition, the
upgraded database will allow fellows and
alumni to track each other; reach colleagues
who can collaborate with them on projects,
grants, and speaking opportunities; and link
with EPA after their fellowship is completed.
Recommendation adopted
Target completion
date December
2007
7B) Market the STAR Fellowship Program
strategically to minority communities and
institutions of higher education. There
remains a lack of data verifying that the
STAR Fellowship Program has successfully
reached underrepresented minority
populations.
Recommendation adopted.
Communication and
outreach strategies will be
a part of the Fellowship
Program Strategic Plan.
Target completion
date for Strategic
Plan is December
2007. Report on
progress by
December 2008.
- 11 -
-------
May 2007 ORD Response to BOSC October 2006 STAR/GRO Fellowship Program Final Report
Recommendation
ORD Action
Timeline for
Action
7C) Enhance efforts to increase the number
of applicants to the GRO Undergraduate
Fellowship Program.
Recommendation adopted.
Communication and
outreach strategies will be
a part of the Fellowship
Program Strategic Plan.
Target completion
date for Strategic
Plan is December
2007. Report on
progress by
December 2008.
7D) Work effectively to communicate
awards, results, and successes to a variety of
audiences, including Congress and sponsor-
ing institutions.
Recommendation adopted.
Communication and
outreach strategies will be
a part of the Fellowship
Program Strategic Plan.
Target completion
date for Strategic
Plan is December
2007. Report on
progress by
December 2008.
- 12-
------- |