£
<
30
O
%
&z.
% pro^
o
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
EPA Needs to Improve
STAR Grant Oversight
Report No. 13-P-0361
August 27, 2013
Scan this mobile
code to learn more
about the EPA OIG.

-------
Report Contributors:	Eileen Collins
Kimberly Crilly
Janet Kasper
Madeline Mullen
Shannon Schofield
LaTanya Scott
Abbreviations
CFR	Code of Federal Regulations
EPA	U.S. Environmental Protection Agency
FY	Fiscal year
IGMS	Integrated Grants Management System
NCER	National Center for Environmental Research
OARM	Office of Administration and Resources Management
OGD	Office of Grants and Debarment
OIG	Office of Inspector General
ORD	Office of Research and Development
STAR	Science to Achieve Results
Cover photo: A mass spectrometer in a research laboratory is an example of laboratory
equipment that can be used to analyze samples during STAR grant research
(EPA OIG photo).
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
email:	OIG Hotline@epa.gov	write: EPA Inspector General Hotline
phone:	1-888-546-8740	1200 Pennsylvania Avenue. NW
fax:	(202) 566-2599	Mailcode 2431T
online:	http://www.epa.gov/oiq/hotline.htm	Washington. DC 20460

-------
^tDST^ U.S. Environmental Protection Agency	13 P 0361
¦ \ Office of Inspector General	August 27,2013
^ At a Glance
z
uj
PRO^
Why We Did This Review
EPA Needs to Improve STAR Grant Oversight
The U.S. Environmental
Protection Agency, Office of
Inspector General, conducted
this audit to answer the
questions:
1.	Is EPA effectively monitoring
Science to Achieve Results
grant recipient activities?
2.	Does EPA verify that STAR
grant recipients take action
to ensure that research
misconduct is not associated
with grant activities?
The STAR competitive grant
program is the primary vehicle
through which the EPA funds
research at universities and
nonprofit groups. From fiscal
years 2010 through 2012, the
EPA funded 220 projects
totaling $150,043,796 through
the STAR grant program.
This report addresses the
following EPA Goal or
Cross-Cutting Strategy:
• Advancing science, research
and technological innovation.
For further information, contact
our Office of Congressional and
Public Affairs at (202) 566-2391.
The full report is at:
www.epa.aov/oia/reports/20131
20130827-13-P-0361.pdf
What We Found
We found that the EPA's project officers did not monitor STAR grant recipients in
a manner consistent with the agency's policy and guidance. For example, project
officers did not take action when recipients submitted annual reports late, did not
follow baseline monitoring guidance, and did not routinely follow up when
disclaimers about EPA's endorsement were not included in published articles.
By not following policy, project officers increased the risk that issues would not
be corrected in a timely manner and that projects might not meet specified goals.
During administrative advanced-monitoring reviews, the EPA did not ensure costs
were allocable to the grant and did not request certified effort reports. We
reviewed drawdowns totaling $639,045 and found $53,854 in costs that were not
allowable. We also found that one grant recipient's certified effort reports did not
comply with the recipient's own internal policies.
Project officers did not actively monitor STAR grant recipients for potential
research misconduct, or review recipients for compliance with research
misconduct terms and conditions. When the EPA does not monitor research
misconduct, the agency puts grant funds at risk.
Recommendations and Planned Agency Corrective Actions
We recommend that the EPA provide mandatory training to STAR grant project
officers. The training should include baseline monitoring performance and
instructions to ensure baseline monitoring reports are accurately completed.
For incrementally funded grants, the EPA should enforce the terms and conditions
that allow withholding of funds for late or missing reports, and amend these
requirements for all awards so that payment will be withheld when progress
reports are missing or late. We recommend that the EPA require grant recipients
to submit corrections to publications when acknowledgement of EPA funding and
disclaimers of EPA endorsement are missing from articles.
To address unallowable costs, we recommend that the EPA follow up on the
unallowable costs that we identified and issue guidance to grant specialists for
improving administrative advanced-monitoring reviews. We also recommend that
the EPA require project officers to verify grant recipients' awareness of research
misconduct reporting requirements. The EPA's completed and planned corrective
actions address all of the OIG's recommendations.
Noteworthy Achievements
The EPA immediately corrected a website containing outdated terms and
conditions after we informed the agency about the problem. Grant recipients are
now directed to the correct website for grant terms and conditions.

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
August 27, 2013
MEMORANDUM
SUBJECT: EPA Needs to Improve STAR Grant Oversight
Report No. 13-P-0361
FROM: Arthur A. Elkins Jr.

TO:
Lek Kadeli, Principal Deputy Assistant Administrator
Office of Research and Development
Craig Hooks, Assistant Administrator
Office of Administration and Resources Management
This is our report on the subject audit conducted by the Office of Inspector General of the
U.S. Environmental Protection Agency. This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends.
Action Required
In responding to the draft report, the agency provided a corrective action plan for addressing the
recommendations with milestone dates. Therefore, a response to the final report is not required.
The agency should track corrective actions not implemented in the Management Audit Tracking
System. This report will be available at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact Richard Eyermann, acting
assistant inspector general for the Office of Audit, at (202) 566-0565 or eyermann.richard@epa. gov;
or Janet Kasper, product line director, at (312) 886-3059 or kasper.ianet@epa.gov.

-------
EPA Needs to Improve STAR Grant Oversight
13-P-0361
	Table of Contents
Chapters
1	Introduction		1
Purpose		1
Background		1
Noteworthy Achievements		2
Scope and Methodology		2
2	Project Officers Did Not Follow EPA Guidance		4
STAR Grants Must Comply With Federal and EPA Requirements		4
Annual Reports Were Frequently Late		5
Project Officers Did Not Follow Baseline Monitoring Guidance		6
Published Articles Were Missing Required Disclaimers		7
Recommendations		8
Agency Comments and OIG Evaluation		8
3	Drawdown Reviews Need Improvement		10
Federal Requirements Define Allowable Costs		10
EPA Reimbursed Unallowable Costs		10
Administrative Advanced-Monitoring Review Needs Improvement		12
Increased Risk of Reimbursing Unallowable Costs		12
Recommendations		13
Agency Comments and OIG Evaluation		13
4	Research Misconduct Terms and Conditions Were Not Monitored		14
EPA's Terms and Conditions Define Research Misconduct
and Reporting Requirements		14
Project Officers Did Not Monitor for Potential Research Misconduct		15
Project Officers Did Not Ask About Research Misconduct		17
Risk of Research Misconduct Not Addressed		17
Recommendation		17
Agency Comments and OIG Evaluation		18
Status of Recommendations and Potential Monetary Benefits		19
Appendices
A STAR Grants Reviewed by the EPA OIG	 20
B Agency's Response to Draft Report	 21
C Distribution	 26

-------
Chapter 1
Introduction
Purpose
The U.S. Environmental Protection Agency, Office of Inspector General,
conducted an audit of the Science to Achieve Results grant program to
answer the questions:
1.	Is EPA effectively monitoring Science to Achieve Results grant recipient
activities?
2.	Does EPA verify that STAR grant recipients take action to ensure that
research misconduct is not associated with grant activities?
Background
The STAR program began in 1995 and was assigned to the EPA's National
Center for Environmental Research and Quality Assurance—currently known as
the National Center for Environmental Research. The program's research focus
was developed specifically to meet the research needs of the EPA. The program
funds targeted research grants in numerous environmental science and
engineering disciplines.
The STAR competitive grant program is the primary vehicle through which the
EPA funds research at universities and nonprofit groups. From fiscal years 2010
through 2012, the EPA funded 220 projects totaling $150,043,796 through the
STAR grant program. Research efforts funded by these grants are aligned with the
EPA's strategic goals, including the cross-cutting strategic goal of advancing
science and research. The public can access progress reports about these research
projects by visiting the NCER website at
http://cfpub.epa.gov/ncer abstracts/index.cfm/fuseaction/search.welcome.
Although the NCER is responsible for administering the programmatic aspects of
STAR grants, the Office of Grants and Debarment is responsible for the financial
management of the STAR program. The NCER and OGD each perform baseline
and advanced-monitoring reviews of grant recipients.
The EPA includes terms and conditions in the grants that it awards. The terms and
conditions cover topics such as financial and program management of the grants,
timing and content of annual and final reports, and reporting requirements for
research misconduct. According to the Code of Federal Regulations through
2 CFR §215.61, grant recipients failing to comply with the terms and conditions
of an award can have their awards terminated.
13-P-0361
1

-------
Noteworthy Achievements
The EPA ensured that an outdated website was corrected so that the terms and
conditions included in the awards would no longer direct grant recipients to
outdated agency-specific information. We found that the EPA's STAR grant
agreements did not always include the correct website that contained the
applicable agency-specific terms and conditions. When we informed EPA staff
about this, they took immediate action to correct the problem.
Scope and Methodology
We conducted this performance audit from July 2012 through June 2013, in
accordance with generally accepted Government Auditing Standards (Yellow
Book) issued by the Comptroller General of the United States. Those standards
require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on
our audit objective. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objective.
The 12 STAR grants that we reviewed had a total funding amount of
$10,385,348. The audit team initially made a judgmental selection of two grants
with a total obligation amount of $4,680,608. After reviewing these two grants to
gain an understanding of the STAR grant process and documents, the team
randomly sampled 10 additional grants. We identified 84 STAR grants with
project start dates after October 1, 2008, and project end dates prior to
September 30, 2013. The total funding amount for these grants was $49,798,525.
The team generated a random sample of 10 grants with a total obligation amount
of $5,704,740.
To determine whether there was evidence of monitoring and whether the
monitoring was conducted in accordance with applicable policies and procedures,
we obtained annual progress reports from the selected grant recipients' project
officers. We reviewed the annual progress reports to determine if the reports were
submitted on time and whether they indicated progress commensurate with the
outputs and outcomes identified in the work plan. We also obtained programmatic
and administrative reports from the EPA's Integrated Grants Management
System. The IGMS reports were completed by project officers and grant
specialists.
To determine whether grant recipients were complying with the terms and
conditions of their grants, we obtained the following documents from selected
grant recipients and from websites where scientific articles are posted.
• Supporting financial documentation for three selected drawdowns for
each of the five judgmentally selected grant recipients to determine if
claimed costs were allowable. We selected the five grants based on
13-P-0361
2

-------
information noted on the grant application indicating the recipient had
received funding for similar grants from EPA or other federal agencies.
•	Six published articles to determine whether the articles contained
an acknowledgment of EPA funding and a disclaimer stating the
EPA does not endorse the article. At the time of the audit, only six
of the grant recipients had published articles.
•	Scientific integrity policies of nine grant recipients to determine whether
their policies were consistent with EPA requirements for reporting
research misconduct, as outlined in the agency's terms and conditions.
We reviewed the scientific integrity policies for the grant recipients that
we were also reviewing for financial documentation and published
articles. We limited our review to nine grant recipients to comply with
requirements of the Paperwork Reduction Act.
We interviewed the OGD and Office of Research and Development staff
(including project officers), and discussed their procedures for monitoring grant
recipients and ensuring scientific integrity. We also interviewed the grant
recipients to discuss applicable terms and conditions, reporting, and interactions
with the EPA. The grants reviewed are listed in appendix A.
There were no prior audits impacting the objectives of this assignment.
13-P-0361
3

-------
Chapter 2
Project Officers Did Not Follow EPA Guidance
Project officers did not monitor STAR grant progress in a manner consistent with
the EPA's policy and guidance. Project officers also allowed grant recipients to
submit annual reports late. STAR grant recipients must comply with numerous
federal and EPA requirements. The project officers emphasized overall progress
rather than compliance with specific requirements. This introduced risks that
could lead to the project officers not detecting issues needing corrective action
and that might impact the project meeting its goals. In addition, missing
acknowledgements and disclaimers in published research put the EPA's
transparency at risk and could give an appearance of the agency endorsing the
research.
STAR Grants Must Comply With Federal and EPA Requirements
The EPA's Assistance Administration Manual (EPA Order 5700) requires agency
project officers to conduct baseline monitoring of all assistance agreements.
Monitoring is supposed to include a review of:
•	Compliance with programmatic terms and conditions.
•	Drawdowns and budget expenditures.
•	Recipient progress, including a comparison of funds available to project
progress or time remaining on the project.
The EPA's I)olicy on Compliance, Review, and Monitoring (EPA Order 5700.6)
requires project officers to conduct baseline monitoring at least annually, and that
baseline monitoring should include a review of the grant recipient's annual report.
Terms and conditions for EPA grants require recipients to submit annual reports
on grant progress. Depending on when the grant was awarded, the due date for the
annual report varies.
Table 1: Annual report due date
Terms and conditions
issuance date1
Annual report due date
July 2008
30 days prior to the end of the reporting period
June 2009
30 days prior to the end of the reporting period
November 2010
90 days after the end of the reporting period
Source: EPA grant terms and conditions.
1 Terms and conditions apply to all grants awarded after the date the terms and conditions were issued.
13-P-0361
4

-------
If the recipient does not submit the annual report by the due date, the terms and
conditions for partially funded grants allow the EPA not to provide further
funding until the recipient has submitted the required annual progress report.
The EPA encourages independent publication of results from its extramural
research in appropriate scientific journals. When doing so, the terms and
conditions require that there be an acknowledgement of EPA funding. The exact
wording of the acknowledgement depends on when the grant was awarded. The
FY 2008 terms and conditions required grant recipients to include the following
statement in all journal articles publishing work funded by an EPA grant:
Although the research described in this article has been funded
wholly or in part by the United States Environmental Protection
Agency through grant/cooperative agreement (number) to (name),
it has not been subjected to the Agency's required peer and policy
review and therefore does not necessarily reflect the views of the
Agency and no official endorsement should be inferred.
The FY 2009 and FY 2010 terms and conditions revised the statement as follows:
This publication was made possible by USEPA grant (number). Its
contents are solely the responsibility of the grantee and do not
necessarily represent the official views of the USEPA. Further,
USEPA does not endorse the purchase of any commercial products
or services mentioned in the publication.
Annual Reports Were Frequently Late
Of the 12 grants that we reviewed, 10 grant recipients submitted late annual
reports for 2 years or more during the life of the projects (see table 2):
Table 2: Annual progress reports that were late or not submitted
Grant
number
2007
2008
2009
2010
2011
2012
83337401
273
109
Not
submitted
231
Not
submitted

83409301


Not
submitted
140
289
Not
submitted
83389201



98
123
225
83435901



277
237
141
83428501



122
139
135
83428201



123
139
Not
submitted
83438601



150
211

83459601




322
161
83399002


454
434
168

83337302




144
1
Source: OIG analysis of recipient annual reports for days late. (Shaded cells indicate
reports were not required that year.)
13-P-0361
5

-------
The EPA's project officers did not enforce the specific annual reporting time
frames listed in the terms and conditions. In many cases, the project officer did
not identify the correct annual report due dates in the terms and conditions,
because award documents directed the reader to an outdated website that
contained incorrect information. Project officers also did not conduct monitoring
in compliance with EPA Order 5700.6, and one project officer chose to allow a
grant recipient to submit late annual reports despite the due dates that were
required by the terms and conditions. The often substantial tardiness of annual
reports introduced an unnecessary risk that could lead project officers not to
detect issues needing corrective action and might impact the project not meeting
its goals.
For grants that were awarded in full, there were no consequences for late or
missing reports. For grants that received incremental funding, the terms and
conditions gave the EPA the option of not providing further funding until grant
recipients submitted annual progress reports. Four of the grants we reviewed
received incremental funding, and three of those four grant recipients consistently
submitted annual reports late. In fact, further funding had already been provided
for two of the recipients before the first progress report was due, and one recipient
was fully funded after the first progress report was submitted late. As a result, the
EPA could not exercise this option.
Project Officers Did Not Follow Baseline Monitoring Guidance
Of the eight EPA project officers that we interviewed, four did not conduct
monitoring in compliance with EPA Order 5700.6 requirements. Specifically:
•	Two project officers did not complete programmatic baseline monitoring
reports for 1 year or more.
•	One project officer conducted baseline monitoring without reviewing the
recipient's annual progress report.
•	One proj ect officer did not review the proj ect's progress against the
timeline in the work plan, because the timeline was missing.
The project officers are supposed to conduct baseline monitoring annually and
document the monitoring in the IGMS. Baseline monitoring includes a review of
the grant recipients' annual progress reports. The 2007 baseline monitoring
checklist guidance did not require project officers to note whether annual reports
were late. This introduced a risk that the EPA could not detect issues needing
corrective action and that projects might not meet their goals. The EPA updated
that checklist in 2012 to include a specific question concerning whether progress
reports were submitted as required. When project officers do not follow baseline
monitoring guidance, the EPA's oversight of grants is hindered.
The terms and conditions require grant recipients to report expenditures to date,
along with providing a comparison of the percentage of the project completed
13-P-0361
6

-------
according to the project schedule. However, one project officer did not think that
a timeline was necessary to review the project progress. Because the grant
recipient's work plan did not include a timeline, the project officer was unable to
monitor sufficient progress within the agreed-upon project period. Subsequently,
he approved a no-cost time extension to perform the work and/or to use the
unexpended funds for additional work. When project officers allow STAR grant
recipients to rely upon no-cost time extensions to complete work that the
recipients had agreed to finish in a shorter time period, the integrity of the STAR
grant award process is called into question.
Published Articles Were Missing Required Disclaimers
Of the six published articles reviewed, three did not contain the disclaimer
required by grant terms and conditions noting the EPA does not endorse the
article (see table 3).
Table 3: Published articles reviewed
Grant
number
Article title
Disclaimer was
not included
83409301
UV Irradiation and Humic Acid Mediate Aggregation of
Aqueous Fullerene (nC60) Nanoparticles
X
83428501
Economically Consistent Long-Term Scenarios for Air
Pollutant Emissions
X
83459601
Exposure to Traffic-related Air Pollution During
Pregnancy and Term Low Birth Weight: Estimation of
Causal Associations in a Semiparametric Model
X
Source: EPA OIG analysis of select published articles.
Project officers learned about the inclusion of the required acknowledgment and
disclaimer in published articles only after the actual publication; therefore, any
corrective action by the agency was after the fact. Some project officers said they
checked and found missing information, but only one acknowledged following up
with the grant recipient. Follow-up was not consistent. During the audit, NCER
management said actions could be taken to improve compliance with this
requirement.
Without the required disclaimer, people may presume that the EPA approves of
the information presented in the articles. This is especially true when EPA
funding is acknowledged. Public confidence in the EPA's mission could be
diminished if the research is contradicted in the future.
13-P-0361
7

-------
Recommendations
We recommend that the Assistant Administrator for Research and Development:
1.	Require all project officers to attend training on how to perform baseline
monitoring. Include a discussion concerning actions to be taken when
publications do not include required acknowledgements and disclaimers.
2.	Conduct a review of baseline monitoring reports to ensure they are
accurately completed.
3.	Enforce the terms and conditions that allow funds to be withheld from
incrementally funded grants, if reports are missing or late.
4.	Amend terms and conditions so that all awards are subject to a condition
that payments will be withheld when reports are missing or late.
5.	Require the grant recipient to submit corrections to publications when
project officers identify missing acknowledgements and disclaimers.
Require immediate correction for online articles and at the next
publication date for printed articles.
6.	Require grant recipients to submit corrections to publications for the
articles that the EPA OIG identified as missing acknowledgments and
disclaimers.
Agency Comments and OIG Evaluation
The EPA agreed with the recommendations and stated that the ORD will provide
all STAR grant project officers with training on the performance of baseline
monitoring. The ORD has drafted standard operating procedures for steps to be
taken when annual progress reports are not received in a timely manner, including
when to pursue withholding funds from incrementally funded grants due to failure
to comply. The ORD is revising its guidance pertaining to annual report reviews, to
include actions to be taken when publications do not have the required
acknowledgements and disclaimers.
The ORD is adding a requirement for grant recipients to ensure that publications
have EPA acknowledgements and disclaimers. The ORD is also revising its terms
and conditions to include a condition that payments may be withheld when reports
are missing or late. This will be applied to all new STAR grants. If
acknowledgements and disclaimers are identified as missing, principal investigators
will be instructed to submit corrections. When implemented, the agency's actions
should address recommendations 1, 3, 4 and 5.
13-P-0361
8

-------
The EPA reported that corrective actions for two of the recommendations have
already been completed. The NCER completed a review of project officer file
records and modified its multiyear review plan to include an annual review of
STAR grants, including baseline monitoring reports. Also, the ORD received
copies of three correction requests that journal editors made to add the required
disclaimers. The EPA's corrective actions address recommendations 2 and 6.
13-P-0361
9

-------
Chapter 3
Drawdown Reviews Need Improvement
During administrative advanced-monitoring reviews, the EPA does not ensure
that costs are allocable to the grant and does not request certified effort reports.
We reviewed drawdowns totaling $639,045 and found $53,854 in costs that were
not allowable because the costs were either unallocable or unsupported. We also
found that one grant recipient's certified effort reports did not comply with the
recipient's own internal policies. EPA Order 5700.6 states that the purpose of the
advanced monitoring is to conduct in-depth assessments, including administrative
and financial progress. Federal regulations state costs are allowable only if they
are allocable to the grant agreement. The lack of detailed cost reviews increases
the risk that more unallowable costs will be reimbursed in the future.
Federal Requirements Define Allowable Costs
An allowable cost is defined in 2 CFR Part 220, Appendix A. A cost is allowable
if it is reasonable, allocable to the sponsored agreements, and given consistent
treatment. A cost is allocable to a particular cost objective if it is incurred solely
to advance the work and benefits both the sponsored agreement and other work of
the institution.
According to 2 CFR Part 220, Appendix A, grant recipients are also required to
confirm the activity of personnel who charge time to the grant through what is
referred to as "certified effort reports." These signed reports verify the work was
performed by employees and can be signed by either the employee or a
responsible official. The effort reports verify that salaries or wages were correctly
charged against the grant.
EPA Order 5700.6 requires the EPA to evaluate the allowability of costs as a part
of advanced monitoring. The order states that the purpose of administrative
advanced monitoring is to conduct in-depth assessments of a recipient's
administrative and financial progress. Additional EPA guidance for administrative
advanced monitoring requires the testing of transactions to determine whether
claimed costs are allowable, allocable, and reasonable under the assistance
agreement.
EPA Reimbursed Unallowable Costs
The EPA incorrectly reimbursed a total of $53,854 (see table 4). Of the
$6,906,269 drawn down by the five grant recipients, we reviewed $639,045.
13-P-0361
10

-------
The costs were not allowable because they were either not allocable to the grant
or were not supported. While we only looked at a sample of the drawdowns, the
sample may be an indicator that other unallowable costs exist.
Table 4: Summary of unallowable costs
Type of cost
Unallowable costs
Note
Personnel
$44,886
1
Travel
426
2
Other
292
3
Indirect costs
8,250
4
Total
$53,854

Source: OIG analysis of recipient drawdown records.
Note 1-Personnel Costs
One grant recipient claimed $44,886 of personnel costs that were not allowable.
The grant recipient did not provide certified effort reports as required by 2 CFR
Part 220, Appendix A to confirm that distribution of labor is accurate.
Recipient effort reports did not always meet federal regulations or the grant
recipient's own internal policies. For example, one grant recipient provided
incorrect effort reports:
•	Three effort reports were certified by individuals who were not authorized
to certify effort reports, according to the grant recipient's own internal
policy.
•	Nine effort reports did not identify the certifier by name, as required by
2 CFR Part 220, Appendix A.
Note 2-Travel Costs
Travel costs totaling $426 were not allowable:
•	Travel expenses totaling $224 were not allocable.
o The travel expense report showed a $200 expense for a personal
side trip to a different location. The costs claimed for the return
flight were not reduced to account for the expense associated with
the personal side trip,
o Internal audit fees related to two travel expense reports and totaling
$24 were included in the draw. However, the purpose of the travel
associated with these expense reports was not entirely related to
the EPA grant.
13-P-0361
11

-------
•	Travel expenses totaling $202 were not supported.
o A $198 hotel expense was not supported with a hotel receipt,
o A $4 local travel expense was not supported because $18 was
claimed, but only $14 was supported.
Note 3-Other Costs
The grant recipient claimed $292 in other costs that were not allowable:
•	Computer services costs totaling $211 were not supported.
•	Communication costs totaling $81 were charged to the STAR grant when
the cost should have been shared with another non-EPA funded project.
Note 4-Indirect Costs
Because personnel, travel and other costs identified above were not allowable,
associated indirect costs totaling $8,250 were also not allowable.
Administrative Advanced-Monitoring Review Needs Improvement
One of the grants in our sample had an administrative advanced-monitoring desk
review, where we found:
•	The grant specialist traced payroll costs to the payroll register, but not to
the certified effort reports. According to OGD staff, the grant specialist
does not request effort reports during an administrative advanced-
monitoring review.
•	The grant specialist traced travel costs to the supporting documents, but
did not determine whether the travel was applicable to the grant.
According to OGD staff, the grant specialist would not know whether the
travel was applicable to the grant, because the specialists do not have the
same technical knowledge about the project as the project officers.
The EPA's grants staff do not collaborate with program staff on cost reviews. The
agency's policy requires grant specialists to review drawdowns as a part of the
administrative advanced monitoring. However, according to EPA staff, drawdown
reviews by grant specialists did not include verification or consultation with the
project officer about cost allocability.
increased Risk of Reimbursing Unallowable Costs
Federal funds drawn for unallowable costs could have been put to better use.
When the OGD and project officers do not collaborate on reviewing costs during
administrative advanced monitoring, there is a risk that unallowable costs will be
13-P-0361
12

-------
reimbursed. Also, not reviewing effort reports increases the risk that unallowable
personnel costs will be reimbursed.
Recommendations
We recommend that the Assistant Administrator for Administration and
Resources Management:
7.	Follow up on unallowable costs identified in the finding. If grant
recipients cannot provide documentation, require repayment of the funds.
8.	Issue guidance to grant specialists, and remind them that during
transaction testing they are required to:
a.	Trace costs to source documents, including a review of certified
effort reports.
b.	Make a determination as to whether costs are related to the
activities funded by the grant. Where grant specialists cannot
determine allocability, they should work with the project officer to
verify costs associated with the grant.
Agency Comments and OIG Evaluation
The EPA agreed with the recommendations and stated that the Office of
Administration and Resources Management will follow up on unallowable costs
and require repayment as appropriate. When implemented, the agency's actions
should address recommendation 7. Also, the OARM reminded grant specialists to
follow advanced-monitoring guidance provisions to trace grant costs to source
documentation and determine whether costs are related to grant-funded activities.
If there are allocability questions, grant specialists will work with the project
officer to verify costs. The agency's action addresses recommendation 8.
13-P-0361
13

-------
Chapter 4
Research Misconduct Terms and Conditions
Were Not Monitored
Project officers did not actively monitor STAR grants for potential research
misconduct or assess compliance with research misconduct terms and conditions.
The grant recipients' scientific integrity policies do not always reflect an
understanding of this requirement and can conflict with agency policy. EPA Order
5700.6 requires project officers to conduct baseline monitoring of all assistance
agreements, including a review of compliance with programmatic terms and
conditions. The EPA's terms and conditions require grant recipients to notify the
agency when the recipients discover research misconduct.
Project officers assumed grant recipients would report research misconduct. The
project officers also assumed the grant pre-award peer-review process and the
published article peer-review process would lessen the likelihood of research
misconduct. However, grant recipients may not understand the requirements for
identifying and reporting research misconduct. The pre-award peer-review
process only searches for one type of research misconduct—a conflict of interest;
while during the publication peer-review process, the NCER does not receive any
information. In both instances, the lack of monitoring increases the risk that
federal funds will be misused.
EPA's Terms and Conditions Define Research Misconduct and
Reporting Requirements
The EPA's terms and conditions for research misconduct state that in accordance
with 40 CFR 30.51(f), the recipient agrees to notify the EPA project officer in
writing about research misconduct involving research activities that are supported
in whole or in part with agency funds. The EPA defines research misconduct as
fabrication, falsification, or plagiarism in proposing, performing, or reviewing
research or in reporting research results; or ordering, advising or suggesting that
subordinates engage in research misconduct. Terms and conditions specified in
the "EPA Research and Related Agency Specific Requirements" call for STAR
grant recipients to:
(1) immediately notify the EPA Project Officer who will then
inform the EPA Office of Inspector General (OIG) if, at any
time, an allegation of research misconduct falls into one of the
categories listed below:
A.	Public health or safety is at risk.
B.	Agency resources or interests are threatened.
13-P-0361
14

-------
C.	Circumstances where research activities should be
suspended.
D.	There is a reasonable indication of possible violations of
civil or criminal law.
E.	Federal action is required to protect the interests of those
involved in the investigation.
F.	The research entity believes that the inquiry or
investigation may be made public prematurely so that
appropriate steps can be taken to safeguard evidence and
protect the rights of those involved.
G.	Circumstances where the research community or public
should be informed.
(2) report other allegations to the OIG when they have conducted
an inquiry and determined that there is sufficient evidence to
proceed with an investigation.
EPA Order 5700.6 requires project officers to conduct baseline monitoring of all
assistance agreements, including a review of compliance with programmatic
terms and conditions. The EPA's programmatic baseline monitoring template
includes the following question: Is the recipient complying with all applicable
programmatic terms and conditions? The 2012 baseline monitoring template
provides additional guidance and states that the project officer has to confirm that
all terms and conditions are being met to the maximum extent practicable.
Project Officers Did Not Monitor for Potential Research Misconduct
Project officers are not monitoring grant recipient compliance with research
misconduct terms and conditions. Project officers said it is the grant recipient's
responsibility to report research misconduct, and the officers assume the
recipients will do so if misconduct occurs. The project officers also did not review
grant recipients' scientific integrity policies to determine whether the recipients
understood the EPA's research misconduct definitions or the agency's notification
requirements.
We reviewed scientific integrity policies for nine grant recipients (see table 5) and
found that the policies did not include:
•	A complete definition of research misconduct.
•	A statement that the sponsoring agency would be notified immediately
upon discovery of research misconduct.
13-P-0361
15

-------
Table 5: Scientific integrity policies not consistent with terms and conditions
Grant
number
Missing notification
requirement
Incomplete definition of
research misconduct
83337401
X
X
83329201
X
X
83409301
X
X
83438601
X
X
83428501
X
X
83428201
X
X
83459601
X
X
83399002
X
X
83337302

X
Source: OIG analysis of grant recipients' scientific integrity policies.
Notification Requirement
The EPA's terms and conditions require grant recipients to immediately notify the
project officer if, at any time, an allegation of research misconduct falls into one
of seven specified categories. However we found that eight grant recipients'
policies contained reporting language that was inconsistent with the terms and
conditions. For example:
•	One policy stated that the dean shall take whatever action he or she
considers appropriate and would provide a copy of the proceedings to the
EPA when required. There is a risk that all applicable allegations of
research misconduct would not be reported unless the dean took action on
the allegations first.
•	Another policy said the grant recipient would only notify the federal
sponsor within 24 hours if there is a reasonable indication of a possible
federal criminal and civil violation. It did not state that the sponsor would
be immediately notified for other conditions and did not include
circumstances where research activities should be suspended. There is a
risk that all applicable allegations of research misconduct would not be
reported.
Definition of Research Misconduct
All of the nine grant recipients' policies defined research misconduct as
"fabrication, falsification or plagiarism, in proposing, performing, or reviewing
research, or in reporting research results," as included in the EPA's terms and
conditions. However, none of the nine grant recipients included "ordering,
advising or suggesting that subordinates engage in research misconduct" in their
definition—terms that are found in the EPA's definition of research misconduct.
13-P-0361
16

-------
Project Officers Did Not Ask About Research Misconduct
The EPA's guidance does not include specific procedures for project officers to
follow when monitoring grant recipients for potential research misconduct.
Project officers relied on the grant recipients' experience and expertise and
assumed recipients would report research misconduct when it occurred. Project
officers made the following assumptions:
•	Grant recipients could be relied on to exercise scientific integrity, and that
it would not be necessary to review the policies to ensure that grant
recipients understood the research misconduct reporting requirements in
the terms and conditions.
•	The prestige of the universities and the professional reputation of the
principal investigators lessened the likelihood of research misconduct.
•	The pre-award process and the peer-review process for articles published
in reputable journals lessened the likelihood of research misconduct.
Risk of Research Misconduct Not Addressed
Because grant recipients' policies do not specifically address the reporting of
research misconduct allegations to the EPA, there is a higher risk that the agency
may not be notified when grant recipients become aware of actual or alleged
research misconduct. An article appearing m Nature, an international journal of
science, described a survey of scientists early in their career and in mid-career,
and noted that 33 percent of the scientists admitted to engaging in at least one
research misconduct activity during the previous 3 years. If the EPA is not
notified when research misconduct is discovered, this could prevent the EPA from
taking corrective action or determining whether grant funds were misused. The
STAR program funds millions of dollars in research each year, and research
misconduct puts those federal dollars at risk.
Recommendation
We recommend that the Assistant Administrator for Research and Development:
9. Require that during annual baseline monitoring, project officers verify that
grant recipients fully understand research misconduct reporting
requirements.
13-P-0361
17

-------
Agency Comments and OIG Evaluation
The EPA agreed with the recommendation and stated that the NCER will modify
its terms and conditions for annual progress reports to include an assertion of
compliance with the research misconduct terms and conditions. When
implemented, the agency's action should address recommendation 9.
13-P-0361
18

-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (In $000s)
Rec.
No.
No.
Subject
Status1
Action Official
Require all project officers to attend training on how
to perform baseline monitoring. Include a discussion
concerning actions to be taken when publications
do not include required acknowledgements and
disclaimers.
Planned
Completion
Date
Claimed
Amount
Agreed-To
Amount
Assistant Administrator for 12/31 /13
Research and Development
8 Conduct a review of baseline monitoring reports to
ensure they are accurately completed.
8 Enforce the terms and conditions that allow funds to
be withheld from incrementally funded grants, if
reports are missing or late.
8 Amend terms and conditions so that all awards are
subject to a condition that payments will be withheld
when reports are missing or late.
8 Require the grant recipient to submit corrections to
publications when project officers identify missing
acknowledgements and disclaimers. Require
immediate correction for online articles and at the
next publication date for print articles.
8 Require grant recipients to submit corrections to
publications for the articles that the EPA OIG
identified as missing acknowledgements and
disclaimers
13 Follow up on unallowable costs identified in the
finding. If grant recipients cannot provide
documentation, require repayment of the funds.
13 Issue guidance to grant specialists, and remind
them that during transaction testing they are
required to:
a.	Trace costs to source documents, including a
review of certified effort reports.
b.	Make a determination as to whether costs are
related to the activities funded by the grant.
Where grant specialists cannot determine
allocability, they should work with the project
officer to verify costs associated with the
grant.
17 Requirethat during annual baseline monitoring,
project officers verify that grant recipients fully
understand research misconduct reporting
requirements,
Assistant Administrator for 7/19/13
Research and Development
Assistant Administrator for 9/30/13
Research and Development
Assistant Administrator for 9/30/13
Research and Development
Assistant Administrator for 9/30/13
Research and Development
Assistant Administrator for 7/19/13
Research and Development
Assistant Administrator for 12/31 /13
Administration and Resources
Management
Assistant Administrator for 08/12/13
Administration and Resources
Management
$54
$54
Assistant Administrator for 12/31 /13
Research and Development
1 0 = Recommendation is open with agreed-to corrective actions pending.
C = Recommendation is closed with all agreed-to actions completed.
U = Recommendation is unresolved with resolution efforts in progress.
13-P-0361	19

-------
Appendix A
STAR Grants Reviewed by the EPA OIG
Grant
number
Project title
Grant
period
EPA
funding
Grant
randomly
selected
Principal
investigators
interviewed
Scientific
integrity
policy
reviewed
Published
articles
reviewed
Drawdown
support
reviewed
83337401
Changes in Climate, Pollutant Emissions,
and United States Air Quality
03/01/07-
02/28/12
$896,597

X
X
X
X
83329201
Center for Children's Environmental
Health
11/01/06 -
10/31/13
$3,784,011

X
X
X
X
83389201
Platinum-Containing Nanomaterials:
Sources, Speciation, and Transformation
in the Environment
02/01/09-
01/31/13
$399,406
X




83409301
Natural Organic Matter/C60 Fullerene
Interactions
12/01/08 —
11/30/12
$399,996
X
X
X
X

83428501
Analysis of Co-benefits of GHG
Abatement
09/01/09-
08/31/13
$300,000
X
X
X
X

83399002
Novel Markers of Air Pollution Vascular
Toxicity
11/01/10 —
10/31/12
$432,664
X
X
X
X

83459601
Children's Environmental Health Center
05/07/10-
05/06/13
$1,091,783
X
X
X
X
X
83428201
Effects of Changing Climate on Fires & Air
Quality
11/01/09-
10/31/13
$599,366
X
X
X

X
83438601
C02 Sequestration Modeling to Protect
Drinking Water Sources
12/01/09 —
11/30/13
$899,567
X
X
X

X
83337302
Impacts of Global Climate and Emissions
Changes on U.S. Air Quality Zones
01/04/11 -
04/14/13
$259,778
X
X
X
X

83482401
Comparison of Mice for Allergenicity
Testing
09/15/10 —
09/14/13
$423,546
X




83435901
Predicting Regional Allergy Hotspots -
Future Climates
09/01/09-
08/31/13
$898,634
X




Total


$10,385,348
10
9
9
7
5
Source: Project officers interviewed, file documents, and annual reports reviewed.
13-P-0361	20

-------
Appendix B
Agency's Response to Draft Report
July 19, 2013
MEMORANDUM
SUBJECT: Response to the Office of Inspector General Draft Report titled, The EPA Needs
to Improve STAR Grant Oversight Project No. OA-FY12-0606, Dated June 18,
2013
FROM: Craig E. Hooks, Assistant Administrator
Office of Administration and Resources Management (OARM)
Lek G. Kadeli, Principal Deputy Assistant Administrator
Office of Research and Development (ORD)
TO:	Arthur A. Elkins, Jr., Inspector General
Office of Inspector General (OIG)
Thank you for the opportunity to respond to the OIG draft report titled, The EPA Needs to
Improve STAR Grant Oversight. EPA is committed to keeping the Science to Achieve Results
(STAR) grant program as a highly competitive, rigorously peer-reviewed, well-managed
extramural research grants program. To further the cross-cutting Agency goal of enhancing
science and research while upholding the integrity of the way in which the Agency utilizes its
resources, the EPA's ORD and the OARM will renew our commitment to collaborate closely to
that end. The STAR grant program helps maximize the Agency's resources by complementing
and supplementing the research of our in-house laboratories that otherwise would not benefit
from the increased interagency collaboration and greater interactions with academic and non-
profit institutions on the forefront of innovative scientific research.
We appreciate the OIG's work to assess EPA's monitoring of STAR grants and proactive efforts
to ensure that research misconduct is not associated with grant activities. Most notably, your
team's efforts to communicate with the Agency on a regular basis about potential findings and
recommendations have allowed us to take some immediate corrective actions during the course
of the review. In addition, these communications have provided us an opportunity to clarify
issues relating to certain findings and recommendations. In short, and as discussed during our
communications with your team, EPA generally agrees with the OIG's recommendations
regarding STAR grants and has provided the corrective actions shown below.
13-P-0361
21

-------
AGENCY'S RESPONSE TO REPORT RECOMMENDATIONS
No.
Recommendation
Responsible
Office
Corrective Action (s)
Estimated
Completion
Date
1
Require all project officers
to attend training on how
to perform baseline
monitoring. Include a
discussion concerning
actions to be taken when
publications do not include
required
acknowledgements
and disclaimers.
ORD
ORD will provide training on
performance of baseline monitoring to
all STAR grant project officers. In
addition, we are revising the annual
report review guidance to include
information on actions to be taken
when publications do not include
required acknowledgements and
disclaimers.
1st Quarter FY
2014
2
Conduct a review of
baseline monitoring reports
to ensure they are
accurately completed.
ORD
In January 2013, NCER completed a
review of official file records of STAR
grants project officers, which included
a review of baseline monitoring reports.
As part of ORD's FY 2013 Annual
Assurance Letter, NCER modified its
2013-2017 Multiyear Review Plan to
include an annual review of STAR
grants, including a review of baseline
monitoring reports. Documentation that
completes this corrective action has
been provided to ORD's Audit Liaison.
Completed
3
Enforce the terms and
conditions that allow funds
to be withheld from
incrementally funded
grants, if reports are
missing or late.
ORD
ORD has drafted Standard Operating
Procedures (SOP), which will be
finalized with assistance from OARM,
to address steps to be taken when
annual progress reports for STAR
grants are not received in a timely
manner. The SOP includes information
on when to pursue withholding funds
from incrementally funded grants due
to failure to comply with this term and
condition.
4th Quarter FY
2013
13-P-0361
22

-------
No.
Recommendation
Responsible
Office
Corrective Action (s)
Estimated
Completion
Date
4
Amend terms and
conditions so that all
awards are subject to a
condition that payments
will be withheld when
reports are missing or late.
ORD
ORD is revising the terms and
conditions for its STAR grant program
to including a condition that payments
may be withheld when reports are
missing or late. The revised terms and
conditions will apply to all new STAR
grants.
4th Quarter FY
2013
5
Require the grant recipient
to submit corrections to
publications when project
officers identify missing
acknowledgements and
disclaimers.
Require immediate
correction for online
articles and at the next
publication date for printed
articles.
ORD
ORD is revising its SOP for STAR
grants annual progress report review to
include a requirement that grant
recipients ensure their publications
have appropriate EPA
acknowledgements and disclaimers. If
acknowledgements and disclaimers are
identified as missing, the principal
investigator will be instructed to submit
corrections to the publications
requesting corrections.
4th Quarter FY
2013
6
Require grant recipients to
submit corrections to
publications for the articles
that the EPA OIG
identified as missing
acknowledgments and
disclaimers.
ORD
ORD has received copies of three
correction requests made of the journal
editors and determined the fourth
publication was submitted in error.
That publication has been removed
from the grants publication list.
Documentation that completes this
corrective action has been provided to
ORD's Audit Liaison.
Completed
7
Follow up on unallowable
costs identified in the
finding. If grant recipients
cannot provide
documentation, require
repayment of the funds.
OARM
OARM will follow up on unallowable
costs and require repayment of costs as
appropriate.
1st Quarter of
FY 2014.
13-P-0361
23

-------
No.
Recommendation
Responsible
Office
Corrective Action (s)
Estimated
Completion
Date
8
Issue guidance to grant
specialists, and remind
them that during
transaction testing they are
required to:
a.	Trace costs to source
documents, including a
review of certified effort
reports.
b.	Make a determination
as to whether costs are
related to the activities
funded by the grant. Where
grant specialists cannot
determine allocability, they
should work with the
project officer to verify
costs associated with the
grant.
OARM
a.	OARM has guidance that relates to
this finding which can be found on the
OARM/OGD G share at
G:\GIAMD\Advanced Monitoring
Guidance. For example, the Standard
Operating Procedures for Grants
Management Office Desk and Onsite
Reviews, Section III, Attachment 1 C
(e), covers the review of supporting
documentation related to payroll and
other costs. OARM will remind grant
specialists to follow the provisions in
the guidance on tracing grant costs to
source documentation.
b.	OARM guidance related to this issue
can be found in the Standard Operating
Procedures for Grants Management
Office Desk and Onsite Reviews,
Section III, Attachment 6. OARM will
remind grant specialists that during
transaction testing they are required to
determine whether costs are related to
grant funded activities and if there are
allocability questions, to work with the
project officer to verify costs.
4th Quarter FY
2013
9
Require that during annual
baseline monitoring,
project officers verify that
grant recipients fully
understand research
misconduct reporting
requirements.
ORD
NCER will modify the term and
condition for annual progress reports to
include an assertion of compliance with
the research misconduct term and
condition.
4th Quarter FY
2013
Should you or your staff have any questions related to EPA's Office of Research and
Development's responsibilities for recommendations 1-6 and 9, please contact Deborah
Heckman at (202) 564-7274. For questions related to EPA's Office of Administration and
Resources Management responsibilities for recommendations 7 and 8, please contact Jennifer
Hublar at (202) 564-5294.
cc: Ramona Trovato
Bob Kavlock
Nanci Gelb
13-P-0361
24

-------
Howard Corcoran
Jim Johnson
Bruce Binder
Darrell Winner
13-P-0361

-------
Appendix C
Distribution
Office of the Administrator
Principal Deputy Assistant Administrator, Office of Research and Development
Assistant Administrator, Office of Administration and Resources Management
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Principal Deputy Assistant Administrator, Office of Administration and Resources Management
Director, Grants and Interagency Agreements Management Division,
Office of Administration and Resources Management
Director, Office of Policy and Resource Management,
Office of Administration and Resources Management
Deputy Director, Office of Policy and Resource Management,
Office of Administration and Resources Management
Audit Follow-Up Coordinator, Office of Research and Development
Audit Follow-Up Coordinator, Office of Administration and Resources Management
Audit Follow-Up Coordinator, Office of Policy and Resource Management,
Office of Administration and Resources Management
13-P-0361
26

-------