$
<
33
O
%
^*os""o,
o
W
PRQ-S^
z
Ui
-------
Report Contributors: Eileen Collins
Allison Dutton
LaTanya Furdge
Patrick Gilbride
Todd Goldman
Madeline Mullen
Michael Petscavage
Luke Stolz
Khadija Walker
Abbreviations
EPA U.S. Environmental Protection Agency
NCER National Center for Environmental Research
OIG Office of Inspector General
ORD Office of Research and Development
P3 People, Prosperity and the Planet
SOP Standard Operating Procedure
STAR Science to Achieve Results
Cover Photo: The U.S. Environmental Protection Agency headquarters offices.
(EPA images)
Are you aware of fraud, waste, or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW(2431T)
Washington, D.C. 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotline@epa.gov
Learn more about our OIG Hotline.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (2410T)
Washington, D.C. 20460
(202) 566-2391
www.epa.gov/oiq
Subscribe to our Email Updates
Follow us on Twitter @EPAoig
Send us your Project Suggestions
-------
tftD srAf U.S. Environmental Protection Agency
Office of Inspector General 20-P-0204
Ł M M t> H June 30,2020
<
^7
41 PRO"^
At a Glance
Why We Did This Project
The U.S. Environmental
Protection Agency's Office of
Inspector General conducted
this audit to determine whether
(1) the Office of Research and
Development oversees and
monitors grants awarded to
universities in accordance with
applicable laws, regulations,
policies, and procedures, and
(2) university grantees are
completing agreed-upon work
that meets the defined purpose
of the grant.
The purpose of post-award
monitoring is to conduct
oversight and monitor a
recipient's performance and
management of assistance
agreements. There are two
main programmatic oversight
activities: baseline monitoring
and advanced monitoring.
This report addresses the
following:
• Operating efficiently and
effectively.
• Improving EPA research
programs.
This project addresses a key
EPA management challenge:
• Improving data quality and
filling identified data gaps.
Address inquiries to our public
affairs office at (202) 566-2391 or
OIG WEBCOMMENTS@epa.gov.
List of OIG reports.
EPA Needs to Improve Oversight of Research
Assistance Agreements
What We Found
Although the ORD has recently made improvements
to its oversight of research assistance agreements,
we found that project officers did not always complete
baseline monitoring accurately or in a timely manner,
enforce recipient compliance with progress reporting
requirements, or document the review of recipient
progress reports.
The EPA's lack of
compliance with
assistance agreement
oversight requirements
can put EPA research
funds at risk.
In addition, of the 12 sampled assistance agreements, 11 were completed and
final reports were submitted. Two of the 11 final recipient reports did not
demonstrate that agreed-upon work was completed. Also, we found that the ORD
is not consistently posting research assistance agreement results on its website.
Recommendations and Planned Agency Corrective Actions
We recommend that the assistant administrator for Research and Development:
• Verify that baseline monitoring and post-award monitoring responsibilities are
completed accurately and in a timely manner.
• Require that project officers enforce recipient reporting requirements and
document reviews of interim and final reports.
• Require that final report reviews include a narrative noting how the project
officer determined that results were achieved.
• Verify that the Grantee Research Project Results website is updated and
accurate.
• Provide training to all ORD project officers on required post-award monitoring
responsibilities.
• Update and correct the Grantee Research Project Results website for
missing and inaccurate information.
• Require that all abstracts, report summaries, and publications for all ORD
research assistance agreements be placed on the EPA's website.
The EPA agreed with our recommendations and provided acceptable planned
corrective actions and estimated completion dates. The recommendations are
resolved with corrective actions pending.
Noteworthy Achievement
The ORD's National Center for Environmental Research developed and
implemented a final report checklist following discussions with the OIG.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
June 30, 2020
MEMORANDUM
SUBJECT: EPA Needs to Improve Oversight of Research Assistance Agreements
Report No. 20-P-0204
FROM:
Sean W. O'Donnell
J
TO:
Jennifer Orme-Zavaleta, Principal Deputy Assistant Administrator for Science
and EPA Science Advisor
Office of Research and Development
This is our report on the subject audit conducted by the Office of Inspector General of the
U.S. Environmental Protection Agency. The project number for this audit was OA&E-FY18-0248. This
report contains findings that describe the problems the OIG has identified and corrective actions the OIG
recommends. Final determinations on matters in this report will be made by EPA managers in accordance
with established audit resolution procedures.
The Office of Research and Development is responsible for the issues discussed in this report.
In accordance with EPA Manual 2750, your office provided acceptable corrective actions and estimated
milestone dates in response to OIG recommendations. All recommendations are resolved and no final
response to this report is required. However, if you submit a response, it will be posted on the OIG's
website, along with our memorandum commenting on your response. Your response should be provided
as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation
Act of 1973, as amended. The final response should not contain data that you do not want to be released
to the public; if your response contains such data, you should identify the data for redaction or removal
along with corresponding justification.
We will post this report to our website at www.epa.gov/oig.
-------
EPA Needs to Improve Oversight of
Research Assistance Agreements
20-P-0204
Table of C
Chapters
1 Introduction 1
Purpose 1
Background 1
Responsible Office 5
Noteworthy Achievements 5
Scope and Methodology 6
Prior Reports 6
2 Baseline Monitoring Was Often Late and Sometimes Inaccurate 7
EPA Requires Monitoring of Assistance Agreements 7
Project Officers Did Not Comply with Monitoring Requirements 8
Staff Turnover and Lack of Administrative Focus Lead to
Higher Risk 9
Recommendation 10
Agency Response and OIG Assessment 10
3 EPA Did Not Consistently Require Timely Progress Reports and Did Not
Always Document Reviews 11
EPA Requirements for Recipient Progress Report Submission and
Program Office Review 11
Project Officers Did Not Enforce Recipient Reporting Requirements 12
Lack of Adequate Controls Increases Agency Risk 14
Recommendations 14
Agency Response and OIG Assessment 14
4 Agreed-Upon Work Not Always Completed 16
Review of Results and Closeout Certification Are Required 16
Final Reports Did Not Always Show Work Was Completed 16
Lack of Oversight Contributes to Unsupported Closeouts 17
Recommendation 18
Agency Response and OIG Assessment 18
- continued -
-------
EPA Needs to Improve Oversight of
Research Assistance Agreements
20-P-0204
5 Transparency of Research Results Needs Improvement 19
EPA Committed to Transparency 19
ORD's Website Does Not Have Complete and Accurate Information 19
Public Website Challenges Lessened Transparency 20
Recommendations 20
Agency Response and OIG Assessment 21
Status of Recommendations and Potential Monetary Benefits 22
Appendices
A ORD Assistance Agreements Reviewed by OIG 23
B Agency Response to Draft Report 24
C Distribution 28
-------
Chapter 1
Introduction
Purpose
The U.S. Environmental Protection Agency's Office of Inspector General
conducted this audit to determine whether
(1) the Office of Research and
Development oversees and monitors
grants awarded to universities, in
accordance with applicable laws,
regulations, policies, and procedures and
(2) university grantees are completing
agreed-upon work that meets the defined
purpose of the grant.
Background
Assistance agreements include grants and cooperative agreements. Grants and
cooperative agreements are appropriate funding mechanisms for the Agency when
the main purpose of the relationship is to support a public purpose authorized by
federal statute and involves "transferring money, property, services, or anything
of value to a recipient," according to the EP A's Assistance Agreement Almanac.
Cooperative agreements are used when there is substantial involvement by EPA
program personnel in the funded project. For the purpose of this audit, the
recipients were higher learning institutions.
The Role of ORD
The ORD is the EPA's scientific research arm, and its research is used to inform
decisions and support the needs of Agency offices, partners, and stakeholders. Its
work is grouped into six research programs to address the most pressing
environmental health research needs:
• Air and Energy Research.
• Chemical Safety for Sustainable Research.
• Health and Environmental Risk Assessment.
• Homeland Security Research.
• Safe and Sustainable Water Resources Research.
• Sustainable and Healthy Communities Research.
The ORD reorganized in September 2019 and now has offices and centers in
ten locations across the country, as shown in Figure 1. The reorganization
changed the structure of the centers and laboratories to combine similar work and
Key Management Challenge
This audit addresses the following key
management challenge for the Agency,
as identified in OIG Report No. 19-N-0235
Fiscal Year 2019 EPA Management
Challenges, issued July 15, 2019:
• Improving data quality and filling
identified data gaps.
20-P-0204
1
-------
remove "stovepipes." Despite the reorganization, most staff are expected to
perform the same work as they performed prior to the reorganization.
Figure 1: ORD office arid research center locations
~ Corvallis, OR Duluth, MN if
~
Narragansett, Rl ~
Edison, NJ ~
Cincinnati, OH
^ Washington, DC "A"
Research
Ada, OK Triangle Park, NC
Athens, GA
~
Gulf Breeze, FL
~
Source: EPA image modified by the OIG for accuracy.
In addition, as the national program manager for the EPA Regional Laboratories,
the ORD is responsible for the strategic direction, budget, and performance of all
the laboratories associated with each of the Agency's regional offices. The ORD's
Immediate Office of the Assistant Administrator works with the Office of Science
Information Management; the Office of Resource Management; the Office of
Science Advisor, Policy and Engagement; and the four research centers
(Figure 2).
Figure 2: ORD's new organizational structure
Center for
Environmental Solutions
& Emergency Response
Office of Resource
Management
Center for Public Health
& Environmental
Assessment
Office of Science
Information
Management
Office of Science
Advisor, Policy
& Engagement
Center for
Computational
Toxicology & Exposure
Center for
Environmental
Measurement & Modeling
ORD Assistant
Administrator
Immediate Office
Source: The EPA.
20-P-0204
2
-------
The ORD awards assistance agreements in the following research areas:
• Air.
• Climate change.
• Ecosystems.
• Health.
• Safer chemicals.
• Sustainability.
• Water.
The ORD requires that all research assistance agreement recipients submit
summaries of their progress and final reports for posting on the EPA's website.
These reports are maintained in a database that also includes the project abstracts.
According to its website, the Agency does not review the principal investigator's
conclusions published on the website because project abstracts, report summaries,
and publications "convey the viewpoints of the principal investigator and may not
represent the views and policies of ORD and EPA."
Post-Award Monitoring of Assistance Agreements
The purpose of post-award monitoring, conducted by the EPA project officers, is
to conduct oversight of a recipient's performance and management of assistance
agreements. Project officers are supervised by their program offices and are
required to follow overarching Agency policies and procedures for post-award
monitoring and oversight.
All assistance agreements have terms and conditions, such as:
• Information that must be reported by research recipients.
• When reports are to be submitted.
• Required summaries in a specific format to place on the EPA's website.
• "Sufficient progress" reference, which states that the EPA may terminate
the assistance agreement if sufficient progress is not being made.
EPA Order 5700.7A1, Environmental Results under EPA Assistance Agreements,
dated January 1, 2005, requires program offices to review both interim and final
performance reports and document these reviews in the official project file.
The EPA's Assistance Agreement Almanac requires project officers to maintain
records used for programmatic direction, such as correspondence and progress
reports.
20-P-0204
3
-------
EPA Order 5700.6A2 CHG 2, Policy on Compliance, Review and Monitoring,
dated September 24, 2007, outlines the two main post-award oversight activities:
baseline monitoring and advanced monitoring. The policy requires the results of
these monitoring activities to be recorded in the EPA's Integrated Grants
Management System databases—Post-Award and Grantee Compliance. The Post-
Award Database records milestones, activities, and baseline monitoring by award
number. The Grantee Compliance Database recorded and tracked advanced
monitoring, as well as audits and reports that are tracked by recipient. The
Agency replaced the Grantee Compliance Database with the Comply App
database, which contains similar information but has some additional features, on
July 1, 2019. The EPA's policy still refers to the Grantee Compliance Database.
Project officers conduct baseline monitoring to evaluate the progress of work and
whether the recipient of the funds has complied with the terms of the assistance
agreement and applicable regulations. To do this, project officers determine
whether:
• Progress reports have been received from the recipient.
• Expended funds are reasonable based on project duration and work plan.
• Unexpended funds are reasonable based on what is necessary to complete
the project.
• The recipient is in compliance with applicable programmatic terms and
conditions of the assistance agreement.
Project officers are responsible for conducting baseline monitoring unless they
receive a waiver from the Office of Grants and Debarment for reasons such as
unique programmatic considerations. The initial and subsequent baseline
monitoring is dependent upon the length of the original project period (Table 1).
Table 1: Baseline monitoring requirements
Project period
Initial
Ongoing
18 months or less
Due within six months after
award date.
Due within 12 months after most
recent monitoring.
Over 18 months
Due within 12 months after
award date.
Due within 12 months after most
recent monitoring.
Source: EPA Order 5700.6A2 CHG 2.
Project officers can conduct additional baseline monitoring at any time based on
amendments to the assistance agreement, receipt of progress reports, or a change
in project officer. EPA policy requires project officers to record all baseline
monitoring reports in the EPA's Post-Award Database, or the baseline monitoring
is considered not conducted.
The ORD's National Center for Environmental Research, also known as the
NCER, issued a Procedure for Annual Progress Report Reviews and Baseline
Monitoring Reports, GM-005, dated July 7, 2017. It requires project officers to
review recipient progress reports and complete an annual report checklist. It also
20-P-0204
4
-------
specifies actions that project officers must take when the recipient reports are
delayed. Prior to the reorganization, this procedure and guidance were applicable
only to the NCER because other ORD offices did not have specific grants
management policies except for those provided by the Office of Grants and
Debarment. Since the 2019 reorganization, the ORD is developing standard
operating procedures pertaining to post-award monitoring, which will apply
across the Office.
Advanced monitoring is an "in-depth assessment" of a project's progress, which:
• Requires interaction with the recipient.
• Can be conducted at the recipient's location or off-site.
• Produces a written report, which could include findings, recommended
corrective actions, and a timeline for recipient response.
While EPA policy requires baseline monitoring for each award, advanced
monitoring is not mandatory. The ORD annually develops a Post-Award
Monitoring Plan, which identifies at least 10 percent of the active recipients for
advanced monitoring during the calendar year. However, the ORD can conduct
advanced monitoring for assistance agreements that are not in the plan. For
example, project officers may request to conduct advanced monitoring if baseline
monitoring results raise concerns or warrant a comprehensive review. Project
officers can also substitute an advanced monitoring activity for a baseline
monitoring activity if the advanced monitoring activity was conducted within 12
months of the last baseline activity. If advanced monitoring is conducted, those
reports must be posted in Comply and the Post-Award Database.
The ORD EPA Project Officer Post-Award Evaluation Protocol was issued on
January 8, 2008, and project officers were strongly recommended to use it during
on-site and off-site reviews and when preparing trip reports. Effective April 18,
2018, the NCER made the protocol mandatory for its staff. As noted above, the
ORD has been developing an SOP subsequent to its September 2019
reorganization that will be required for the entire organization.
Responsible Office
The ORD is responsible for the issues discussed in this report.
Noteworthy Achievements
The ORD's NCER developed and implemented a final report checklist following
a discussion with the OIG about the annual report checklist. The NCER had also
determined that project officer performance standards will include an element for
grants management regarding standards and documentation and that supervisors
will review random project officer files.
20-P-0204
5
-------
Scope and Methodology
We conducted this performance audit from June 2018 through April 2020 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions.
To answer our objectives, we reviewed applicable regulations and EPA policies
and procedures. We obtained a list of assistance agreements the ORD awarded that
were active or closed out from October 1, 2014, through September 30, 2018 (fiscal
years 2015 through 2018). We judgmentally selected 12 out of the 245 assistance
agreements from FY 2015 through FY 2018:
• We selected two out of the 170 assistance agreements active from FY 2015
through FY 2018. See Appendix A, Samples 1 and 2, for more details.
• We selected ten out of the 75 assistance agreements that were closed
during FY 2018. See Appendix A, Samples 3 through 12, for more details.
During our audit, we sampled assistance agreements that were awarded and
monitored by one of the ORD research centers, the NCER, and two of ORD's
laboratories: the National Health and Environmental Effects Research Laboratory
and the National Risk Management Research Laboratory. As a result of the
2019 reorganization, these research centers and laboratories no longer exist in
their previous forms.
We obtained documents from the ORD and the Integrated Grants Management
System database. We interviewed management and staff within the ORD regarding
oversight activities of assistance agreements.
Prior Reports
OIG Report No. 13-P-0361. EPA Needs to Improve STAR Grant Oversight, issued
on August 27, 2013, found that project officers did not monitor Science to
Achieve Results, known as STAR, grant recipients' assistance agreements in a
manner consistent with the Agency's policy and guidance. The EPA reported that
all corrective actions were completed.
OIG Report No. 16-P-0125. EPA Offices Are Aware of the Agency's Science to
Achieve Results Program, but Challenges Remain in Measuring and Internally
Communicating Research Results That Advance the Agency's Mission, issued on
March 30, 2016, found that the ORD needs to enhance communication of STAR
grant research results. The EPA reported that all corrective actions were
completed.
20-P-0204
6
-------
Chapter 2
Baseline Monitoring Was Often Late and
Sometimes Inaccurate
EPA policy requires baseline monitoring to be completed, but project officers
have not always completed baseline monitoring accurately or in a timely manner.
The ORD attributed the noncompliance to the high turnover of project officers
who are hired as subject matter expert scientists and may not adequately focus on
administrative tasks. Deficient monitoring increases the risk that recipients of
assistance agreements will not meet their objectives, and it leaves the EPA an
inaccurate record of recipient performance, putting taxpayer funds at risk.
EPA Requires Monitoring of Assistance Agreements
EPA Order 5700.6A2 CHG 2 requires project officers to conduct the initial
baseline monitoring within 12 months of the award date for awards with a project
period exceeding 18 months, and every 12 months thereafter. If the project period
is less than 18 months, project officers are required to conduct baseline
monitoring no more than six months after the award date. No matter the project
period, project officers are responsible for conducting baseline monitoring at least
annually. Project officers are required to post baseline monitoring reports in the
Post-Award Database. Project officers can substitute advanced monitoring for
baseline monitoring, provided it is documented in both Comply and the Post-
Award Database.
In 2008, the ORD requested and received a baseline monitoring waiver from the
Office of Grants and Debarment for all People, Prosperity and the Planet, known
as P3, Phase I assistance agreements because these agreements receive a small
amount of funding and normally have a one-year project period. The P3 awards
provide funding for teams of college students to design environmental solutions
for a sustainable future. The waiver requires the ORD to conduct baseline
monitoring if the project period is extended beyond one year.
EPA Order 5700.7A1 refers to reporting performance, specifically:
All competitive funding announcements for assistance agreements
must include ranking criteria for evaluating the applicant's past
performance in reporting on outputs and outcomes.
20-P-0204
7
-------
Project Officers Did Not Comply with Monitoring Requirements
We found that project officers were late in completing baseline monitoring reports
for almost half of the sampled assistance agreements. In some instances, the
reports were inaccurate. For our sample of 12 research assistance agreements, ten
required baseline monitoring because the other two were P3 Phase I and under a
waiver. The project officers for eight of those ten assistance agreements
(80 percent) did not complete at least one baseline monitoring report by the
required due date. We found baseline monitoring reports ranged from nine to
596 days late (Table 2).
Table 2: Missing and late baseline monitoring reports for FY 2015-FY 2018
How many
How many
How late was
Sample
were
were
How many
each overdue
number
required?
completed?
were late?
report?
1
3
3
2
• 9 days
• 35 days
2
3
2
2
• 20 days
• 199 days
3
3
3
1
• 122 days
4
2
2
2
• 54 days
• 68 days
5
4
4
0
-
6
2
2
1
• 200 days
7
3
4a
0
-
8
2
1
1
• 200 days
9
3
3
1
• 13 days
10
2
1
1
• 596 days
Total
27
24
11
Percent
89%
46%
Source: OIG analysis of reporting requirements and EPA's monitoring records.
a Project officer completed four baselines although only three were required.
In addition, we found project officers had inaccurately noted on some of the
baseline monitoring reports that there were no issues with progress reports,
despite recipients not submitting them in a timely manner or including the
required annual summaries. As a result, the Agency may not have been aware of
project status or difficulties. For example:
• In one baseline monitoring report completed on January 25, 2018, the
project officer noted that the recipient was compliant with applicable
programmatic terms and conditions and had submitted progress reports as
required. However, we found that the recipient was more than five months
late in submitting a progress report that was originally due on June 30, 2017.
• Another project officer reported in the baseline monitoring report,
completed on May 4, 2017, that the recipient submitted progress reports as
required. However, we found the recipient had not, at the time the Agency
20-P-0204
8
-------
completed its baseline monitoring report, submitted the annual progress
report that was originally due on September 28, 2016. In fact, the recipient
submitted the annual progress report on August 30, 2017, almost a year
after the due date.
During our audit, ORD management recognized grants management as an area for
improvement. The NCER was responsible for the majority of ORD assistance
agreement awards prior to the Office's reorganization. According to the ORD,
starting February 2019, NCER management began reviewing post-award
monitoring status reports to address monitoring concerns during monthly business
review meetings as well as identify any issues with post-award monitoring
deadlines. Also starting in FY 2019, according to the ORD, NCER management
included grants management in project officers' performance standards—such as
adhering to standards, requirements, and policies for supporting documentation.
Results of supervisory reviews would help identify missing file contents, project
officer training needs, and topics to include in regular project officer meetings.
However, we noted that the ORD had not yet instituted internal controls to
address inaccurate baseline monitoring.
Staff Turnover and Lack of Administrative Focus Lead to Higher Risk
ORD management attributed the delays and inaccuracies in completing baseline
monitoring reports to increased turnover due to retirements and having fewer
project officers. Also, ORD management stated that project officers are hired as
subject matter expert scientists who may not adequately focus on administrative
tasks. However, this lack of focus on administrative tasks makes it critical for the
ORD to establish and enforce strong internal controls. Baseline and advanced
monitoring of assistance agreements provide direct oversight of federal dollars to
verify that they are properly spent. Deficient monitoring increases the risk that
assistance agreement recipients will not meet their objectives and that taxpayer
funds may be subject to waste, fraud, and abuse.
When project officers do not accurately document recipients" performance, the
EP A will not have an accurate record of performance information to aid in future
competitive assistance agreements awards. The EPA's Grants Competition
Advocate's Guidance on the Evaluation of Applicants Past Performance in
Managing Grants and Reporting on Outputs and Outcomes, dated January 2014,
states that to evaluate past performance, the EPA should focus on evaluating how
well applicants documented or reported their progress toward achieving expected
outputs and outcomes under prior assistance agreements, such as filing timely
progress and final reports.
Furthermore, without EPA project officer oversight, these recipients might not be
aware that they have performance issues to disclose in future applications. As a
result, recipients with undocumented performance issues may be unfairly
20-P-0204
9
-------
considered for future awards over applicants without negative performance
histories.
Recommendation
We recommend that the assistant administrator for Research and Development:
1. Implement additional internal controls across the Office of Research and
Development to verify project officers complete baseline monitoring and
post-award monitoring responsibilities accurately and in a timely manner.
Agency Response and OIG Assessment
The Agency concurred with Recommendation 1. The ORD will implement
additional internal controls by developing and implementing an SOP for all ORD
assistance agreements that will codify how to accurately complete and document
baseline and post-award monitoring and will include timeliness metrics. Project
officers will be evaluated for their adherence with the SOP and monitoring due
dates during the annual performance appraisal cycle. The development and
implementation of the procedure and verification of compliance are internal
controls that meet the intent of the recommendation. The EPA provided an
estimated completion date of October I, 2020. We consider this recommendation
resolved with corrective action pending. See Appendix B for the Agency's
response to the draft report.
20-P-0204
10
-------
Chapter 3
EPA Did Not Consistently Require Timely Progress
Reports and Did Not Always Document Reviews
Project officers did not always enforce recipient compliance with progress
reporting requirements, and the program office did not always document reviews
of recipient progress reports. Although EPA policy and award agreements require
recipient reporting and EPA review of reports, the Agency does not have adequate
controls to verify that those steps take place. When project officers do not enforce
recipient reporting requirements, the Agency may not be able to determine
whether sufficient progress is being made. Also, when program offices do not
document reviews of progress reports, it is unknown if the program offices
reviewed progress reports as required.
EPA Requirements for Recipient Progress Report Submission and
Program Office Review
EPA award agreements include a condition that specifies when reports are due
and requires research recipients to report on specific elements regarding their
progress. Depending on the type of assistance agreement awarded, recipient
progress report due dates vary (Table 3).
Table 3: Sampled research assistance agreement reporting requirements
Assistance program
Progress reports
Final report
P3 Phase I
March
None
STAR
Annually
Within 90 days of expiration date
Consolidated research
Quarterly or annually
Within 90 days of expiration date
Source: OIG analysis of reporting requirements.
Since January 2014, the EPA's "General Terms and Conditions" states that the
EPA may terminate the assistance agreement if the Agency determines there is a
"failure to ensure reasonable completion of the project within the project period."
Since September 2013, the "EPA Research and Related Agency Specific
Requirements" has included the following statement:
EPA may withhold payment if progress reports are not submitted
by the due date. In addition, if EPA determines that the recipient
has not made sufficient progress toward completing its research,
EPA may terminate the assistance agreement.
"EPA Research and Related Agency Specific Requirements" are not applicable to
research centers, conferences, training projects, fellowships, or P3 awards.
20-P-0204
11
-------
EPA Order 5700.7A1 requires program offices to review both interim and final
performance reports and document them in the official project file. Additionally,
the EP A's Assistance Agreement Almanac requires project officers to maintain
records of documents used for programmatic direction, such as correspondence
and progress reports.
The NCER's GM-005 requires project officers to complete an annual report
checklist. This procedure was updated on August 1, 2019, to include a final report
checklist. As a result of the 2019 reorganization, the ORD is developing SOPs for
post-award monitoring, which will apply across the Office.
Project Officers Did Not Enforce Recipient Reporting Requirements
We found that nine out of 12 (75 percent) sampled assistance agreements had at
least one late or missing report. Seven of those nine recipients (78 percent)
requested project period extensions to complete their work. With late or missing
reports, recipients may not have informed EPA project officers in a timely manner
of any potential delays or obstacles in completing grant objectives or work
(Table 4).
Table 4: Sampled research assistance agreement reporting
Sample
Reports
Reports
Late
Reports with
Time extensions
number
required
submitted
reports
Days late
missinq elements
awarded
• 3
1
4
4
3
• 1
• 2
4
1
2
3
3
2
• 165
• 26
3
0
3
3
3
2
• 711
• 134
1
2
• 59
4
4
4
4
• 19
• 146
• 1
4
1
5
4
2
1
• 26
0
1
6
4
3
2
• 34
• 32
3
1
7
3
3
1
• 336
3
2
• 48
8
11
5
4
• 97
• 88
• 69
5
0
• 761
9
4
4
4
• 165
• 43
• 2
4
1
10
1
2
0
• -
0
2
11
1
1
0
• -
0
0
12
1
1
0
• -
0
0
Total
43
35 (81%)
23 (66%)
1-761 days
(range)
27
11
Source: OIG analysis of sampled assistance agreement files for report oversight, October 2014-September 2018.
20-P-0204 12
-------
Project officers did not enforce recipient reporting requirements for submission
dates and reporting elements. Also, some recipients have requested and received
extensions or additional funds without explaining such requests in the project
reports. For example:
• One recipient was required to submit quarterly reports that included
progress, results, and planned activities. However, the submitted reports
only included expenditure information, student names, faculty advisors,
and EPA mentors. In addition, one report covered a 12-month period,
which did not comply with the requirement to submit quarterly reports.
• For another recipient, the project officer discovered that an annual
progress report was missing when preparing to close out the assistance
agreement, more than one year after the annual progress report was due.
The project officer followed up with the recipient about the missing report,
and the recipient submitted it almost two years after it was due.
• One recipient did not submit the annual report summary, which is required
in addition to the annual progress report. The project officer did not
discover the missing summary until preparing to upload it to the NCER's
website after it was due. The project officer followed up with the recipient,
and the recipient submitted it more than two years after it was due.
Not only did recipients submit their progress reports late according to the
reporting requirements of the award agreement, but the EPA program office also
did not always document its reviews of these reports as required by EPA
Order 5700.7A1 (Figure 3).
Figure 3: Percent of reports with documented reviews for sampled assistance
agreements
E
ns
(D
1
2
3
4
5
6
7
8
9
10
11 0%
12 0%
67%
33%
50%
100%
0%
100%
100%
100%
100%
0 20 40 60
Percentage
Source: OIG analysis of ORD documentation.
80
100
20-P-0204
13
-------
Prior to the issuance of GM-005, project officers were only required to print a
copy of the progress report and annotate it with their signature or initials and the
date. As of July 7, 2017, the ORD's NCER instituted an annual report review
checklist for NCER assistance agreements. Three of the progress reports were
received after that, but the project officers did not complete the new required
checklist.
Lack of Adequate Controls Increases Agency Risk
The EPA does not have adequate controls to verify that project officers are
enforcing recipient reporting requirements such as submission dates, required
reporting elements, and an annual summary format. The NCER project officers
did not complete the required checklists for reasons such as lack of training.
When the ORD does not enforce reporting requirements, it increases the risk that
the Agency will be unaware of project delays and lack of progress. Project
officers should enforce terms and conditions such as withholding payment for late
reports or terminating the assistance agreement due to lack of progress.
Recommendations
We recommend that the assistant administrator for Research and Development:
2. Develop and implement an Office of Research and Development policy
establishing internal controls to enforce recipient reporting requirements
and document project officer reviews of interim and final reports.
3. Provide training to all Office of Research and Development project officers
on required post-award monitoring responsibilities.
Agency Response and OIG Assessment
The Agency concurred with Recommendations 2 and 3.
For Recommendation 2, the ORD will revise the interim and final report checklist
to detail proper documentation of the reviews conducted by the project officers to
ensure all goals are met. This will be codified—or implemented—in an SOP. The
procedure will apply to all ORD project officers who manage assistance
agreements. Adherence with the procedure will be evaluated as part of the project
officers' annual performance review cycle. The OIG has reviewed the checklist
and finds that the development and implementation of the procedure and the
verification of compliance are internal controls that meet the intent of the
recommendation.
For Recommendation 3, the ORD will develop and deploy project officer training
on post-award monitoring requirements and responsibilities, including
20-P-0204
14
-------
recommendations from this report. During a meeting with the OIG on May 13,
2020, the ORD confirmed that the training is mandatory and attendance will be
tracked.
The EPA provided an estimated completion date of October 1, 2020, for both
recommendations. We consider these recommendations to be resolved with
corrective actions pending. See Appendix B for the Agency's response to the draft
report.
20-P-0204
15
-------
Chapter 4
Agreed-Upon Work Not Always Completed
Of the 12 sampled assistance agreements, 11 were completed and required final
reports. One of the sampled assistance agreements has a project end date of
March 31, 2022. However, two of the 11 sampled final reports—encompassing
$1.8 million—did not demonstrate that agreed-upon work was completed.
Program offices are required to review performance reports to determine whether
the recipient achieved outputs and outcomes, and project officers must certify at
closeout that all required programmatic work has been satisfactorily completed.
The project officers' lack of oversight and incomplete documentation resulted in
closeout decisions that could not always be supported. The EPA does not have
assurance that tax dollars have been spent wisely when final reports do not clearly
discuss successes and failures, and project officers do not include a narrative with
closeout certifications.
Review of Results and Closeout Certification Are Required
EPA Order 5700.7A1 requires program offices to review both interim and final
performance reports to determine whether the recipient achieved outputs and
outcomes per the assistance agreement work plan. Work plan documents should
specify outputs and outcomes for each assistance agreement.
EPA Order 5700.6A2 CHG 2 states that at the completion of an assistance
agreement, project officers must "certify that all programmatic terms and
conditions are met." Also, the EPA has developed a Project Officer Closeout
Certification form that requires the project officer to certify that required
programmatic work was satisfactorily completed.
Final Reports Did Not Always Show Work Was Completed
As shown in Figure 4, the final reports for two out of the 11 sampled assistance
agreements did not demonstrate that agreed-upon work was accomplished.
Figure 4: OIG analysis of final reports
Two awards
($1.8 million)
did not demonstrate
that agreed-upon work
was accomplished
Source: OIG analysis of recipient final reports.
Nine awards
($23.7 million)
demonstrated that
agreed-upon work
was accomplished
20-P-0204
16
-------
The two reports include the following:
• One recipient's goal was to end hunger, homelessness, and unemployment
among veterans by developing an aquaponics demonstration unit that
combines aquaculture and hydroponics to raise fish and grow vegetables.
The recipient experienced setbacks and was not able to complete and test
the demonstration unit as expected in the first phase of the one-year P3
agreement. Therefore, the work was only partially complete despite the
project officer's certification and acceptance of the final technical report.
• A training program was designed to produce environmental scientists and
engineers through educational experiences at a federal research facility. In
the program, undergraduate and graduate students worked at an EPA
laboratory. The recipient only included limited information in the final
report and did not include required elements such as project activities over
the entire funding period, achievements, or technical details of the project,
both positive and negative. It was unclear whether agreed-upon work was
completed. Despite the lack of formal, required documentation, the project
officer answered "yes" on the closeout certification in response to the
question as to whether programmatic work was satisfactorily completed.
Lack of Oversight Contributes to Unsupported Closeouts
The project officers' lack of oversight and incomplete records resulted in closeout
decisions that cannot always be supported or lacked required documentation. The
ORD, however, explained that project officers may rely on progress reports and
attend annual meetings to monitor research progress. We concur that review of
progress reports and attendance at meetings may enable oversight, but those
actions, if they occurred, were not documented.
According to the ORD, research assistance agreement recipients may not
accomplish all tasks. The ORD stated that:
Research grants are made to carefully selected investigators to
explore questions and generate new insights. Most planned tasks
are completed. Some tasks don't work out the way we expect, but
we still learn something from trying.. .Project Officers are trained
as scientists and work with [Principal Investigators] to understand
the successes and "failures" and help identify the lessons to be
learned from the on-going research.
20-P-0204
17
-------
We agree with the ORD's view that research objectives may embrace broad and
sweeping goals, such as new insights and lessons learned. However, we maintain
that it is especially important for final report reviews to be comprehensive enough
to determine whether recipients achieved objectives and captured lessons learned
when tasks are not completed.
The ORD NCER's procedures do not address final report reviews. While the OIG
concluded that the ORD's annual report checklist was helpful, we noted that there
was not a similar checklist for final reports. The OIG discussed this with the
ORD, who then updated the NCER's GM-005 on August 1, 2019. The updated
GM-005 incorporated a final report checklist, but it does not require a narrative.
The ORD stated that due to the reorganization, it is developing SOPs for post-
award monitoring that will apply across the Office. Also, the ORD noted that the
upcoming reorganization is expected to provide increased stability and continuity
for project officers, as well as better workload oversight and work practice
consistency.
When it is unclear whether results have been achieved, the EPA cannot assure that
tax dollars have been spent wisely. Some research assistance agreements exceed
five years, and with project officer turnover and minimal documentation, it is
difficult to determine the results obtained from the funded research. As a result,
clear and complete documentation is especially important.
Recommendation
We recommend that the assistant administrator for Research and Development:
4. Implement controls throughout the Office of Research and Development
to require that final report reviews include a narrative noting how the
project officer determined that results were achieved.
Agency Response and OIG Assessment
The Agency concurred with Recommendation 4. The ORD will implement a
control by revising the interim and final report checklist to include guidance that
specifies how project officers are to "document their evaluation of final and interim
reports to ensure that goals are met." The checklist and guidance will be codified—
or implemented—in an SOP that will apply to all ORD project officers who
manage assistance agreements. Adherence with the procedure will be evaluated as
part of the project officers' annual performance review cycle. During a meeting
with the OIG on May 13, 2020, the ORD confirmed that the checklist will include a
required narrative field. The SOP, required narrative, and verification of
compliance are internal controls that meet the intent of the recommendation. The
EPA provided an estimated completion date of October 1, 2020. We consider this
recommendation to be resolved with corrective actions pending. See Appendix B
for the Agency's response to the draft report.
20-P-0204
18
-------
Chapter 5
Transparency of Research Results Needs
Improvement
The ORD is not consistently reporting research assistance agreement results on its
website. The FY 2018-2022 U.S. EPA Strategic Plan states that the Agency "will
increase transparency with industry, environmental groups, and other
stakeholders." According to ORD management, a backlog to post information on
the website was formed after an open position was not immediately filled. The
EPA is not being completely transparent when its website does not make relevant
information available to the public.
EPA Committed to Transparency
Objective 2.2 of the Strategic Plan, "Increase Transparency and Public
Participation," states that the "EPA will increase transparency with industry,
environmental groups, and other stakeholders." In addition, the EPA's
programmatic reporting conditions require recipients to submit report summaries
to place on the EPA's website.
ORD's Website Does Not Have Complete and Accurate Information
The ORD is not consistently reporting research results on its public website. It
places only NCER project abstracts, annual and final report summaries, and lists
of publications onto the Grantee Research Project Results website. This is despite
the fact that the ORD requires all recipients to submit report summaries. Of the
12 assistance agreements we sampled, six were missing report summaries or had
inaccurate information on the website (Table 5).
Table 5: FY 2015-2018 research report summaries missing from website
Information found on website
Sample
number
Project
abstracts
Annual
report
Final
report
Publication
lists
Explanation
1
~
~
~
~
N/A
2
X
X
N/A
X
Nothing on website on
10/24/19. Added later.3
Missing 2016 and final
3
V
X
X
V
report summaries for
individual subprojects.
4
V
V
V
V
N/A
2016 report summaries
5
V
X
V
V
for individual subprojects
missing.
6
V
V
V
V
N/A
7
•/
•/
•/
•/
N/A
20-P-0204
19
-------
Sample
number
Information found on website
Explanation
Project
abstracts
Annual
report
Final
report
Publication
lists
8
X
N/A
X
X
Nothing on website on
10/24/19. Added later.3
9
V
V
V
X
Publication linked to
assistance agreement
but not EPA-funded.
10
V
V
X
N/A
Recipient submitted two
reports; website includes
three.
11
V
N/A
V
N/A
N/A
12
•/
N/A
•/
N/A
N/A
Source: OIG analysis of recipient reports and website (/ = found on website; X = missing or
inaccurate).
a The Grantee Research Projects Results website only includes abstracts, report summaries,
and publications for the NCER's assistance agreements. There is no website for the results of
the National Health and Environmental Effects Research Laboratory and National Risk
Management Research Laboratory assistance agreements.
For example, the 2016 annual assistance agreement report summary was missing
for the main research center as well as the individual subprojects of the main
research center. The EPA website was partially corrected after we notified the
ORD of the discrepancy. The ORD added the main research center report
summary, but the report summaries for the individual subprojects were not added
to the website. As a result, it appears that report summaries were not provided in
2016 for individual subprojects.
Public Website Challenges Lessened Transparency
According to ORD management, a backlog was created after the website manager
retired. The ORD pushed to update the website after the new person was hired,
but the backlog was long at that point. The OIG noted that the abstracts and
results of the National Health and Environmental Effects Research Laboratory
and National Risk Management Research Laboratory assistance agreements were
not on the website, and they have since been added. The EPA is not being
completely transparent when its website does not make relevant information
available to the public.
Recommendations
We recommend that the assistant administrator for Research and Development:
5. Develop and implement a policy that includes internal controls to
semiannually verify that the Grantee Research Project Results website is up-
to-date and accurate.
20-P-0204
20
-------
6. Update and correct the Grantee Research Project Results website for
missing and inaccurate information.
7. Require all abstracts, report summaries, and publications for all Office of
Research and Development research assistance agreements be placed on the
EPA's website.
Agency Response and OIG Assessment
The Agency concurred with Recommendations 5, 6, and 7.
For Recommendation 5, the ORD is developing a new policy to update the
Grantee Research Project Results website within 120 days of the project period
end. In addition, a monthly report of upcoming and late website updates is created
and provided to ORD project officers and supervisors. The monthly report was
developed in response to our audit. Adherence with the policy will be evaluated as
part of the project officers' annual performance review cycle. The development of
the policy and verification of compliance are internal controls that meet the intent
of the recommendation.
For Recommendation 6, the ORD stated that all of its active grants were updated
and accurate on the Grantee Research Project Results website. As noted above,
the ORD is developing a policy and created a monthly report to verify the website
is up to date within 120 days of the project period end. During a meeting with the
OIG on May 13, 2020, the ORD confirmed the missing and inaccurate
information identified in this report will be corrected on the website.
For Recommendation 7, the ORD will ensure abstracts and report summaries are
uploaded within 120 days of the end of the project period.
During a meeting with the OIG on May 13, 2020, the ORD confirmed that the
website policy will be codified. The ORD also clarified that the website would be
updated within 120 days of the end of each project reporting period.
The EPA initially provided an estimated completion date of July 1, 2020, for
these three recommendations. In a June 1, 2020 email, the EPA provided an
updated completion date of October 1, 2020. We consider these recommendations
resolved with corrective actions pending. See Appendix B for the Agency's
response to the draft report.
20-P-0204
21
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
Potential
Planned Monetary
Rec. Page Completion Benefits
No. No. Subject Status1 Action Official Date (In $000s)
10 Implement additional internal controls across the Office of
Research and Development to verify project officers complete
baseline monitoring and post-award monitoring responsibilities
accurately and in a timely manner.
14 Develop and implement an Office of Research and Development
policy establishing internal controls to enforce recipient reporting
requirements and document project officer reviews of interim and
final reports.
14 Provide training to all Office of Research and Development
project officers on required post-award monitoring
responsibilities.
18 Implement controls throughout the Office of Research and
Development to require that final report reviews include a
narrative noting how the project officer determined that results
were achieved.
Assistant Administrator for 10/1120
Research and
Development
Assistant Administrator for 10/1120
Research and
Development
Assistant Administrator for 10/1120
Research and
Development
Assistant Administrator for 10/1120
Research and
Development
20 Develop and implement a policy that includes internal controls to
semiannually verify that the Grantee Research Project Results
website is up-to-date and accurate.
21 Update and correct the Grantee Research Project Results
website for missing and inaccurate information.
21 Require all abstracts, report summaries, and publications for all
Office of Research and Development research assistance
agreements be placed on the EPA's website.
Assistant Administrator for 10/1120
Research and
Development
Assistant Administrator for 10/1120
Research and
Development
Assistant Administrator for 10/1120
Research and
Development
1C = Corrective action completed.
R = Recommendation resolved with corrective action pending.
U = Recommendation unresolved with resolution efforts in progress.
20-P-0204 22
-------
Appendix A
ORD Assistance Agreements Reviewed by OIG
Sample
number
Award
number
Applicant
Project title
Assistance
program
ORD office
Start
date
End
date
Award
amount
Closeout
date
1
83575201
University of
Colorado
Climate Change in Colorado:
Weatherization and Indoor Air
STAR
NCER
11/01/14
06/30/18
$999,889
4/9/19
2
83578501
University of
North Carolina
at Chapel Hill
Human Health Effects of
Environmental Pollution
Consolidated
Research
National
Health and
Environmental
Effects
Research
Library
04/01/15
03/31/22
5,327,650
Not
closed
3
83451301
University of
California,
Berkeley
University of California Berkeley -
Children's Environmental Health
Research
STAR
NCER
08/01/09
07/31/17
3,585,482
8/7/18
4
83543801
Colorado State
University
Climate & Health Benefits of
Improved Cookstoves
STAR
NCER
09/01/13
08/31/17
1,511,072
5/14/18
5
83479901
Emory
University
Emory/Georgia Tech
Collaborative: Assessment-
Health Effects
STAR
NCER
01/01/11
12/31/16
7,999,356
3/16/18
6
83479701
Michigan State
University
Michigan State University -
GLACIER
STAR
NCER
01/01/11
12/31/16
7,994,014
3/7/18
7
83528401
University of
Maryland
College Park
Community Stormwater
Management for Chesapeake
Bay
STAR
NCER
07/01/12
06/30/17
647,496
10/12/17
8
83558601
University of
Cincinnati
Research Training for College &
University Students
Consolidated
Research
National Risk
Management
Research
Library
05/01/14
04/30/17
1,799,704
2/22/18
9
83555401
Trustees of the
University of
Pennsylvania
Enabling Green Infrastructure
Investment in Philadelphia
STAR
NCER
11/01/13
10/31/17
979,246
6/23/18
10
83571301
John Brown
University
Development of an Affordable
Solar Thermal Pasteurizer
P3
NCER
09/01/14
08/31/17
14,520
3/23/18
11
83676501
Fashion
Institute of
Technology
Threading the Needle:
Composting to Colorants
P3
NCER
10/01/16
09/30/17
15,000
4/24/18
12
83612801
Georgia Tech
Research
Corporation
Sustainable Aquaponic Systems
for Inner Cities
P3
NCER
11/01/15
10/31/18
15,000
3/20/18
Source: EPA assistance agreement data.
20-P-0204 23
-------
Appendix B
Agency Response to Draft Report
/'os\
* o \
May 1, 2020
MEMORANDUM
SUBJECT: Response to Office of Inspector General (OIG) Draft Report No.
OA&E-FY18-0248 "EPA Needs to Improve Oversight of Research Assistance
Agreements" dated April 8, 2020
FROM: Jennifer Orme-Zavaleta
Principal Deputy AssistantfAdmfriistrator for Science
Office of Research and Development
TO: Sean W. O'Donnell
Inspector General
Office of Inspector General
The EPA's Office of Research and Development (ORD) appreciates the opportunity to review
and comment on the OIG's Draft Report titled "EPA Needs to Improve Oversight of Research
Assistance Agreements" (Project No. OA&E-FY18-0248 Draft Report).
We are grateful for the OIG's acknowledgement that ORD has made significant improvements in
its oversight of assistance agreements. The notable achievements identified in the report by the
National Center for Environmental Research (NCER) have been expanded and improved upon in
the new ORD organizational structure following the ORD reorganization in October of 2019.
Management of federal resources is critically important to ORD, and we welcome the OIG
recommendations to ensure that our grant program is fiscally responsible, scientifically sound,
and administered with the highest degree of integrity.
Our comments and suggestions are intended to promote accuracy and clarity of the final product.
Immediately below are ORD's responses to the OIG's specific recommendations identified in the
draft report. The attachment provides additional detailed comments, including specific language
suggestions and recommendations to improve accuracy.
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
20-P-0204
24
-------
Recommendation 1: Implement additional internal controls across the Office of Research
and Development to verify project officers complete baseline monitoring and post-award
monitoring responsibilities accurately and in a timely manner.
Response 1: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 1: ORD will develop and implement a standard operating procedure
(SOP) that codifies how to accurately complete and document baseline and post-award
monitoring including metrics for timeliness. This SOP will apply to all assistance
agreements managed throughout ORD. A quarterly report will be utilized by project
officers and their supervisors to identify upcoming baseline monitoring and post-award
monitoring due dates for verification of timeliness. The degree to which project officers
adhere to the SOP and due dates will be evaluated as part of the annual PARS cycle.
Planned Completion Date: October 1, 2020
Recommendation 2: Develop and implement an Office of Research and Development policy
establishing internal controls to enforce recipient reporting requirements and document
project officer reviews of interim and final reports.
ORD Response 2: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 2: ORD has already developed a checklist that specifies reporting
requirements for interim and final reports. This checklist will be revised to also specify
proper documentation of project officer reviews ensuring all goals are met. This checklist
and guidance will be codified in an SOP that will apply to all project officers managing
assistance agreements in ORD. The degree to which project officers adhere to the SOP
will be evaluated as part of the annual PARS cycle.
Planned Completion Date: October 1, 2020
Recommendation 3: Provide training to all Office of Research and Development project
officers on required post-award monitoring responsibilities.
ORD Response 3: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 3: ORD will develop and deploy training for project officers on the
requirements and responsibilities associated with post-award monitoring. In addition, the
20-P-0204
25
-------
training will cover the recommendations outlined in the draft OIG report including
standard operating procedures developed in response to those recommendations.
Planned Completion Date: October 1, 2020
Recommendation 4: Implement controls throughout the Office of Research and
Development to require that final report reviews include a narrative noting how the project
officer determined that results were achieved.
ORD Response: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective action 4: As described in Corrective Action 2, the checklist developed to
evaluate final and interim reports will include guidance specifying how project officers
should document their evaluation of final and interim reports to ensure that goals are met.
The degree to which project officers adhere to the SOP will be evaluated as part of the
annual PARS cycle.
Planned Completion Date: October 1, 2020
Recommendation 5: Develop and implement a policy that includes internal controls to
semiannually verify that the Grantee Research Project Results website is up-to-date and
accurate.
ORD Response: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 5: ORD developed a policy that Grantee Research Project Results are
updated within 120 days of the end of a grant project period. A monthly report is created
that identifies upcoming and late website updates. The monthly report will be shared with
ORD project officers and their supervisors. The degree to which project officers adhere to
the SOP will be evaluated as part of the annual PARS cycle.
Planned Completion Date: July 1, 2020
Recommendation 6: Update and correct the Grantee Research Project Results website for
missing and inaccurate information.
ORD Response: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 6: ORD has up to date and accurate Grantee Research Project Results
for all its active grants and has developed a policy and monthly report that verifies that
20-P-0204
26
-------
Grantee Research Project Results are updated within 120 days of the end of a grant
project period. This will ensure that all active ORD grants have up to date and correct
Grantee Research Project Results.
Planned Completion Date: July 1, 2020
Recommendation 7: Require all abstracts, reports, and publications for all Office of
Research and Development research assistance agreements be placed on the EPA's website.
ORD Response: ORD concurs with this recommendation and proposes the following
corrective action and completion date.
Corrective Action 7
ORD will ensure that abstracts and report summaries for ORD research assistance
agreements will be uploaded on EPA's website within 120 days of the end of a grant
project period.
Planned Completion Date: July 1, 2020
Attached please find specific comments on the Draft Report. We request the OIG include our full
response to the Draft Report, including the attachment of detailed technical comments. If you
have any questions regarding this response, please contact Kelly van Bronkhorst, Office
of Research and Development, Office of Program Accountability and Resource
Management at 202-566-2907.
OIG Response: We made the necessary changes to incorporate the technical comments into
the final report. As a result, the ORD responded that the detailed technical comments no
longer needed to be included as an attachment to the final report.
Attachment
cc: Heather Cursio
Mary Ross
Kathleen Deener
James Gentry
Maggie LaVay
Michael Hiscock
Yvonne Murphy
Michael Boucher
Kelly van Bronkhorst
Patrick Gilbride, OIG
Madeline Mullen, OIG
20-P-0204
27
-------
Appendix C
Distribution
The Administrator
Assistant Deputy Administrator
Associate Deputy Administrator
Chief of Staff
Deputy Chief of Staff/Operations
Assistant Administrator for Research and Development and EPA Science Advisor
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Director, Office of Continuous Improvement, Office of the Administrator
Principal Deputy Assistant Administrator for Science, Office of Research and Development
Associate Director for Science, Office of Research and Development
Deputy Assistant Administrator for Management, Office of Research and Development
Deputy Assistant Administrator for Science Policy, Office of Research and Development
Associate Assistant Administrator for Research and Development
Director, Office of Grants and Debarment, Administration and Resource Management Offices,
Office of Mission Support
Director, Grants and Interagency Agreements Management Division, Administration and
Resource Management Offices, Office of Grants and Debarment, Office of Mission Support
Audit Follow-Up Coordinator, Office of the Administrator
Audit Follow-Up Coordinator, Office of Research and Development
20-P-0204
28
------- |