j
\
\
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Audit Report
EPA Actions Should Lead to
Improved Grants Accountability
Report No. 08-P-0276
September 24, 2008
-------
Report Contributors: Andres Calderon
Randy Holthaus
Jennifer Hutkoff
Abbreviations
EPA U.S. Environmental Protection Agency
GAO Government Accountability Office
MATS Management Audit Tracking System
OARM Office of Administration and Resources Management
OGD Office of Grants and Debarment
OIG Office of Inspector General
PARS Performance Appraisal and Recognition System
PO Project Officer
-------
,V16D Sr/|,
I
5
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
08-P-0276
September 24, 2008
Catalyst for Improving the Environment
Why We Did This Review
We performed this audit to
determine what corrective
actions the U.S.
Environmental Protection
Agency (EPA) took as a result
of our 2005 audit of grants
accountability.
Background
In February 2005, the
Chairman of the House
Committee on Transportation
and Infrastructure asked the
Office of Inspector General
(OIG) to evaluate whether
EPA held supervisors and
their project officers
accountable for grants
management responsibilities.
The resulting review, dated
September 27, 2005, identified
a number of continuing grants
management weaknesses. In
response to the OIG's report,
EPA provided a corrective
action plan with milestones for
completing the plan steps.
EPA Actions Should Lead to Improved
Grants Accountability
What We Found
EPA implemented the corrective action plan it prepared in response to the
September 2005 OIG report on grant accountability. In the 2005 report, we
recommended that EPA establish a process to measure project officer, supervisor,
and manager performance against grants management requirements. EPA
established a process for measuring project officer performance, including
quantitative performance measures such as the average number of days to
transmit funding recommendations and the number of baseline monitoring
activities for active awards. The 2005 report recommended that EPA ensure
managers and supervisors discuss grants management during performance
reviews. Surveys of project officers and supervisors indicate that the discussions
are occurring.
The 2005 report also recommended that EPA ensure weaknesses identified in
management reviews are communicated to project officers. Office of Grants and
Debarment provided program offices and regions with the results of grants
management reviews and comprehensive performance reviews with the
instruction to distribute the information to supervisors and project officers. EPA's
actions should lead to improvements in managing assistance agreements.
This report contains no recommendations. In its response, EPA stated that it will
continue to work with the grants management community to ensure that grants
accountability is a focal point in managing assistance agreements.
For further information,
contact our Office of
Congressional and Public
Liaison at (202) 566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2008/
20080924-08-P-0276.pdf
-------
\ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
S WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
September 24, 2008
MEMORANDUM
SUBJECT: EPA Actions Should Lead to Improved Grants Accountability
Report No. 08-P-0276
FROM: Melissa M. Heist
Assistant Inspector General for Audit
TO: Luis Luna
Assistant Administrator for Administration and Resources Management
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This audit report represents the opinion of the
OIG and contains no recommendations. The estimated cost of this report - calculated by
multiplying the project's staff days by the applicable daily full cost billing rates in effect at the
time-is $200,145.
Action Required
In accordance with EPA Manual 2750, we are closing this report on issuance in our tracking
system. No further action is necessary.
If you or your staff has any questions regarding this report, please contact me at (312) 886-3059 or
kasper.janet@epa.gov: or Randy Holthaus, Project Manager, at (214) 665-6620 or
holthaus.randy@epa.gov.
-------
EPA Actions Should Lead to Improved 08-P-0276
Grants Accountability
Table of Contents
Purpose 1
Background 1
Scope and Methodology 2
Results 2
Conclusion 4
Agency Response 4
Status of Recommendations and Potential Monetary Benefits 5
Appendices
A Details on Scope and Methodology 6
B Agency Response 8
C Distribution 9
-------
08-P-0276
Purpose
The Office of Inspector General (OIG) issued an audit report entitled EPA Managers Did Not
Hold Supervisors and Project Officers Accountable for Grants Management, 2005-P-00027, on
September 27, 2005. During the course of that audit, the OIG found that EPA managers were
not holding personnel accountable for grants management.
We conducted this follow-up review to determine whether EPA followed through in completing
the corrective action plan steps presented in response to our 2005 audit report and, ultimately,
whether EPA addressed the recommendations.
Background
In March 2003, we reported that EPA leadership did not always stress the importance of
project officer duties and did not hold project officers accountable for conducting complete pre-
award reviews. As a result, there was insufficient assurance that proposed costs were reasonable,
that recipients were technically capable to perform the work, and that the projects would
accomplish program objectives or achieve desired environmental results. We reported that it is
crucial that EPA creates an environment where the management of assistance agreements1 and
the project officer function are considered vital to EPA's mission.
Project officers' duties include conducting competitions, evaluating proposals, and
recommending and overseeing assistance agreements. In 2006, 72 percent of project officers
stated that they spent greater than 10 percent of their time on project officer responsibilities. As
of March 2008, EPA had 1,455 active project officers managing grants and over 70 percent of
those project officers managed 2 or more grants.
In February 2005, the Chairman of the House Committee on Transportation and Infrastructure
requested that the OIG examine accountability in EPA's grants management process. The OIG
determined that EPA had not completely established a system of accountability for grants
management and that the systemic weaknesses that had plagued the Agency for several years
continued to exist. The OIG made these three recommendations in that 2005 audit report.
1. Establish a process to measure project officer, supervisor, and manager performance
against grant management requirements to form the basis for performance ratings and
discussions.
2. Ensure managers and supervisors review and discuss grants management during
performance evaluations as appropriate.
3. Ensure that the weaknesses identified in a management review or self-assessment are
communicated to the appropriate project officer and supervisor.
For purposes of this report, we will use the terms "grant" and "assistance agreement" interchangeably.
-------
08-P-0276
As a result, EPA developed a corrective action plan designed to address these report
recommendations.
Throughout our prior audit, we found that systemic grants management weaknesses continued to
exist. In 2007, the OIG removed the use of assistance agreements to accomplish the Agency's
mission from its list of key Agency management challenges. The OIG believed that the Agency
had taken substantial actions to improve its management of assistance agreements through
updated policies, increased training, and improved accountability.
Scope and Methodology
We performed our audit in accordance with generally accepted government auditing standards
issued by the Comptroller General of the United States. We conducted out audit field work from
February 2008 to June 2008. EPA's proposed actions and corrective action plan milestones were
the scope of our follow-up review. We interviewed EPA grants personnel to obtain information
about the Performance Appraisal and Recognition System (PARS) appraisal process. We
conducted internal surveys of program supervisors and staff to obtain information about whether
elements of grants accountability had been incorporated into the performance appraisal process.
We also reviewed a random sample of project officer (PO) and supervisor PARS agreements to
confirm whether grants management language had been incorporated into individual
performance agreements. We interviewed EPA staff responsible for inputting information into
the Management Audit Tracking System (MATS) and tracking that information. For further
details on the audit scope and methodology used, see Appendix A.
Results
EPA Established a Process to Measure Performance
EPA established a process to measure performance against grant management requirements
which formed the basis for performance ratings and discussions. The EPA Office of
Administration and Resources Management's Office of Grants and Debarment (OARM-OGD)
issued Guidance for Addressing Grants Management and the Management of Inter agency
Agreements under the Performance Appraisal and Recognition System on January 17, 2008.
The guidance demonstrated fulfillment of key elements of the corrective action plan and
addressed the first two recommendations from the prior audit. First, it established a process to
measure PO, supervisor, and manager responsibilities during year-end evaluations. Second, it
encouraged managers and supervisors to engage in discussions of grants management during
performance evaluations. The guidance listed five quantitative grants management performance
measures of project officer performance to evaluate POs:
• Number of awards made to non-profit organizations with open monitoring findings;
• Average number of days for project officers to transmit funding recommendation
package;
• Percentage of advanced programmatic monitoring reports completed in 60 days;
• Percentage of advanced monitoring reports closed in 120 days; and
• Number of programmatic baseline monitoring activities conducted for active awards.
-------
08-P-0276
The guidance also included A Manager's Guide: Reviewing Project Officers Grants Management
Performance under PARS as an attachment. This manager's guide was designed to assist the
manager in giving performance appraisals to POs by asking the PO questions in five key areas:
managerial support, PO basic responsibilities, compliance monitoring, environmental results, and
communication.
Project Officers and Supervisors Report Discussing Grants Management During
Performance Evaluations
A large majority of supervisors and project officers reported that they discussed grants
management during performance reviews. OGD conducted a survey of POs in 2006 to ensure
that project officers and supervisors discussed grants management during performance reviews.
In the Agency's survey, over 82 percent of affected POs said that their performance reviews
(both mid- and year-end) with their supervisors contained a discussion of their grants
management responsibilities. During this audit, we conducted a survey of project officers and
supervisors nationwide who reported that they discussed grants management during performance
reviews. Ninety-one percent of POs told us that their supervisors discussed grants accountability
with them during their last performance reviews and 100 percent of supervisors reported talking
to the POs they supervised about grants management and accountability. A majority of surveyed
POs (54 percent) and supervisors (75 percent) believed the EPA policies encouraging discussion
of grants management had improved performance or increased accountability within the EPA.
EPA Communicated Identified Weaknesses to Appropriate Project Officers and
Supervisors
EPA issued a series of survey results and guidance documents to communicate potential grants
management weaknesses to POs and supervisors. On January 20, 2006, OGD re-sent results of
grants management reviews to senior resource officials, junior resource officials, and grants
management officers. Program offices were told to remind managers that they must
communicate results of the reviews to supervisors and project officers and incorporate them into
performance reviews as appropriate. Data we collected during this audit indicated that the
outcomes of these grants management reviews were generally known among POs and
supervisors. POs and supervisors reported that they are more aware of grants accountability
issues than they were in 2005.
More recently, the Agency used comprehensive performance review survey material to highlight
project officer grants management responsibilities and challenges. Survey results were
distributed to, and discussed by, the Grants Management Council (composed of EPA senior
resource officials) at its July 11, 2007, meeting. The Council agreed, based on the results of the
survey, that greater emphasis should be placed on ensuring that managers/supervisors carry out
their grants management responsibilities. The Council suggested that OGD prepare and
distribute a performance appraisal guide for managers to use when assessing project officers'
performance. OGD added Attachment C, A Manager's Guide: Reviewing Project Officers
Grants Management Performance under PARS, to Guidance for Addressing Grants Management
and the Management of Interagency Agreements under the Performance Appraisal and
-------
08-P-0276
Recognition System. Our work showed that 72 percent of supervisors were aware of the
Manager's Guide and that 62 percent of supervisors had used it.
EPA Tracked Recommendations in MATS
The MATS system is used to track the Agency's progress in complying with milestones and
implementing recommendations in OIG reports. In the MATS report, Staff Not Held
Accountable for Grants Management., 2005-P00027-150, the Agency agreed to implement all of
the OIG's recommendations and stated that it would complete planned actions to meet those
recommendations. The MATS entry listed actual completion dates for corrective actions as
5 days after planned completion dates and showed that the action official did not issue the
certification memorandum until January 2, 2008. This was 4 months after the final action date
listed in the system as August 31, 2007. The action official said that it was an oversight that
OGD had not completed the certification memo. He emphasized that OGD would provide
needed certifications in a more timely manner in the future. In this instance, the lack of timely
input into MATS did not appear to have any significant negative impact.
Conclusion
EPA completed the corrective action plan steps addressing the recommendations in our 2005
report. By incorporating new procedural and communications elements into project officer and
grants supervisor duties, EPA has encouraged an environment where grants accountability is a
focal point in managing assistance agreements. These actions should lead to improved grants
management.
Agency Response
EPA stated that it would continue to work with the grants management community to ensure that
grants accountability is a focal point in managing assistance agreements.
-------
08-P-0276
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS BENEFITS (in $OOOs)
Planned
Rec. Page Completion Claimed Agreed To
No. No. Subject Status1 Action Official Date Amount Amount
No recommendations
0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
-------
08-P-0276
Appendix A
Details on Scope and Methodology
We performed our work in accordance with generally accepted government auditing standards,
issued by the Comptroller General of the United States. These standards require that we plan
and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on our audit
objectives.
We performed audit field work from February to June 2008. We analyzed the laws, regulations,
and guidance pertaining to grants accountability and performance. We reviewed internal
controls related to measuring, reporting, and monitoring EPA personnel performance. We
gained an understanding of internal controls through carrying out the procedures outlined below.
We interviewed OARM-OGD and OARM Office of Policy and Resources Management
managers and staff members in EPA Headquarters in Washington, DC. We assessed prior-year
survey information provided by OARM-OGD and reviewed applicable documentation in the
MATS system to gain an initial impression of EPA progress in completing the related corrective
action plan. We reviewed EPA Order 2750 to gain an understanding of the audit follow-up
process.
We conducted project officer and supervisor surveys to determine the extent to which grants
accountability had been incorporated into EPA performance appraisal discussions. Surveys were
developed from Attachments A (Grants Management Performance Measures for Project
Officers) and B (Grants Management Performance Measures for Supervisors/Managers) to the
Guidance for Addressing Grants Management and the Management of Inter agency Agreements
under the Performance Appraisal and Recognition System., issued on January 17, 2008. We
selected as survey recipients a random, stratified sample of those project officers in EPA's
Integrated Grants Management System with active grants as of March 31, 2008, and also
surveyed the supervisors of those POs. The selected sample included POs and supervisors from
Headquarters and all regions. The POs were spread across 3 strata - those managing 1 to 5
grants, those managing 6 through 20 grants, and those managing over 20 grants. We received 77
completed PO surveys out of 100 total surveys sent (77-percent response rate). We received 25
completed supervisor surveys out of 48 total surveys sent (52-percent response rate).
We reviewed PARS and Excellence in Grants Management guidance to become familiar with
current performance standards. We assessed random samples of project officer and supervisor
PARS agreements to ensure that grants management language had been incorporated. The OIG
team reviewed 22 project officer PARS agreements and 18 supervisor PARS agreements.
The OIG team examined the MATS entry for the original audit to determine whether EPA had
used the system to appropriately track and finalize proposed audit response actions.
-------
08-P-0276
Prior Audit Coverage
We issued OIG Report No. 2005-P-00027, EPA Managers Did Not Hold Supervisors and
Project Officers Accountable for Grants Management, on September 27, 2005. In that report, we
noted that, although the Agency had made some progress to establish accountability, managers
did not sufficiently hold supervisors and project officers accountable for grants management
because there was no process to measure most grants management activity. Managers and
supervisors generally did not discuss grants management responsibilities during year-end
evaluations. In the limited cases where grants management weaknesses were identified,
managers did not effectively communicate these weaknesses to staff.
We issued OIG Report No. 2003-P-00007, EPA Must Emphasize Importance ofPre-Award
Reviews for Assistance Agreements, on March 31, 2003. In that report, we noted that EPA
leadership did not always stress the importance of project officer duties and did not hold project
officers accountable for conducting complete pre-award reviews. As a result, there was
insufficient assurance that proposed costs were reasonable, that recipients were technically
capable of performing the work, and ultimately that the projects would accomplish program
objectives or achieve desired environmental results.
-------
08-P-0276
Appendix B
Agency Response
September 23, 2008
MEMORANDUM
SUBJECT: Follow-up on Grants Accountability Audit - EPA Actions Should Lead to
Improved Grants Accountability (August 29, 2008, Assignment No. 2008-0163)
FROM: Luis A. Luna
Assistant Administrator
TO: Janet Kasper
Director, Contracts and Assistance Agreement Audits
Thank you for the opportunity to provide comments on the subject draft audit report. The
draft report is a follow-up review of actions taken by the Agency in response to the September
27, 2005 Grants Accountability Audit report, EPA Managers Did Not Hold Supervisors and
Project Officers Accountable for Grants Management (Audit Report 2005-P-00027).
I am pleased to note that the draft report concludes that EPA completed the corrective
action plan steps addressing the recommendations in the 2005 report and that such actions should
lead to improved grants management. We will continue to work with the grants management
community to ensure that grants accountability is a focal point in managing assistance
agreements.
Again, thank you for the opportunity to comment on the draft report, and for helping the
Agency create a culture of accountable grants management. If you have any questions about this
response, please contact Howard Corcoran, Director, Office of Grants and Debarment, at (202)
564-1903.
cc: Melissa Heist
Susan B. Hazen
Senior Resource Officials
Randy Holthaus
Howard Corcoran
Stefan Silzer
Jeanne Conklin
John Nolan
Laurice Jones
-------
08-P-0276
Appendix C
Distribution
Office of the Administrator
Assistant Administrator for Administration and Resources Management
Director, Grants and Interagency Agreement Management Division
Agency Follow-up Official (the CFO)
Agency Follow-up Coordinator
Office of General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Audit Follow-up Coordinator, Office of Administration and Resources Management
Deputy Inspector General
------- |