1C
\
A
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Audit Report
EPA Effectively Reviewed
Recovery Act Recipient Data but
Opportunities for Improvement
Exist
Report No. 10-R-0234
September 27, 2010
-------
Report Contributors: Andres Calderon
Nancy Dao
Denise Darasaw
Doug LaTessa
Les Partridge
Abbreviations
EPA U.S. Environmental Protection Agency
GAO U.S. Government Accountability Office
OARM Office of Administration and Resources Management
OIG Office of Inspector General
OMB Office of Management and Budget
-------
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
10-R-0234
September 27, 2010
Catalyst for Improving the Environment
Why We Did This Review
We conducted this review as
part of a national review
coordinated by the U.S.
Department of Agriculture
Office of Inspector General.
The effort was designed to
respond to concerns raised by
Members of Congress and the
U.S. Government
Accountability Office
regarding reporting errors in
the October 2009 recipient
reports.
Background
The American Recovery and
Reinvestment Act of 2009
states that the use of Recovery
Act funds should be
transparent and reported
clearly, accurately, and in a
timely manner. The Office of
Management and Budget
(OMB) issued government-
wide guidance for carrying out
the reporting requirements
included in Section 1512 of
the Recovery Act.
For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202)566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2010/
20100927-10-R-0234. pdf
EPA Effectively Reviewed Recovery Act
Recipient Data but Opportunities for
Improvement Exist
What We Found
The U.S. Environmental Protection Agency's (EPA's) controls for reviewing
recipient-reported data pursuant to the Recovery Act resulted in low error rates.
For the fields that OMB identified as those for which there are major concerns for
significant reporting errors, we identified errors in 3 percent of recipient entries.
While EPA did not identify any of these errors as significant, we believe two of
the errors are significant. We also identified errors in other fields that OMB did
not identify as major concerns for significant reporting errors. Significant errors
could mislead the public about how much money the recipients are receiving.
OMB guidance requires federal agencies to provide Recovery Act recipients with
a list of key award information. EPA prepared a reference guide that instructed
recipients where to find this information. However, the reference guide was not
specific for some key data fields, leaving recipients to interpret how to report
information. As a result, some recipients reported inaccurate data.
OMB guidance identified the award amount data element as a major concern for
significant reporting errors. Instead of adopting an Agency-wide action limit for
identifying errors, EPA allowed program offices to adopt their own action limits.
As a result, there were some discrepancies between recipient-reported data and
EPA award amounts that were not corrected.
What We Recommend
We recommend that the Assistant Administrator for Administration and
Resources Management develop an Agency-wide threshold for identifying
significant errors for those fields OMB identified as major concerns, clarify the
reference guide to reduce the incidence of varying interpretations, and adopt a
policy to investigate all award amount differences. EPA agreed with the findings
and provided corrective action plans or acceptable alternatives for addressing the
recommendations in the report.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
FROM:
TO:
September 27, 2010
EPA Effectively Reviewed Recovery Act Recipient Data but
Opportunities for Improvement Exist
Report No. 10-R-0234
-
Arthur A. Elkins, Jr.
Inspector General
Craig E. Hooks, Assistant Administrator
Office of Administration and Resources Management
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
EPA managers, in accordance with established resolution procedures, will make final
determinations on matters in this report.
The estimated cost of this report - calculated by multiplying the project's staff days by the
applicable daily full cost billing rates in effect at the time - is $278,859.
Action Required
In responding to the draft report, the Agency provided corrective action plans for addressing all
of the recommendations. Therefore, a response to the final report is not required. The Agency
should track corrective actions not implemented in the Management Audit Tracking System. We
have no objections to the further release of this report to the public. The report will be available
at http://www.ega.gov/oig.
If you or your staff have any questions regarding this report, please contact Melissa Heist at
(202) 566-0899 or heist.melissa@epa.gov: or Janet Kasper, Product Line Director, at
(312) 886-3059 or kasper.janet@epa.gov.
-------
EPA Effectively Reviewed Recovery Act Recipient Data but 10-R-0234
Opportunities for Improvement Exist
Table of Contents
Chapters
1 Introduction 1
Purpose 1
Background 1
Noteworthy Achievements 2
Scope and Methodology 2
Prior Audit Coverage 3
2 EPA's Internal Controls Resulted in Low Error Rate 4
OMB Guidance Focused on Significant Reporting Errors and
Material Omissions 4
EPA's Controls for Reviewing Data Effective 4
EPA Did Not Define What Constitutes a Significant Error 6
Recipient-Reported Data Were Generally Accurate 6
Recommendation 6
Agency Comments and OIG Evaluation 6
3 EPA Guidance Requires Recipients to Interpret Reporting Instructions 8
Recipients Responsible for Finding Key Award Information 8
Improved Guidance Needed to Ensure Accuracy of Recipient Reporting... 8
Guidance Should Be More Specific 9
Recommendation 9
Agency Comments and OIG Evaluation 10
4 EPA Should Define Standard Action Limits 11
OMB Cites Amount of Award as a Data Element of Major Concern 11
EPA Program Offices Selected Varying Action Limits 11
Inaccurate Award Amounts Could Mislead Public 12
Recommendation 12
Agency Comments and OIG Evaluation 12
Status of Recommendations and Potential Monetary Benefits 14
Appendices
A Details on Scope and Methodology 15
B Agency Response to Draft Report 16
C Distribution 19
-------
10-R-0234
Chapter 1
Introduction
Purpose
Our audit was part of a multiagency effort coordinated by the U.S. Department of
Agriculture Office of Inspector General (OIG) at the request of the Recovery
Accountability and Transparency Board. The effort was designed to respond to
concerns raised by Members of Congress and the U.S. Government
Accountability Office (GAO) regarding reporting errors in the October 2009
recipient reports. Our objective was to determine whether EPA has implemented
an internal control structure that effectively ensures that recipient data are
reported completely, accurately, and in a timely manner; and that any material
omissions and/or significant errors are identified and corrected.
Background
EPA awards multimillion-dollar assistance agreements and contracts under the
American Recovery and Reinvestment Act of 2009. The Recovery Act states that
the use of Recovery Act funds should be transparent and reported clearly,
accurately, and in a timely manner. The Recovery Act requires federal agencies
to review recipient reporting. The Office of Management and Budget (OMB)
issued OMB M-09-21, Implementing Guidance for the Reports on Use of Funds
Pursuant to the American Recovery and Reinvestment Act (Recovery Act) of 2009,
on June 22, 2009. OMB M-09-21 provides government-wide guidance for
carrying out the reporting requirements included in Section 1512 of the Recovery
Act. Section 1512 requires recipients to report on the use of Recovery Act
funding by the tenth day of each calendar quarter. Federal agencies then have an
opportunity to review and comment on the recipient-reported data before they are
made public.
Recipient-reported data are made available to the public at www.Recovery.gov.
The reports are designed to provide the public with an unprecedented level of
transparency with respect to how Recovery Act money is being spent and will
help drive accountability for the timely, prudent, and effective spending of
Recovery Act dollars.
GAO issued a report titled Recovery Act: Recipient Reported Jobs Data Provide
Some Insight into Use of Recovery Act Funding, but Data Quality and Reporting
Issues Need Attention, on November 19, 2009. In this report, GAO recommended
that OMB and federal agencies reexamine review and quality assurance
processes, procedures, and requirements in light of issues identified in the first
round of recipient reporting. As a result, OMB issued OMB M-10-08, Updated
-------
10-R-0234
Guidance on the American Recovery and Reinvestment Act - Data Quality, Non-
Reporting Recipients, and Reporting of Job Estimates, on December 18, 2009.
This guidance incorporates GAO guidance and lessons learned from the first
quarter of recipient reporting, specifically those related to the quality of data
collected and the reporting of jobs created and retained.
Noteworthy Achievements
EPA was proactive about ensuring data quality. During its review of recipient-
reported data, EPA identified four recipients that did not report information
related to their Recovery Act awards. EPA considered this nonreporting as
material omissions. In three cases, technical issues prevented the recipients from
reporting in a timely manner. In the fourth case, the recipient had also not
reported in the first reporting period. EPA tried to work with that recipient, but
the recipient did not respond to EPA's efforts and did not report during the quarter
ending December 31, 2009. The regional grants office tried to work with the
recipient to resolve issues relating to the assistance agreement, but eventually
determined that it needed to initiate a process for terminating the agreement
because the recipient did not comply with the terms and conditions of the award.
Scope and Methodology
We performed our work in accordance with generally accepted government
auditing standards, issued by the Comptroller General of the United States. These
standards required that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the evidence obtained
provides a reasonable basis for our findings and conclusions based on our audit
objectives.
We reviewed recipient-reported data for the quarter ending December 31, 2009.
For our analysis, we compared two data sets. The first data set was an extract of
recipient-reported data from www.FederalReporting.gov. The second data set
was a spreadsheet of Recovery Act awards EPA submitted to OMB as a result of
a February 26, 2010, information request. We compared all of the recipient-
reported data to all of the EPA data for the data fields listed in Table 1-1.
-------
10-R-0234
Table 1-1: Contract and Assistance Agreement Data Fields
Contracts
Award Type
Award Number
Funding Agency Code
Awarding Agency Code
Amount of Award
Assistance Agreements
Award Type
Award Number
Funding Agency Code
Awarding Agency Code
Award Date
Amount of Award
Catalogue of Federal Domestic Assistance Number
Activity Code
Program Source (Treasury Account Symbol)
Code
Source: OMB guidance M-10-08.
After identifying discrepancies between EPA data and recipient-reported data, the
audit team independently verified the data. The audit team then asked EPA staff
to explain the discrepancies noted with respect to assistance agreements and
contracts. Based on the explanations received, the audit team determined whether
the discrepancy was due to an EPA error or a recipient error, or whether no error
occurred (i.e., the discrepancy was due to a timing issue).
Further details on scope and methodology are in Appendix A.
Prior Audit Coverage
We issued a related report, EPA Recovery Act Recipient Reporting and Data
Review Process, Report No. 10-R-0020, in October 2009. The purpose of that
review was to determine whether EPA established a process to perform limited
data quality reviews intended to identify material omissions and/or significant
reporting errors, and to notify the recipients of the need to make appropriate and
timely changes. No recommendations were identified in that report.
-------
10-R-0234
Chapter 2
EPA's Internal Controls Resulted in Low Error Rate
EPA developed procedures to assist Agency staff in reviewing Recovery Act
recipient quarterly reports, which resulted in an error rate of 3 percent in the fields
OMB identified as major concerns for significant reporting errors. During our
review, we identified 23 errors pertaining to award amount and award number.
We considered two of these errors to be significant, although EPA did not. OMB
guidance requires federal agencies to establish data review processes that focus on
significant reporting errors and material omissions, and report significant errors to
OMB each quarter. EPA did not regard as significant the errors we identified
because it did not define what constitutes a significant error. Significant errors
could mislead the public about how much money the recipients are receiving.
OMB Guidance Focused on Significant Reporting Errors and Material
Omissions
OMB guidance states that federal agencies should establish data review processes
that focus on significant reporting errors and material omissions, and report
significant errors to OMB. OMB identified several data elements, including
award amount and award number, as major concerns for significant reporting
errors. Significant reporting errors are instances in which inaccurate reporting of
required data could mislead or confuse the public. However, OMB did not define
for agencies what the margin of error threshold should be. After federal agencies
review Recovery Act recipient quarterly reports, agencies are to report on any
significant errors to OMB.
EPA's Controls for Reviewing Data Effective
EPA's low error rate in recipient-reported data can be attributed to the process the
Agency instituted pursuant to the Recovery Act. For the data fields OMB
identified as major concerns for significant reporting errors, we found errors in
23, or 3 percent, of the 764 recipient reports for the period ending December 31,
2009 (Table 2-1). For the other fields that we tested, we found 48 errors in the
recipient reports.
-------
10-R-0234
Table 2-1: OMB-ldentified Fields of Major Concern for Significant Reporting Errors
Data field
Number of
errors in
recipient data
Recipient errors
EPA identified
as significant
Recipient errors
OIG identified as
significant
Award Amount
Assistance Agreements
Contracts
3
9
0
0
1
1
Award Number
Assistance Agreements
Contracts
8
3
0
0
0
0
Source: OIG analysis.
EPA's process for reviewing recipient data for the reporting period ending
December 31, 2009, included two areas of emphasis. First, EPA conducted a
macro review of all recipient reports. The macro review was designed to identify
missing reports, missing data elements, clear anomalies, and reported information
that exceeded preestablished thresholds for action. Next, program offices
conducted a separate review to ensure completeness of recipient data and to
identify anomalies and obvious errors. EPA received 764 recipient reports (for
585 assistance agreements and 179 contracts) and made 178 formal comments to
recipients regarding their reports.
We considered 2 of the 12 errors we found in the award amount field for
assistance agreements and contracts to be significant. The other 10 errors were
below the OIG's judgmental threshold of 5 percent. When we called attention to
the two award amount errors that were over 5 percent, EPA told us it did not
consider these errors to be significant. In one case, the recipient underreported
the award amount by 37.9 percent, or $30,600. This ecipient did not report an
amendment, which accounted for the entire error. EPA did not identify the error
during its review process. In the other case, the contractor only reported the
amount that had been awarded on approved work plans, and not the amount that
had been obligated. Thus, the contract was underreported by 7.9 percent, or
$171,787.
We found 11 award number errors for assistance agreements and contracts. EPA
told us that it did not consider any of them to be significant because the Agency
could still track those awards. Recipient errors included an extra hyphen or a one-
character error in the award number. In two cases, the recipient did not report the
correct award number because EPA had not updated the contract number
correctly.
We identified 48 errors in other fields that we reviewed. However, none of these
fields were considered of major concern for significant reporting errors by the
terms of the OMB guidance (Table 2-2), and we did not find the differences
sufficiently significant to mislead the public.
-------
10-R-0234
Table 2-2: Fields Not Identified by OMB as of Major Concern for Significant
Reporting Errors
Data field
Number of
errors in
recipient data
Assistance Agreements
Award Type
Award Date
Funding Agency Code
Awarding Agency Code
Catalogue of Federal Domestic Assistance
Program Source (Treasury Account Symbol Code)
1
32
0
0
0
13
Contracts
Award Type
Funding Agency Code
Awarding Agency Code
0
1
1
Source: OIG analysis.
EPA Did Not Define What Constitutes a Significant Error
EPA did not consider significant the two errors for the award amount field that we
considered to be significant. OMB provided a broad definition of significant
errors. Based on that broad definition, EPA developed various reports and
procedures to assist Agency staff in reviewing recipient-reported data. Although
the procedures were generally effective in identifying recipient errors, EPA did
not provide its staff with specifics for what should be considered a significant
error.
Recipient-Reported Data Were Generally Accurate
Recipient-reported data were generally accurate, ensured transparency, and
provided the public reliable information about Recovery Act projects. However,
errors in the award amount field may mislead the public about how much money
recipients are receiving.
Recommendation
We recommend that the Assistant Administrator for Administration and
Resources Management:
2-1 Develop an Agency-wide threshold to identify significant errors for those
fields OMB identified as major concerns for significant reporting errors.
Agency Comments and OIG Evaluation
In responding to the draft report, the Office of Administration and Resources
Management (OARM) agreed to tighten the threshold for acceptable variances for
key data elements identified in OMB M-09-21. EPA also noted that the
-------
10-R-0234
multiagency audit of data quality review processes conducted by the U.S.
Department of Agriculture OIG, dated June 25, 2010, recommended that the
Recovery Accountability and Transparency Board provide additional guidance to
agencies to ensure consistency and uniformity in addressing significant errors.
EPA stated that it supports that recommendation and is prepared to apply the
board's guidance when issued to its future Section 1512 reviews.
We followed up with EPA to clarify the response and obtain a milestone date for
implementation of the corrective action. The Director, Office of Grants and
Debarment, stated that EPA will tighten the threshold for the following data
elements by September 15, 2010: award date, award amount, jobs created and
retained, and recipient name. The planned corrective action meets the intent of
the report recommendations.
The Agency's full response is in Appendix B.
-------
10-R-0234
Chapter 3
EPA Guidance Requires Recipients to
Interpret Reporting Instructions
EPA's guidance to recipients on what to report in certain fields in the Recovery
Act reporting system was not always sufficient to ensure that recipients reported
the correct information. OMB guidance requires federal agencies to provide
Recovery Act recipients with a list of key award information, such as award date,
activity code, and order number. EPA prepared a reference guide for recipients to
instruct them where to find this key award information. However, certain fields
of the reference guide were not specific and required recipients to interpret how to
report information. Improved guidance will reduce recipient-reported errors and
improve EPA's ability to verify the accuracy of recipient-reported data.
Recipients Responsible for Finding Key Award Information
OMB guidance requires federal agencies to provide Recovery Act recipients with
a list of key award information. EPA prepared a reference guide for recipients
that instructed recipients where to find key award information, such as award
date, activity code, and order number.
Improved Guidance Needed to Ensure Accuracy of Recipient
Reporting
EPA guidance to recipients for Recovery Act reporting should be more specific so
as to improve reporting. Below are examples of key data fields for which we
found that EPA can improve guidance to ensure that accurate and consistent
information is reported by recipients.
• Award Date: For the award date data field, EPA guidance for assistance
agreements instructs recipients to enter the date that EPA signed the
award. However, we noted several agreements in which EPA awarded an
amendment after the original award. EPA's guidance does not address
whether recipients should enter the award date for an amendment or the
original award. Because EPA's guidance did not specify which date to
use, we were not able to determine whether the recipient-reported
information was correct.
• Activity Code: For the activity code data field, EPA's reference guide for
assistance agreements identifies the activity codes that the recipient should
use for three programs. For the other eight programs, the reference guide
requires the recipients to navigate to the Office of Grants and Debarment
-------
10-R-0234
Website. From there, recipients open another link that contains a table of
14 programs. The recipients then must identify their type of assistance
agreement and select from the codes provided. We were unable to
electronically verify the activity codes because EPA does not track the
information in its databases, and the electronic file the Office of Grants and
Debarment provided to the OIG was not accurate. We did, however,
manually compare some recipient reports to the guidance information EPA
provided recipients and found that in some cases, the activity code that was
used was not one of the options EPA identified in its reference guide. The
OMB-provided template that recipients use to file their reports includes a
drop-down menu for the activity code field. However, the drop-down menu
identifies over 2,000 activity codes, complicating the activity code
determination process.
Guidance Should Be More Specific
The award date and activity code fields found in the reference guide were not
specific and required recipients to interpret how to report information. In
explaining how to fill out some of the fields, the guide directed recipients to
Websites, phone numbers, or other documents for more information. But that
information in turn was not specific and was open to interpretation. For the
instances in which the guide told recipients exactly how to fill out fields, few
errors were noted.
EPA does not routinely track activity codes, and these codes are not identified
within EPA's data systems. Therefore, staff members were not familiar with
them and thus may not identify errors when reviewing recipient reports.
Increased specificity in EPA guidance will reduce recipient-reported errors and
increase EPA's ability to verify the accuracy of recipient-reported data. Recipient
reports are designed to provide the public with transparency as to how Recovery
Act dollars are being spent in their communities. Incorrect data may lead to
mistaken conclusions about the funding.
Recommendation
We recommend that the Assistant Administrator for Administration and
Resources Management:
3-1 Clarify the reference guide to reduce the incidence of varying recipient
interpretations. The clarifications should:
a. Address whether to use the original date of the award or the date
of the amendment for assistance agreements.
b. Specify which activity codes to use for assistance agreements.
-------
10-R-0234
Agency Comments and OIG Evaluation
In responding to the draft report, EPA agreed to take actions to address the
recommendation and provided milestone dates. OARM stated that the Office of
Grants and Debarment would work with the EPA Stimulus Tracking and
Reporting Subcommittee to make appropriate revisions to the Agency's reference
guide and the EPA Standard Review Procedure by October 1, 2010. The planned
corrective action meets the intent of the report recommendations. The Agency's
full response is in Appendix B.
10
-------
10-R-0234
Chapter 4
EPA Should Define Standard Action Limits
EPA program offices selected varying action limits for the award amount data
element reported by Recovery Act recipients. The action limits identify the error
rate in the award amount data element that EPA program offices will accept as
accurate. OMB identified the award amount data element as a major concern for
significant reporting errors. Instead of adopting an Agency-wide action limit,
EPA allowed program offices to adopt their own action limits. The lack of an
Agency-wide action limit could affect the error rate in the award amount data
elements. Recipients that underreport or overreport Recovery Act awards could
mislead the public because of the significant dollar amounts they receive.
OMB Cites Amount of Award as a Data Element of Major Concern
OMB identified the award amount data element as a major concern for significant
reporting errors. OMB defined significant reporting errors as those instances in
which required data are not reported accurately and such errors result in
significant risk that the public will be misled or confused. OMB guidance
requires federal agencies to establish data review processes that focus on
significant reporting errors.
EPA Program Offices Selected Varying Action Limits
EPA program offices selected varying action limits for the award amount data
element reported by its Recovery Act funding recipients. These action limits are
identified in EPA Procedure for Review ofARRA Section 1512 Recipient
Reported Information, issued in October 2009. The action limits identify the error
rate in the award amount data field that EPA program offices will accept as
accurate.
Instead of adopting an Agency-wide action limit, EPA allowed program offices to
adopt their own action limits (Table 4-1). The Clean Water State Revolving
Fund/Drinking Water State Revolving Fund programs set the action limit at
2 percent, because OMB had not outlined a standard procedure and because the
programs had no experience reviewing large amounts of data under extremely
tight deadlines. The Leaking Underground Storage Tank program staff decided
an action limit of+/-10 percent was reasonable. Representatives from both
program offices told us that despite these action limits, they investigated any
difference between the recipient-reported award amount and EPA award amount.
11
-------
10-R-0234
Table 4-1: Program-Identified Action Limits
Program Offices
Clean Water State Revolving Fund
Drinking Water State Revolving Fund
Water Quality Management Planning
Leaking Underground Storage Tank
Superfund
Brownfields
National Clean Diesel
Action Limit
Greater than 2%
Greater than +/-10%
Any discrepancy
Source: EPA Procedure for Review ofARRA Section 1512 Recipient Reported Information,
October 2009.
Inaccurate Award Amounts Could Mislead Public
Recipients that underreport or overreport Recovery Act awards could mislead the
public. By investigating all differences between what the recipient reported and
what the EPA data showed for the award amount, EPA may identify systemic
issues. For example, for the reporting period ending December 31, 2009, we
identified some contract modifications that were either issued near the end of the
reporting period or made in the next reporting cycle. Some contract modifications
issued near the end of the reporting period that we identified were not included in
the recipient-reported totals, while some contract modifications made in the next
reporting cycle that we identified were prematurely reported by the recipients in
the quarter ending December 31, 2009. By investigating all differences, similar
patterns can be identified and then specifically looked for during future reviews.
Public confidence increases when recipient-reported data are accurate.
Recommendation
We recommend that the Assistant Administrator for Administration and
Resources Management:
4-1 Adopt a policy that requires program offices to investigate all differences
- regardless of dollar amount - between what recipients report for their
award amount and what EPA has recorded in its information systems.
Agency Comments and OIG Evaluation
In response to the draft report, OARM stated that the EPA standard review
procedure document will be modified to eliminate variances when examining
award amounts. The written response did not provide a milestone date, so we
followed up with the Director, Office of Grants and Debarment. EPA indicated it
will modify the standard review procedure document by September 15, 2010.
12
-------
10-R-0234
The planned corrective action meets the intent of the report recommendations.
The Agency's full response is in Appendix B.
13
-------
10-R-0234
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS BENEFITS (in SOOOs)
Planned
Rec. Page Completion Claimed Agreed To
No. No. Subject Status1 Action Official Date Amount Amount
2-1 6 Develop an Agency-wide threshold to identify 0 Assistant Administrator for 9/15/2010*
significant errors for those fields OMB identified as Administration and
major concerns for significant reporting errors. Resources Management
3-1 9 Clarify the reference guide to reduce the incidence 0 Assistant Administrator for 10/01/2010
of varying recipient interpretations. The Administration and
clarifications should: Resources Management
a. Address whether to use the original date
of the award or the date of the
amendment for assistance agreements.
b. Specify which activity codes to use for
assistance agreements.
4-1 12 Adopt a policy that requires program offices to 0 Assistant Administrator for 9/15/2010*
investigate all differences - regardless of dollar Administration and
amount - between what recipients report for their Resources Management
award amount and what EPA has recorded in its
information systems.
1 0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
* Although the milestone for Recommendations 2-1 and 4-1 have passed, the Agency has not informed the DIG that the corrective actions for these
recommendations are completed or provided updated milestones.
14
-------
10-R-0234
Appendix A
Details on Scope and Methodology
We performed audit field work from March to July 2010. We analyzed the Recovery Act and
guidance pertaining to EPA's Recovery Act assistance agreements and contracts. We reviewed
internal controls related to analyzing and reporting on recipient-provided data. We gained an
understanding of internal controls through performance of the procedures outlined below:
• We gathered recipient data from www.FederalReporting.gov and assistance agreements
and contracts data from EPA's internal data systems. We then compared the two sets of
data using data mining software. After organizing the data discrepancies we found by
EPA program, we asked the relevant programs about the discrepancies we found for the
key data fields and documented the results. The information provided from
Federalreporting.gov, and EPA's internal data systems were sufficient for the purposes of
our review.
• We interviewed a senior advisor on the Tracking and Reporting Subcommittee of the
EPA Stimulus Steering Committee and a senior analyst in the Office of Environmental
Information. We interviewed points of contact in the Clean Water and Drinking Water
State Revolving Funds as well as in the Offices of Acquisition Management, Grants and
Debarment, the Chief Financial Officer, Air and Radiation, Solid Waste and Emergency
Response, and Administration and Resources Management. We also interviewed
personnel in the Water Quality Management Planning and Construction Grants and
Territories programs.
We reviewed relevant criteria documents, such as OMB's M-10-08 guidance, issued
December 18, 2009, and the EPA Procedure for Review ofARRA Section 1512 Recipient
Reported Information., issued in October 2009.
15
-------
10-R-0234
Appendix B
Agency Response to Draft Report
August 26, 2010
MEMORANDUM
SUBJECT: Draft Audit Report:
EPA Effectively Reviewed Recovery Act Recipient Data but
Opportunities for Improvement Exist
(Project No. 2010-1222)
FROM: Craig E. Hooks
Assistant Administrator
TO: Janet Kasper
Director, Contracts and Assistance Agreement Audits
Thank you for the opportunity to comment on the draft audit report, "EPA Effectively
Reviewed Recovery Act Recipient Data but Opportunities for Improvement Exist (Project No.
2010-1222)."
I am pleased that the report recognizes the steps EPA has taken to carry out our
responsibilities to review recipient-reported information in accordance with Section 1512 of the
American Recovery and Reinvestment Act (ARRA). The report also recognizes the Agency's
proactive approach to reviewing recipient information and concludes that EPA's system of
controls resulted in low error rates. Additionally, I want to commend the OIG for the analytical
method used in the report to compare authoritative Agency master list information to recipient-
reported extracted data. This method provides an extra macro-level check of Section 1512 data
to identify potential deficiencies, and has been adopted by our Stimulus Tracking and Reporting
Subcommittee.
The report contains three recommendations designed to improve the process for
identifying significant reporting errors, clarify guidance to recipients and establish a more
stringent threshold for identification of errors. OARM's response to these recommendations is
presented below.
Recommendation #1: Develop an Agency-wide threshold for identifying significant errors for
those fields OMB identified as significant.
16
-------
10-R-0234
OARM Response: EPA is required to follow the guidance issued to all agencies and
departments contained in OMB M-10-08 as well as supplemental instructions contained in OMB
M-09-21 to identify significant errors. The threshold established by OMB in this guidance
suggests significant errors are instances where incorrect data results in "significant risk the
public will be misled." As agencies implement this threshold, OMB has indicated they must
make a judgment regarding the materiality of any data discrepancy and its potential to present a
significant risk of misleading the public. Rather than attempt to redefine OMB's definition of a
significant error as suggested by the OIG recommendation, EPA will tighten the threshold for
acceptable variances for key data elements identified in OMB M-09-21. The tightened action
limits are discussed further in our response to Recommendation #3.
OARM also notes a relevant recommendation of the United States Department of Agriculture's
(USDA) OIG multi-agency audit of data quality review processes dated June 25, 2010. In that
audit, which also addressed EPA's processes, the USDA OIG recommended that the Recovery,
Accountability and Transparency Board provide additional guidance to agencies to ensure
consistency and uniformity in addressing significant errors. We support this recommendation
and are prepared to apply the Board's guidance when issued to our future Section 1512 reviews.
Recommendation #2: Clarify guidance to reduce the need for recipients to interpret the
reference guide for their specific award. The clarifications should (1) address whether to use the
original date of the award or the date of the amendment for assistance agreements, and (2)
specify exactly which activity codes to use for assistance agreements.
OARM Response: OARM agrees with this recommendation. EPA's Office of Grants and
Debarment (OGD), working with the EPA Stimulus Tracking and Reporting Subcommittee, will
make appropriate revisions to the Agency's reference guide and the EPA Standard Review
Procedure document prior to the next Section 1512 review cycle beginning October 1, 2010.
Recommendation #3: Adopt a policy that requires program offices to investigate all differences
- regardless of dollar amount - between what recipients report for their award amount and what
EPA has recorded in its information systems.
OARM Response: OARM agrees with this recommendation. The EPA Standard Review
Procedure document will be modified to eliminate variances when examining award amounts.
Thank you for the opportunity to comment on the report. If you have any questions about
these comments, please contact Howard Corcoran, Director, OGD, at (202) 564-1903 or Don
Flattery, Chair, Stimulus Tracking and Reporting Subcommittee, at (202) 564-4677.
cc: Nanci Gelb
Renee Wynn
Sheila Frace
Jerry Kurtzweg
Don Flattery
17
-------
1O-R-0234
John Showman
Marian Cooper
Howard Corcoran
Denise Benjamin-Sirmons
Catherine Vass
John Bashista
John Oliver
18
-------
10-R-0234
Appendix C
Distribution
Office of the Administrator
Assistant Administrator for Administration and Resources Management
Assistant Administrator, Office of Environmental Information and Chief Information Officer
Director, Office of Acquisition Management, Office of Administration and
Resources Management
Director, Office of Grants and Debarment, Office of Administration and Resources
Management
Agency Follow-up Official (the CFO)
Agency Follow-up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Audit Follow-up Coordinator, Office of Administration and Resources Management
Audit Follow-up Coordinator, Office of Grants and Debarment,
Office of Administration and Resources Management
Audit Follow-up Coordinator, Office of Acquisition Management,
Office of Administration and Resources Management
Inspector General
19
------- |