Q
w,
o	I U.S. ENVIRONMENTAL PROTECTION AGENCY
%%pro^° OFFICE OF INSPECTOR GENERAL
Improving OIG processes
Quality Assurance Review
of EPA OIG Audit
Assignments Completed in
Fiscal Year 2018
Project No. 19-N-0167
June 4, 2019

-------
Report Contributors:	Jerri Dorsey-Hall
Janet Kasper
Abbreviations
CMR
Compliance Monitoring Review
CPE
Continuing Professional Education
EPA
U.S. Environmental Protection Agency
FY
Fiscal Year
GAGAS
Generally Accepted Government Auditing Standards
OA&E
Office of Audit and Evaluation
OIG
Office of Inspector General
PMH
Project Management Handbook
Are you aware of fraud, waste or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW (2431T)
Washington, DC 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotline@epa.gov
Learn more about our OIG Hotline.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (2410T)
Washington, DC 20460
(202) 566-2391
www.epa.gov/oiq
Subscribe to our Email Updates
Follow us on Twitter @EPAoig
Send us your Project Suggestions

-------
x-^to sr4j.
*. U.S. Environmental Protection Agency	19~n~0167
^	~*0	^ffiAO /\f In^nAA^Ai* r\v**r\ r-% I	JUH6 4, 2019
•	u.o. LiiviiuMMiciiidi nuietu
^ ^ \ Office of Inspector General
At a Glance
Why We Did This Project
The U.S. Environmental
Protection Agency's (EPA's)
Office of Inspector General
(OIG) operates and maintains a
system of quality controls
designed to provide reasonable
assurance that personnel
performing audit or evaluation
functions comply with all
generally accepted government
auditing standards (GAGAS)
and established OIG policies
and procedures.
Quality assurance staff from
the OIG's Office of Audit and
Evaluation report annually on
systemic issues identified
during referencing and
compliance monitoring reviews.
They also make observations
on compliance with GAGAS
and OIG policy.
In addition, GAGAS requires
that each organization
performing audits in
accordance with these
standards have an external
peer review. Peer reviews of
the OIGs must be performed at
least once every 3 years.
This report addresses the
following EPA OIG goal:
• Improve OIG processes,
resource allocation and
accountability to meet
stakeholder needs.
Address inquiries to our public
affairs office at (202) 566-2391 or
OIG WEBCOMMENTS@epa.oiq.
Quality Assurance Review of EPA OIG Audit
Assignments Completed in Fiscal Year 2018
What We Found
Internal reviews of EPA OIG audit
assignments completed in fiscal year
(FY) 2018 found that the OIG complied with
GAGAS and substantially complied with OIG
policies and procedures. An external peer
review conducted in FY 2018 also found that
the EPA OIG's system of quality control was suitably designed and complied
with, providing the OIG with reasonable assurance that audits are performed and
reported in conformity with GAGAS.
OIG audit reports issued in
FY 2018 demonstrated high
levels of compliance with OIG
quality assurance procedures,
receiving average compliance
scores of 94 percent.
As part of our internal quality review, we evaluated the OIG's audit activities, from
the preliminary research to the final report. Projects with reports issued in
FY 2018 complied with GAGAS and generally complied with OIG policies. The
average compliance monitoring score was 94.31 out of 100. Similar to FY 2017,
the only systemic issue identified was the estimation and approval of project time
frames and cost estimates. The issue continued to occur because the
recommendation from the FY 2017 quality control review was not implemented in
FY 2018. When reports are not timely and current, their relevance and
usefulness can be diminished.
In addition, our internal quality review assessed the OIG's compliance with
GAGAS independence and competence standards, and we did not identify any
significant issues. However, the office should ensure that it is maintaining all
personal impairment forms for record-keeping purposes.
Recommendations
We recommend that the Assistant Inspector General for Audit and Evaluation
finalize the planned corrective actions to address the FY 2017 quality control
review recommendation. We also recommend that the Assistant Inspector
General for Audit and Evaluation establish internal controls to verify that personal
impairment forms are timely completed and readily accessible for all staff.
Assistant Inspector General Response
The Assistant Inspector General for Audit and Evaluation agreed with the findings
and recommendations and provided milestone dates for implementing the
recommendations.
List of OIG reports.

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
June 4, 2019
MEMORANDUM
SUBJECT: Quality Assurance Review of EPA OIG Audit Assignments
Completed in Fiscal Year 2018
Report No. 19-N-0167
FROM: Richard Eyermann, Deputy Assistant Inspector General
Office of Audit and Evaluation
TO:
Kevin Christensen, Assistant Inspector General
Office of Audit and Evaluation
This is our report on the U.S. Environmental Protection Agency Office of Inspector General's (OIG's)
adherence to quality control elements and compliance with generally accepted government auditing
standards in OIG audit assignments completed in fiscal year 2018.
This report offers observations and recommendations to enhance and strengthen the OIG's project
execution process. It also provides opportunities for improving adherence to internal controls within
the OIG.
cc: Edward Shields, Associate Deputy Inspector General

-------
Quality Assurance Review of	19-N-0167
EPA OIG Audit Assignments
Completed in Fiscal Year 2018
	 Table of 	
Purpose		1
Background		1
Prior Reports		2
Scope and Methodology		3
Results		3
Estimation and Approval of Project Time Frames and Cost Estimates
Need Improvement		4
OIG Complied with Education Requirements		5
Annual Personal Impairment Forms Were Missing		5
Recommendations		5
Assistant Inspector General Response and OIG Evaluation		5
Appendices
A OIG CMR Results for FY 2018		6
B Reports Without CMRs		8
C Implementation Plan		9

-------
Purpose
The U.S. Environmental Protection Agency (EPA) Office of Inspector General
(OIG) operates and maintains a system of quality control designed to provide
reasonable assurance that all personnel performing audit or evaluation functions
comply with generally accepted government auditing standards (GAGAS) and
established OIG policies and procedures. Quality assurance staff from the OIG's
Office of Audit and Evaluation (OA&E) analyze and summarize the results of
their monitoring activities at least annually to identify any systemic or repetitive
issues needing improvement, along with recommendations for corrective action.
This quality assurance report summarizes the results of our compliance
monitoring reviews (CMRs) of 43 audit projects for which reports were issued in
fiscal year (FY) 2018. Appendix A lists the projects we reviewed.1 This report
also includes an update on the implementation of recommendations from our
FY 2017 quality assurance report.
Background
The Inspector General Act of 1978, as amended, requires that federal Inspectors
General comply with standards established by the Comptroller General of the
United States for audits. The EPA OIG conducts its audits in accordance with these
standards, known as GAGAS. The OIG also maintains an internal system of quality
controls to provide the organization with reasonable assurance that its products,
services and personnel comply with professional standards and applicable legal and
regulatory requirements.
GAGAS Section 3.95 states that an audit organization:
should analyze and summarize the results of its monitoring
process at least annually, with identification of any systemic or
repetitive issues needing improvement, along with
recommendations for corrective action. The audit organization
should communicate to appropriate personnel any deficiencies
noted during the monitoring process and make recommendations
for appropriate remedial action.
A measuring process should provide a mechanism to evaluate individual products
against specific quality criteria. The process should also present the information in
a manner that, over time, will allow the OIG to assess adherence to quality control
elements so that necessary adjustments can be made to policies, procedures and
activities.
1 A CMR is prepared for each assignment. Some assignments may have multiple reports, but only one CMR is
completed for each assignment. Appendix B lists the reports issued without CMRs and the reason CMRs were not
performed.
19-N-0167
1

-------
In July 2014, the EPA Inspector General signed OIG Policy and Procedure 006,
OIG Quality Control and Assurance Program, which identifies the OIG's quality
control and assurance process, including internal and external components such as
the CMR. Our system of quality control includes the use of CMRs to assess
completed work and determine whether the OIG followed our professional
standards and operated according to OIG Policy 101, Project Management
Handbook (PMH).
Each CMR encompasses an evaluation of the audit activities that occurred, from
when the preliminary research started to when the team submitted the final report
and closed the working papers. The CMR results, trends and subsequent
recommendations are summarized in our annual quality control report. The
categories evaluated in the CMR and associated point values are presented in
Table 1.
Table 1: CMR categories
Category
Point value
Planning and Execution
12
Communication
13
Supervision
30
Report Quality
20
Timeliness
15
Post Reporting/Data Quality
10
Total
100
Source: OIG.
Our review of the EPA OIG's compliance with general auditing standards—such
as independence, professional judgment, competence and adherence to continuing
professional education requirements—is not part of the CMR and is conducted
independently of the CMR.
Prior Reports
On June 18, 2018, the Department of Defense OIG issued a report on its external
peer review of the EPA OIG's system of quality assurance in effect as of
September 30, 2017. The review included a sample of reports issued in FY 2017.
The Department of Defense OIG's overall conclusion was that the EPA OIG's
system of quality control was suitably designed and complied with, providing the
EPA OIG with reasonable assurance that audits are performed and reported in
conformity with applicable professional standards. In its management letter, the
Department of Defense OIG identified some issues that were not of sufficient
significance to affect its overall conclusion. In response, the EPA OIG proposed
corrective actions to address the issues.
On July 9, 2018, the EPA OIG issued Report No. 18-N-0219. Quality Assurance
Review of EPA OIG Reports Issued in Fiscal Year 2017. Based on the results of
19-N-0167
2

-------
the CMRs conducted that year, we recommended that the Assistant Inspector
General for Audit and Evaluation update the OIG's PMH. The Assistant Inspector
General for Audit and Evaluation agreed with the recommendation and estimated
that corrective actions would be completed by the end of FY 2018.
Scope and Methodology
We performed this review on projects with final reports issued from October 1,
2017, to September 30, 2018. This review covered GAGAS-compliant reports by
the OIG's OA&E, which were reviewed and scored by the OIG's quality
assurance staff. We did not include any reports with work performed by external
auditors. The work performed in this review does not constitute an audit
conducted in accordance with GAGAS.
The scores for each report were reviewed, and any project that received less than
80 percent of the possible points in any of the six CMR categories (outlined
previously in Table 1) was reviewed to assess the issues identified. Identified
issues were then analyzed to determine whether any were systemic. An issue was
considered systemic if it occurred in more than 15 percent of the reports.
As part of the quality control review, we also assessed compliance with two
GAGAS general standards: independence and competence. Records documenting
compliance with these standards are maintained on an organizational basis, rather
than by assignment.
Results
OIG audit assignments completed in FY 2018 complied with GAGAS and
generally complied with OIG policies. The average compliance monitoring score
was 94.31 out of 100. Similar to FY 2017, the only systemic issue identified was
the estimation and approval of project time frames and cost estimates. The issue
continued to occur because, as of March 2019, the recommendation from the EPA
OIG's FY 2017 quality control review had not been implemented. When reports
are not timely and current, their relevance and usefulness can be diminished.
In addition, our review of the OIG's compliance with GAGAS independence and
competence standards did not identify any significant issues. However, the office
does need to ensure that personal impairment forms are maintained for record-
keeping purposes.
19-N-0167
3

-------
Estimation and Approval of Project Time Frames and Cost Estimates
Need Improvement
The FY 2018 CMRs continued to show a systemic issue related to the timeliness
and cost of projects. The CMRs found that some projects exceeded estimated staff
days and calendar days and that revisions to milestone dates were not always
approved. This issue continued to occur in FY 2018 because the recommendation
from the EPA OIG's FY 2017 quality assurance report was not implemented in
FY 2018. As of March 2019, OA&E management estimated that the PMH will
not be updated until the fourth quarter of FY 2019.
GAGAS identifies timeliness as a report quality element, noting in
paragraph A.7.02 that "timely issuance of the report is an important reporting goal
for auditors." OIG procedures describe requirements relating to the estimated
project time frames, approval of revisions to time frames and documentation of
approval of time frames. When reports are not timely and current, their relevance
and usefulness can be diminished.
Eight of the 43 reports we reviewed scored less than 80 percent in the category of
timeliness. The timeliness score is based on a comparison of the approved
milestone date and staff day estimate with the actual milestone date and staff day
estimate.
In response to EPA OIG Report No. 18-N-0219. the Assistant Inspector General
for Audit and Evaluation agreed to update the OIG's PMH by September 30,
2018, to clarify the following issues:
(a)	What key information regarding assignment calendar days and staff
days must be approved.
(b)	When revisions are needed to the key information.
(c)	Who can approve revisions to key information.
(d)	How key information and revisions are to be documented.
Although OA&E quality assurance staff drafted an update for the PMH before
September 30, 2018, final issuance of the PMH amendment was delayed for two
reasons. During FY 2018, the OA&E implemented a new organizational structure.
Also in FY 2018, the acting Inspector General tasked the acting Deputy Inspector
General to improve OA&E processes. As a result, OA&E management decided to
delay issuing a PMH amendment until these changes were designed and
implemented. As of March 2019, management estimates that the PMH revisions
will be issued during the fourth quarter of FY 2019. In addition to addressing the
finalized process changes, the PMH revisions will also incorporate changes
required by the 2018 update to GAGAS.2
2 U.S. Government Accountability Office, Government Auditing Standards: 2018 Revision, GAO-18-568G. July 17,
2018.
19-N-0167
4

-------
OIG Complied with Education Requirements
Auditors performing work in accordance with GAGAS should maintain their
professional competence through continuing professional education (CPE).
FY 2018 was the first year of a 2-year CPE period. Auditors are required to
complete at least 20 hours of CPE each year. All staff met the requirement in
FY 2018.
Annual Personal Impairment Forms Were Missing
GAGAS states that independence documentation must provide evidence of the
auditor's judgments in forming conclusions regarding compliance with
independence requirements (GAGAS 3.59). OIG Policy and Procedure 102, OIG
Independence, states that staff and contractors must be independent, in fact and
appearance, and sign a personal impairment form upon joining the OIG and
annually as performance agreements are established. Directors were asked to save
copies of completed impairment forms in a specific directory on the OIG share
drive. However, a review of the share drive in February 2019 found that
23 percent of the forms were missing. The Deputy Assistant Inspector General for
Audit and Evaluation requested that staff submit the missing impairment forms. A
final review of the impairment forms in March 2019 identified seven forms as still
missing and four forms as out-of-date or lacking supervisory review.
Recommendations
We recommend that the Assistant Inspector General for Audit and Evaluation:
1.	Finalize the EPA OIG's implementation of the recommendation from EPA
OIG Report No. 18-N-0219. Quality Assurance Review of EPA OIG
Reports Issued in Fiscal Year 2017, issued July 9, 2018.
2.	Establish internal controls to verify that personal impairment forms are
timely completed and readily accessible for all staff.
Assistant Inspector General Response and OIG Evaluation
The Assistant Inspector General for Audit and Evaluation agreed to implement
the prior report recommendation by completing the updates to the PMH by
September 30, 2019. In response to the second finding and recommendation,
action was taken to complete the outstanding FY 2019 personal impairment
forms. For FY 2020, the Assistant Inspector General for Audit and Evaluation
agreed to establish an internal control to require Directors to attest that all the
personal impairment forms are completed and properly stored prior to the
Directors' end-of-year performance reviews. These corrective actions are
acceptable and, when implemented, will address the recommendations.
19-N-0167
5

-------
Appendix A
OIG CMR Results for FY 2018
Report no.
Planning
Communication
Supervision
Report
quality
Timeliness
Data
accuracy
Compliance
review score
18-P-0001
11.8
11.5
28.8
19
9
8
88.1
18-P-0024
12
12
28.8
14
9
10
85.8
18-P-0030
12
13
30
20
15
10
100
18-P-0031
12
13
30
18
15
10
98
18-P-0036
10
13
30
16
15
10
94
18-P-0037
12
10
29
20
15
8
94
18-P-0038
12
13
27.25
19
13
10
94.25
18-F-0039
12
13
28.5
20
15
10
98.5
18-P-0059
12
13
28
20
15
10
98
18-P-0068
12
13
30
20
15
10
100
18-P-0071
12
13
28.8
20
14
10
97.8
18-P-0074
12
13
23.8
18
15
9.8
91.6
18-P-0079
10.5
10
27
20
11
10
88.5
18-P-0080
9
13
27.2
20
11
10
90.2
18-P-0089
12
13
29.25
20
9
10
93.25
18-P-0105
12
13
30
20
12
10
97
18-P-0129
12
13
25
20
15
10
95
18-P-0151
10.5
13
30
20
15
10
98.5
18-P-0153
12
13
24
20
15
10
94
18-P-0175
10
13
30
20
15
8
96
18-P-0176
12
11
30
14
10
10
87
18-P-0206
12
12
29.4
20
12
10
95.4
18-P-0207
12
13
29
17
15
10
96
18-P-0217
12
13
25
20
15
10
95
18-P-0218
10
13
30
20
15
10
98
18-P-0221
12
12.5
28.8
20
7.5
10
90.8
18-P-0222
12
11
30
20
15
9.5
97.5
18-P-0226
12
12.5
30
20
15
10
99.5
18-P-0227
10
12
28.2
20
12
6
88.2
18-P-0231
12
13
29.25
19
15
10
98.25
19-N-0167
6

-------
Report no.
Planning
Communication
Supervision
Report
quality
Timeliness
Data
accuracy
Compliance
review score
18-P-0232
12
13
30
20
15
10
100
18-P-0233
12
12.5
21.4
20
13
8
86.9
18-P-0234
12
12
30
19
13
8
94
18-P-0238
8
11
30
20
15
8
92
18-P-0239
11.5
13
30
20
15
10
99.5
18-P-0240
12
13
30
20
15
10
100
18-P-0241
12
13
30
20
14
10
99
18-P-0270
12
11.5
25
20
12
10
90.5
18-P-0271
12
12
30
10
3
8
75
18-P-0281
11.8
12
24.6
20
13
8
89.4
18-P-0283
12
13
27
20
14
10
96
18-P-0288
12
13
27
20
14
10
96
18-P-0298
12
12
30
20
15
10
99
Average
11.56
12.45
28.37
19.14
13.27
9.52
94.31
Source: OIG.
19-N-0167
7

-------
Appendix B
Reports Without CMRs
Report no.
Report title
Reason for No CMR
18-P-0181
EPA Did Not Identify Volkswagen Emissions Cheating; Enhanced
Controls Now Provide Reasonable Assurance of Fraud Detection
First report of two expected
from assignment.
18-N-0154
Management Alert: Salary Increases for Certain Administrative
Determined Positions
Interim report and non-
GAGAS.
18-F-0040
Audit of U.S. Chemical Safety and Hazard Investigation Board's
Fiscal Years 2017 and 2016 Financial Statements
Contracted audit
18-B-0228
OIG Office of Investigations' Records Demonstrate Compliance
with Law Enforcement Pay Requirements.
Non-GAGAS report
18-P-0250
Management Alert: EPA Oversight of Employee Debt Waiver
Process Needs Immediate Attention
Interim report
18-N-0174
EPA's Fiscal Year 2018 Management Challenges
Non-GAGAS report
18-P-0220
EPA Complied with the Statement of Financial Accounting
Standards No. 47 and Treasury Financial Manual, Part 2,
Chapter 4700
Limited scope report
Source: OIG.
19-N-0167
8

-------
Appendix C
Implementation Plan
No.
Recommendation
Action official
Status
Planned
implementation
date
1
Finalize the EPA OIG's implementation of the
recommendation from EPA OIG Report
No. 18-N-0219, Quality Assurance Review of
EPA OIG Reports Issued in Fiscal Year 2017,
issued July 9, 2018.
Assistant Inspector
General for Audit
and Evaluation
R
9/30/19
2
Establish internal controls to verify that personal
impairment forms are timely completed and
readily accessible for all staff.
Assistant Inspector
General for Audit
and Evaluation
R
10/30/19
Source: OIG.
19-N-0167
9

-------