^tosrx
\ y^7/ I U.S. ENVIRONMENTAL PROTECTION AGENCY
V,Ro^° OFFICE OF INSPECTOR GENERAL
High-Performing Organization
Quality Control Review of
EPA OIG Reports Issued in
Fiscal Year 2016
Report No. 17-N-0295
June 28, 2017

-------
Report Contributors:
Kevin L. Chaffin
Jerri Dorsey-Hall
Janet Kasper
Abbreviations
CMR
Compliance Monitoring Review
CPE
Continuing Professional Education
CSB
U.S. Chemical Safety and Hazard Investigation Board
EPA
U.S. Environmental Protection Agency
FY
Fiscal Year
GAGAS
Generally Accepted Government Auditing Standards
IGEMS
Inspector General Enterprise Management System
OA
Office of Audit
OIG
Office of Inspector General
OM
Office of Management
OPE
Office of Program Evaluation
PLD
Product Line Director
PM
Project Manager
PMH
Project Management Handbook
Are you aware of fraud, waste or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW (2431T)
Washington, DC 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotline@epa.gov
Learn more about our OIG Hotline.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (2410T)
Washington, DC 20460
(202) 566-2391
www.epa.gov/oiq
Subscribe to our Email Updates
Follow us on Twitter @EPAoig
Send us your Project Suggestions

-------
^EDSrx
* A \
\ Wj
%pR0^°
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
17-N-0295
June 28, 2017
Why We Did This Review
The U.S. Environmental
Protection Agency's (EPA's)
Office of Inspector General
(OIG) operates and maintains
a system of quality controls
designed to provide
reasonable assurance that
personnel performing audit or
evaluation functions comply
with all generally accepted
government auditing standards
(GAGAS) and established OIG
policies and procedures.
Quality assurance staff
from the OIG's Office of
Management, Office of Audit,
and Office of Program
Evaluation report annually on
systemic issues identified
during referencing and
compliance monitoring
reviews. They also make
observations on compliance
with GAGAS and OIG policy.
This report addresses the
following EPA OIG goal:
• Contribute to improved
business practices and
accountability.
Quality Control Review of EPA OIG Reports
Issued in Fiscal Year 2016
What We Found
During the fiscal year (FY) 2016 quality
assurance monitoring process, the OIG
continued to make internal
improvements related to the planning
and supervision of audits and
evaluations, quality of evidence
collected, and reporting of audit status
and accomplishments in OIG
information systems. Nonetheless,
further improvements can be made.
OIG reports issued in FY 2016
demonstrated high levels of
compliance with OIG quality
assurance procedures and
received average compliance
scores of 93 percent. Most of the
issues identified during the
FY 2015 review have improved.
The FY 2016 review noted a few
additional areas for improvement.
Product Line Directors and staff have improved their ensuring that working
papers are clear, concise and easy to follow. Staff also continued to resolve
Project Manager and Product Line Director comments in working papers in a
timely manner, and to ensure that the clearance of comments is documented in
working papers. Personal impairment forms were clearly legible to the reviewer
based on action taken in response to our recommendation in the prior quality
control review. In addition, a review of a sample of EPA OIG staff training
records showed that they met the required Continuing Professional Education
requirement for the 1-year period ending September 30, 2016.
We found that improvements should be made in the frequency and
documentation of agency status meetings. We noted that 12 of the 54
assignments scored, or 22 percent, lacked documentation of meetings or that
the agency was updated on the status of findings during the meeting. Also, the
agency was not regularly updated throughout the assignment; specifically,
agency status meetings were not consistently provided every 4 to 6 weeks.
Recommendations for Improvement
We recommend that the Deputy Inspector General require OIG managers to
reinforce Project Management Handbook and OIG policy requirements that
teams have regular status meetings at least every 4 to 6 weeks or as agreed to
with the reviewed entity. Projects should begin with a documented kickoff
meeting and continue with regular documented status meetings throughout the
project.
Deputy Inspector General Response
Send all inquiries to our public
affairs office at (202) 566-2391	The Deputy Inspector General agreed with this report's recommendation and
or visit www.epa.gov/oig.	suggestions, and directed the Office of Audit and Office of Program Evaluation
to provide specific milestone and/or completion dates. The Office of Audit and
Listing of OIG reports.	Office of Program Evaluation implemented the corrective actions to meet the
intent of the recommendation and suggestions.

-------
^EDSX
*>'	r0	UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
| r-SbB-7 5	WASHINGTON, D.C. 20460
1 vTv7 *
x
"
-------
Quality Control Review of EPA OIG Reports
Issued in Fiscal Year 2016
17-N-0295
Table of C
Chapters
1	Introduction		1
Purpose		1
Background		1
Measuring Adherence to Quality Control Elements in OIG Reports		2
Noteworthy Achievements		3
Scope and Methodology		4
2	High Levels of Compliance Noted, but Documentation and
Frequency of Meetings With Agency Can Be Improved		5
Deputy Inspector General Response and OIG Evaluation		6
3	Other Matters for Consideration to Strengthen
Adherence to Quality Control Elements		7
Presentation of Audit Guide Documentation		7
Quality of Indexing		8
Deputy Inspector General Response and OIG Evaluation		8
4	Monitoring of Independence and CPE Compliance		9
Annual Personal Impairment Form		9
CPE Monitoring and Documentation		9
Appendices
A OIG Reports Reviewed With CM R for FY 2016	 11
B OIG CMR Results for FY 2016	 13
C Planned Implementation Dates for Recommendations
and Suggestions for Improvement	 15
-------
Chapter 1
Introduction
Purpose
The U.S. Environmental Protection Agency (EPA) Office of Inspector General
(OIG) operates and maintains a system of quality control designed to provide
reasonable assurance that all personnel performing audit or evaluation functions
comply with generally accepted government auditing standards (GAGAS) and
established OIG policies and procedures. Quality Assurance staff from the OIG's
Office of Management (OM), Office of Audit (OA), and Office of Program
Evaluation (OPE) analyze and summarize the results of their monitoring
procedures at least annually. In addition, these offices identify any systemic or
repetitive issues needing improvement, along with recommendations for
corrective action.
This report summarizes our observations from our review of compliance
monitoring reviews (CMR) for 30 OA and 21 OPE assignments for which reports
were issued from October 1, 2015, through September 30, 2016.1 Follow-up on
the completion of the proposed corrective action will be included as part of the
fiscal year (FY) 2018 monitoring cycle.
Background
The Inspector General Act of 1978, as amended, requires that federal Inspectors
General comply with standards established by the Comptroller General of the
United States for audits of federal establishments, organizations, programs,
activities and functions. The OIG conducts its audits and evaluations in
accordance with these standards, known as GAGAS. The OIG also maintains an
internal system of quality controls to provide the organization with reasonable
assurance that its products, services and personnel comply with professional
standards and applicable legal and regulatory requirements.
The OIG is scheduled to be subject to an external peer review during FY 2017.
The peer review is to provide an independent assessment by another organization
of the OIG's system of quality control that is designed to provide reasonable
assurance that the OIG and its personnel comply with professional standards and
applicable requirements. The peer review is also to include a review of policies
and procedures; selected reports; and other documentation, such as independence
1 There were 40 OA and 25 OPE reports issued during FY 2016, but only 30 OA and 21 OPE reports were evaluated
using the CMR criteria because some assignments had multiple reports; the CMR is an assessment of compliance for
the assigmnent as a whole, not each individual report. Further, non-GAGAS reports are not evaluated using the
CMR criteria.
17-N-0295
1
-------
certifications and Continuing Professional Education (CPE) records. Given the
anticipated scope of the peer review, our quality control review report planned for
FY 2017 will only cover limited areas related to monitoring of the CMRs.
Measuring Adherence to Quality Control Elements in OIG Reports
GAGAS Section 3.95 states that an audit organization:
... should analyze and summarize the results of its monitoring
process at least annually, with identification of any systemic or
repetitive issues needing improvement, along with
recommendations for corrective action. The audit organization
should communicate to appropriate personnel any deficiencies
noted during the monitoring process and make recommendations
for appropriate remedial action.
A measuring process should provide a mechanism to evaluate individual products
against specific quality criteria. The process should also present the information in
a manner that, over time, will allow the OIG to assess adherence to quality control
elements, so that necessary adjustments can be made to policies, procedures and
activities. In July 2014, the Inspector General signed OIG Policy and Procedure
006, OIG Quality Control and Assurance Program, which identifies the OIG's
quality control and assurance process that includes internal and external
components such as the CMR. Our system of quality control includes the use of
CMRs as our ongoing periodic assessment of work completed to determine
whether the professional standards are followed and the OIG is operating
according to OIG Policy 101, Project Management Handbook (PMH). The CMR
encompasses an evaluation of activities from the start of preliminary research
(the "kickoff' meeting) to the point that a team submits a final report and closure
of the working papers. The CMR results, trends and resulting recommendations
are summarized in our annual quality control report.
Compliance with general auditing standards—such as independence, professional
judgment, competence and adherence to CPE requirements—is not part of the
CMR. The CMR examines fieldwork and reporting standards conducted in
accordance with GAGAS and the PMH by checking for compliance with
identified activities associated with preliminary research, Project Manager (PM)
and Product Line Director (PLD) indexing and certification of the report, and the
timeliness of the assignment against proposed timeframes.
The PMH is the OIG's guide for conducting all reviews in accordance with
GAGAS financial standards (GAGAS Chapter 4) and fieldwork and reporting
standards (GAGAS Chapters 6 and 7, respectively), as well as other professional
standards. The reports evaluated with the CMR are listed in Appendix A; the
actual scoring of the reports is shown in Appendix B.
17-N-0295
2
-------
In October 2016, the Inspector General signed the revised PMH, which serves as
the OIG's guidebook for complying with the Inspector General Act of 1978, as
amended, and with Government Auditing Standards. Other elements associated
with reporting, post reporting and data quality have also been identified for
evaluation. Table 1 provides the scoring and categories associated with the CMR.
Table 1: CMR scoring and categories
Quality control category
Points
Planning and Execution
15 points
Evidence
20 points
Supervision
30 points
Reporting
20 points
Post Reporting/Data Quality
15 points
Source: OIG PMH 2012
The CMR was revised during the FY 2017 update of the PMH. The revisions to
the CMR are based on comments from previous quality control and peer reviews.
We revised CMR measurement groupings and adjusted the value related to some
of the quality control categories.
Noteworthy Achievements
In FY 2016, the OIG continued to make improvements to ensure audit and
evaluation reports, records of independence certifications, and documentation of
training adhered to applicable GAGAS and OIG policies and procedures. During
our analysis, we found that the OIG implemented six out of seven
recommendations for improvement from the last quality assurance review issued
July 18, 2016 (EPA OIG Report No. 16-N-0223). The only open recommendation
from the prior review involves ongoing work to include in OIG Procedure 102,
OIG Independence, an Appendix A (Personal Impairment Form) for staff to
complete annually. In the interim, the OIG staff implemented a temporary
practice that involves OIG staff using the enhanced impairments form to
document independence for this current fiscal year. In addition, the OIG
continued to make significant achievements since the last review, including for
planning and supervision of audits and evaluations, quality of evidence collected,
and the reporting of audit status and accomplishment in OIG information systems.
In addition, on November 17, 2016, the OIG held an all-hands lunch-and-learn
session that covered all of the recent revisions to the PMH, including:
•	Adding guidelines for conducting internal reviews.
•	Adding requirements for initiation and acceptance of audits.
•	Replacing flowcharts with listings of activity sequence at beginning of
chapters.
•	Describing the process when preliminary research is not conducted.
•	Adding a requirement for a 30-day meeting.
•	Replacing quick reaction and early warning reports with management alerts.
17-N-0295
3
-------
•	Clarifying the process after the go/no-go meeting.
•	Changing report-type descriptions.
•	Replacing quality assurance checklists with GAGAS compliance
checklists.
•	Deleting some appendices and moving them to the OIG intranet.
•	Updating the compliance monitoring checklist.
Scope and Methodology
We performed this review on assignments with reports issued from October 1,
2015, to September 30, 2016. This review covered final GAGAS-compliant
reports that were issued by OA and OPE during this period, and were reviewed
and scored by the OIG's quality assurance staff. We did not include any reports
with work performed by external auditors.
We reviewed the cost and time data stored in the Inspector General Enterprise
Management System (IGEMS) for each OIG audit and evaluation project scored
for quality. We reviewed the assignment working papers in the OIG's
AutoAudit® working paper system, and analyzed the final reports using the
applicable scoring form. Our review also consisted of examining sampled OIG
staff personal impairment certifications and CPE requirements to determine
overall compliance with GAGAS. In addition, during the CMR, we contacted the
supervisors on each assignment, as needed, to obtain additional information. The
work performed in this review does not constitute an audit conducted in
accordance with GAGAS.
The OIG's quality assurance team decided that an issue was significant enough to
be included in this report if areas of concern from the CMRs exceeded 20 percent
of the total reports scored. Areas of concern warrant the attention of leadership
and corrective actions to resolve the issues identified during this review. If an
issued was identified frequently, but in less than 20 percent of the assignments,
the issue is presented as an "other matter" for consideration.
17-N-0295
4
-------
Chapter 2
High Levels of Compliance Noted, but
Documentation and Frequency of Meetings
With Agency Can Be Improved
During the FY 2016 quality assurance monitoring process, OIG reports continued
to demonstrate high levels of compliance with OIG quality assurance procedures,
and received average compliance scores of 93 percent or greater. In addition, we
noted improvements for most of the issues identified during the FY 2015 quality
assurance monitoring process. Nevertheless, we found that improvement should
be made in the frequency and documentation of status meetings with the agency.
It is vital that OIG communication with the reviewed entity be held with the
appropriate frequency and be clearly documented. These meetings provide a
valuable real-time opportunity to discuss with the agency progress, time frames,
and preliminary observations or findings. Improvements will aide in enhancing
and strengthening OIG adherence to quality control elements.
We determined that working paper documentation and the holding of agency
status meetings could be improved. Twelve out of 54 assignments evaluated
(22 percent) contained instances where some working papers had weaknesses
regarding documentation pertaining to the team not having regularly scheduled
status meetings with the agency, and communicating with the entity's action
officials every 4 to 6 weeks. This negatively impacted the team's ability to
adequately communicate issues, preliminary observations, and status of work with
the reviewed entity. Communication with the reviewed entity should be clearly
documented in the working papers and held periodically throughout the audit.
Audit teams should establish a section in the working papers that documents all
communications with the reviewed entity regarding audit status, findings,
conclusions and recommendations.
GAGAS Section A1.05 states that, during the course of GAGAS audits, auditors
should communicate with those charged with governance. Those charged with
governance are responsible for overseeing the strategic direction of the entity.
PMH Section 1.17 states that:
... to help successfully facilitate a project that is fair, complete and
objective, the OIG's policy is to communicate issues, preliminary
observations, and the status of our work with the reviewed entity at
a regular interval (at least every 4 to 6 weeks or as agreed to with
the reviewed entity), beginning with the kickoff meeting.
17-N-0295
5
-------
We recommend that the Deputy Inspector General direct OA and OPE staff to:
1. Reinforce the PMH and OIG policy requirements that teams have regular
status meetings at least every 4 to 6 weeks or as agreed to with the reviewed
entity. Projects should begin with a documented kickoff meeting and
continue with regular documented status meetings throughout the project.
Deputy Inspector General Response and OIG Evaluation
The Deputy Inspector General agreed with our recommendation. As a result of
our recommendation, OA and OPE conducted training presentations with staff on
April 19 and June 21, 2017. The presentations were held to reinforce the PMH
and GAGAS requirement that working papers contain appropriate documentation
regarding communication with the entity throughout the project. The presentation
also reminded staff of the PMH requirement that teams have regular status
meetings (at least every 4 to 6 weeks or as agreed to with the reviewed entity),
beginning with the kickoff meeting, and document the meetings in the working
papers. The corrective actions meet the intent of the recommendation. The
planned implementation date is included in Appendix C.
17-N-0295
6
-------
Chapter 3
Other Matters for Consideration to
Strengthen Adherence to Quality Control Elements
During this review, as with prior quality control reviews, we continued to identify
other areas in which the OIG can enhance and strengthen its project execution
process and improve adherence to quality control elements. These areas involve
audit guide documentation and the overall quality of indexing.
Presentation of Audit Guide Documentation
The presentation of audit guide documentation can be improved. For 10 of the
54 assignments evaluated (18 percent), we found that the audit guide was not
approved timely, revisions to the audit guide were not approved, or the audit
guide was not completely indexed to supporting working papers. The audit guide
is the team's most tangible tool for managing its work from preliminary research
to issuance of a final product, and is an essential element of audit quality. Audit
teams must ensure that audit guide documentation contains timely approvals prior
to the kickoff meeting and/or entrance conference and periodic approvals related
to revisions, and that they index the guide to the supporting working papers.
GAGAS Section 6.51 states that auditors must prepare a written audit plan for
each audit. The form and content of the written audit plan may vary among audits,
and may include an audit strategy, audit program, project plan, audit planning
paper, or other appropriate documentation of key decisions about the audit
objectives, scope and methodology, and the auditors' basis for those decisions.
Auditors should update the plan as necessary to reflect any significant changes to
the plan made during the audit. PMH Section 2.5 states that the PLD is to review
and approve the guide prior to the kickoff meeting and/or entrance conference.
In addition, the PMH also states that the project guide is a living document that
should be adjusted throughout the course of the project, with revisions approved
by the PM or PLD. This project guide issue should have been identified during
the team and PLD review of the working papers.
We suggest that the Deputy Inspector General direct OA and OPE staff to:
2.	Reinforce the PMH and GAGAS requirement that working papers contain
approved project audit guides prior to the kickoff meeting and/or entrance
conference, and that significant changes need to be re-approved.
3.	Ensure that all audit steps are indexed to working papers or that the PM or
PLD has approved the step not being performed before the working papers
are closed.
17-N-0295
7
-------
Quality of Indexing
Quality of indexing has improved since reported in the 2015 review, in which we
noted that 11 out of 50 reports evaluated (22 percent) contained instances where
the overall quality of indexing could be improved. During our 2016 review, we
identified that this issue has decreased. Specifically, we found that seven out of
54 assignments evaluated (13 percent) were not always indexed to the source
document, did not use summary working papers to support overall conclusions, or
otherwise needed improvement to the indexing quality. Indexing issues result in
the need for additional time spent in referencing, and delay report issuance.
GAGAS Section 6.82 and PMH Section 3.5 require auditors to obtain sufficient,
appropriate evidence to provide a reasonable basis for their findings and
conclusions in their reports. OIG conclusions and opinions in draft and final
reports, summaries and finding outlines must be indexed to the supporting audit
working papers, and show the complete facts and rationale for a conclusion or
opinion. GAGAS identified referencing as a quality control process to help audit
organizations prepare accurate audit reports (GAGAS Section A7.02a). Accurate
indexing facilitates the process of ensuring the quality of reports, and also helps to
reduce the time it takes for reports to go through the quality assurance process.
We suggest that the Deputy Inspector General direct OA and OPE staff to:
4.	Reinforce GAGAS and PMH requirements related to indexing, and the
role of indexing in the OIG's quality control process, to the teams
requiring additional training in this area.
5.	Reinforce the "indexing tools" section on the OIG intranet to the teams
requiring additional training in this area.
Deputy Inspector General Response and OIG Evaluation
The Deputy Inspector General agreed with our suggestions. OA and OPE training
presentations on April 19 and June 21, 2017, reinforced the PMH and GAGAS
requirement that working papers contain approved project audit guides prior to the
kickoff meeting and/or entrance conferences, and that significant changes need to
be re-approved. Staff was reminded to ensure that all audit steps are indexed to
working papers or that the PM or PLD has approved the step not being performed
before the working papers are closed. Further, the presentation included an
overview of the importance of indexing, the roles indexing plays in the OIG's
quality control process, and resources available on the OIG intranet to assist if
additional training is needed. The presentation also reminded staff of the "indexing
tools" section on the OIG intranet that provides guidance on indexing. The
corrective actions meet the intent of the suggestions. The planned implementation
dates are included in Appendix C.
17-N-0295
8
-------
Chapter 4
Monitoring of Independence and CPE Compliance
As part of the FY 2016 monitoring cycle, we conducted a review to confirm that
all audit and evaluation staff assigned to the audits and evaluations scored were in
compliance with the requirement to declare and report personal impairments.
We also sought to determine whether OIG audit and evaluation staff were meeting
CPE requirements. We found that all staff reviewed met requirements to declare
and report personal impairments, and all staff reviewed met CPE requirements.
Annual Personal Impairment Form
GAGAS Section 3.59 states that independence documentation must provide
evidence of the auditor's judgments in forming conclusions regarding compliance
with independence requirements. OIG Policy and Procedure 501, Ethics Roles
and Responsibilities, was posted to the OIG Policy and Procedures intranet page
on March 17, 2017. The updated procedure deleted Appendix A, Personal
Impairment Form, and related guidance, to move the form to OIG Procedure 102,
OIG Independence. OIG Policy and Procedure 102, OIG Independence, will be
amended to state that staff and contractors must be independent, in fact and
appearance, and sign a personal impairment form upon joining the OIG and
annually as performance agreements are established (Section 2.1 and Appendix A,
Standards of Conduct).
The review confirmed that 30 out of 30 staff sampled (100 percent) completed the
personal impairment certifications. Current OIG policies and procedures for
completing annual independence certifications appear to be adequate. Based on
our previous recommendation made in the FY 2015 quality control review to
revise the personal impairment form, OIG offices are using the improved
impairment form that includes a space for employees to indicate a specific fiscal
year.
CPE Monitoring and Documentation
As part of the FY 2016 monitoring cycle, we reviewed information entered into
the IGEMS Training Module. We reviewed a random sample of 15 OA and
15 OPE staff for the 1-year period of the 2-year training cycle that ends on
September 30, 2017. The 30 EPA OIG staff were randomly selected from staff
identified as needing to comply with the GAGAS CPE requirements. The
evaluation was performed to determine whether the individuals met the GAGAS
CPE requirements that specify at least 20 hours are earned in year 1 of the
training cycle. Auditors and evaluators must have, for the 2-year period, a total of
24 hours of training in government auditing, the government environment, or
17-N-0295
9
-------
specialized information; and 56 other hours of training related to conducting
audits during the 2-year period. Auditors and evaluators performing work
(e.g., planning, directing, performing or reporting) should maintain their
professional competence through CPEs, as required by GAGAS Section 3.76.
All EPA OIG staff sampled met the required CPE requirement for the 1-year
period ending September 30, 2016. Employees and supervisors are expected to
continue to meet CPE requirements and have periodic discussions to ensure
continued compliance.
17-N-0295
10
-------
Appendix A
OIG Reports Reviewed With CMR for FY 2016
Publication No.
Assignment No.
Title
16-P-0006
OA-FY14-0078
EPA Needs to Improve Security Planning and Remediation of Identified Weaknesses in Systems Used to Protect Human
Health and the Environment
16-F-0251
OA-FY15-0174
Audit of Financial Statements for EPA's Hazardous Waste Electronic Manifest System Fund From Inception (October 5, 2012)
Through September 30, 2014
16-P-0081
OA-FY14-0381
EPA's Tracking and Reporting of Its Conference Costs Need Improvement
16-P-0107
OA-FY15-0044
Positioning EPA for the Digital Age Requires New Mindsets Toward Printing
16-P-0222
OA-FY15-0153
EPA Regional Offices Need to More Consistently Conduct Required Annual Reviews of Clean Water State Revolving Fund
16-P-0167
OA-FY16-0049
EPA Complied With Improper Payment Legislation, but Stronger Internal Controls Are Needed
16-P-0035
16-P-0086
OA-FY15-0173 *
CSB Needs Better Security Controls to Protect Critical Data Stored on Its Regional Servers
16-P-0036
OA-FY15-0187
Administrative Leave Decisions for EPA Employee Disciplinary Actions Should Be Better Documented, and Parameters on
Use of Such Leave Should Be Established
16-P-0039
OA-FY14-0381
FY 2015 FISMA Report - Status of EPA's Information Security Program
16-P-0048
OA-FY15-0026
Awards Made by EPA's Office of the Chief Financial Officer Raise Questions
16-P-0100
OA-FY14-0056
EPA Needs to Improve Its Information Technology Audit Follow-Up Processes
16-P-0109
OA-FY16-0058
CSB Complied With Improper Payment Legislation Requirements for Fiscal Year 2015
16-P-0111
OA-FY15-0026
Management of Overtime Improved at EPA's Immediate Office of Air and Radiation
16-P-0124
OA-FY15-0276
EPA's Fiscal Year 2015 Purchase Card and Convenience Check Program Assessed as Low Risk
16-P-0135
OA-FY14-0386
EPA Should Timely Deobligate Unneeded Contract, Purchase and Miscellaneous Funds
16-F-0040
OA-FY15-0176
Audit of EPA's Fiscal Years 2015 and 2014 Consolidated Financial Statements
16-P-0268
OA-FY15-0080
EPA Needs to Improve Oversight of Its Transit Subsidy Benefits Program
16-P-0260
OA-FY16-0059
CSB Has Improved Its Controls Over Purchase Cards
16-P-0259
OA-FY16-0126
Cybersecurity Act of 2015 Report: EPA's Policies and Procedures to Protect Systems With Personally Identifiable Information
16-P-0254
OA-FY16-0127
Cybersecurity Act of 2015 Report: CSB's Policies and Procedures to Protect Systems With Personally Identifiable Information
16-P-0212
OA-FY13-0104
EPA Improved Controls Over Billing Reimbursable Interagency Agreement Expenditures to Other Agencies
16-P-0218
OA-FY16-0102
Hawaii Department of Health Needs to Reduce Open Grants and Unspent Funds
16-P-0217
OA-FY15-0054
EPA's Financial Oversight of Superfund State Contracts Needs Improvement
16-P-0179
16-P-0112
OA-FY15-0031 *
CSB Needs to Continue to Improve Agency Governance and Operations
16-P-0207
16-P-0166
16-P-0181
OA-FY14-0035 *
EPA Region 9 Needs to Improve Oversight Over Commonwealth of the Northern Mariana Islands Consolidated Cooperative
Agreements
16-P-0078
OA-FY15-0029
EPA's Background Investigation Support Contracts and OPM Billings Need Better Oversight and Internal Controls
16-P-0276
OA-FY16-0024
EPA Region 9 Needs to Improve Oversight of San Francisco Bay Water Quality Improvement Fund Grants
16-P-0282
OA-FY15-0156
EPA Oversight of Travel Cards Needs to Improve
17-N-0295
11
-------
Publication No.
Assignment No.
Title
16-F-0322
OA-FY15-0053
Fiscal Years 2014 and 2013 Financial Statements for the Pesticides Reregistration and Expedited Processing Fund
16-F-0323
OA-FY15-0052
Fiscal Years 2014 and 2013 Financial Statements for the Pesticide Registration Fund
16-P-0333
OA-FY15-0180
Enhanced Controls Needed to Prevent Further Abuse of Religious Compensatory Time
16-P-0194
OPE-FY15-0055
EPA Needs Better Data, Plans and Tools to Manage Insect Resistance to Genetically Engineered Corn
16-P-0101
OPE-FY15-0054
Follow-Up: EPA Has Developed Measures to Improve Training for Risk Management Program Inspectors
16-P-0104
OPE-FY15-0018
EPA Has Not Met Statutory Requirements for Hazardous Waste Treatment, Storage and Disposal Facility Inspections, but
Inspection Rates Are High
16-P-0082
OPE-FY14-0039
EPA's Bristol Bay Watershed Assessment: Obtainable Records Show EPA Followed Required Procedures Without Bias or
Predetermination, but a Possible Misuse of Position Noted
16-P-0108
OPE-FY14-0047
Drinking Water: EPA Needs to Take Additional Steps to Ensure Small Community Water Systems Designated as Serious
Violators Achieve Compliance
16-P-0019
OPE-FY15-0004
EPA Needs Policies and Procedures to Manage Public Pesticide Petitions in a Transparent and Efficient Manner
16-P-0079
OPE-FY14-0011
EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring
Network
16-P-0125
OPE-FY15-0017
EPA Offices Are Aware of the Agency's Science to Achieve Results Program, but Challenges Remain in Measuring and
Internally Communicating Research Results That Advance the Agency's Mission
16-P-0122
OPE-FY15-0020
No Intent to Underestimate Costs Was Found, but Supporting Documentation for EPA's Final Rule Limiting Sulfur in Gasoline
Was Incomplete or Inaccurate in Several Instances
16-P-0059
OPE-FY15-0019
EPA Is Documenting How It Addresses Time-Critical Public Health Risks Under Its Superfund Authority
16-P-0219
OPE-FY15-0012
EPA Has Developed Guidance for Disaster Debris but Has Limited Knowledge of State Preparedness
16-P-0211
OPE-FY16-0012
Follow-Up Report: EPA Has Completed Actions to Improve Implementation of the Rulemaking Process
16-P-0162
OPE-FY15-0009
EPA Needs to Assess Environmental and Economic Benefits of Completed Clean Water State Revolving Fund Green
Projects
16-P-0164
OPE-FY15-0015
Clean Air Act Facility Evaluations Are Conducted, but Inaccurate Data Hinder EPA Oversight and Public Awareness
16-P-0296
OPE-FY14-0044
Progress Made, but Improvements Needed at CTS of Asheville Superfund Site in North Carolina to Advance Cleanup Pace
and Reduce Potential Exposure
16-P-0316
OPE-FY16-0001
EPA Needs a Risk-Based Strategy to Assure Continued Effectiveness of Hospital-Level Disinfectants
16-P-0279
OPE-FY15-0021
EPA Achieved Scientific Benefits When Using Reimbursable Research Agreements, but Better Estimating of In-Kind Costs Is
Needed
16-P-0275
OPE-FY16-0005
EPA Has Not Met Certain Statutory Requirements to Identify Environmental Impacts of Renewable Fuel Standard
16-P-0246
OPE-FY15-0001
EPA Cannot Assess Results and Benefits of Its Environmental Education Program
16-P-0308
OPE-FY16-0020
Follow-Up Review: EPA Updated Information for Indoor Mold Research Tools
16-P-0196
OPE-FY15-0057
EPA Improved Its National Security Information Program, but Some Improvements Still Needed
16-P-0313
OA-FY15-0118
Oregon Health Authority's Prior Labor Charging Practices Under EPA Grants Do Not Meet Requirements
16-P-0320
OA-FY14-0282
Manchester Band of Pomo Indians Needs to Improve its Financial Management System and Demonstrate Completion of
Grant Work
'Multiple reports were issued for a few assignments scored during FY 2016.
17-N-0295
12
-------
Appendix B
OIG CMR Results for FY 2016
Publication No.
Planning
Evidence
Supervision
Reporting
Post reporting/
data accuracy
Compliance
review
score
16-P-0006
9.5
20
27.75
16
13
89.25
16-F-0251
15
20
26
20
15
96
16-P-0081
15
19.5
30
18
15
97.5
16-P-0107
12
10
21
12
14
79
16-P-0222
14.5
20
30
16
15
95.5
16-P-0167
15
20
26.25
20
12
93.25
*16-P-0035
16-P-0086
15
20
29.25
19
10
93.25
16-P-0036
15
20
28.3
20
14
97.3
16-P-0039
12
20
30
20
15
97
16-P-0048
14
20
30
20
15
99
16-P-0100
13
20
30
12
15
90
16-P-0109
15
20
30
20
14
99
16-P-0111
15
20
28.5
20
15
98.5
16-P-0124
15
20
30
20
15
100
16-P-0135
9
19
29.0
11
13
81
16-F-0040
15
20
28.5
20
15
98.5
16-P-0268
15
20
27
20
15
97
16-P-0260
15
17
30
18
15
95
16-P-0259
15
20
30
18
15
98
16-P-0254
15
20
30
18
15
98
16-P-0212
4
20
15
17
8
64
16-P-0218
15
20
30
20
15
100
16-P-0217
15
18
29
20
15
97
*16-P-0179
16-P-0112
9
20
29.25
20
15
93.25
*16-P-0207
16-P-0166
16-P-0181
14
15
23.5
20
15
87.5
16-P-0078
15
20
27
20
15
97
17-N-0295
13
-------
Publication No.
Planning
Evidence
Supervision
Reporting
Post reporting/
data accuracy
Compliance
review
score
16-P-0276
15
20
30
20
15
100
16-P-0282
14
20
30
14
15
94
16-F-0322
15
20
28.5
11.5
11
86
16-P-0194
14
20
27
20
14
95
16-F-0323
15
20
28
14
14
91
16-P-0333
13
20
27
8
15
83
16-P-0101
14
12
10
19
14
69
16-P-0104
14
20
25.7
20
15
94.7
16-P-0082
15
20
28.8
20
15
98.8
16-P-0108
12
20
27
20
15
94
16-P-0019
15
12
24.6
20
12
83.6
16-P-0079
11
20
29.4
20
14
94.4
16-P-0125
15
15
28.8
20
12
90.8
16-P-0122
13
20
24.25
18
15
90.25
16-P-0059
15
20
29.4
14
15
99.4
16-P-0219
15
20
28.8
20
15
99.8
16-P-0211
15
14
30
20
15
94
16-P-0162
14
20
29.4
18
10.5
91.9
16-P-0164
14
18
30
20
15
97
16-P-0296
14
20
24
18
14
90
16-P-0316
14
20
30
19
15
98.0
16-P-0279
15
20
28.8
18
15
96.8
16-P-0275
14
20
29.4
20
15
98.4
16-P-0246
15
20
27.6
20
15
97.6
16-P-0308
15
20
30
20
13
98.0
16-P-0196
15
20
29.4
20
15
99.4
16-P-0320
15
20
29
16
15
95.0
16-P-0313
14
20
30
20
14
98.0
Total
676.50
940.00
1,366.90
868.50
677.00
4,483.45
Average
13.81
19.18
27.90
18.09
14.10
93.41
No. of Reports
54





'Multiple reports were issued for a few assignments scored during FY 2016. For the purpose of developing averages, these reports are
treated as one report instead of multiple reports.
17-N-0295
14
-------
Appendix C
Planned Implementation Dates for
Recommendations and Suggestions for Improvement
Recommendation/
suggestion No.
Page No.
Recommendation/suggestion for improvement
Status
OIG
action
office
Planned
implementation
date
1
6
Reinforce the PMH and OIG policy requirements that teams
have regular status meetings at least every 4 to 6 weeks or
as agreed to with the reviewed entity. Projects should begin
with a documented kickoff meeting and continue with regular
documented status meetings throughout the project.
Corrective
action
implemented
OA
OPE
6/21/17
2
7
Reinforce the PMH and GAGAS requirement that working
papers contain approved project audit guides prior to the
kickoff meeting and/or entrance conference, and that
significant changes need to be re-approved.
Corrective
action
implemented
OA
OPE
6/21/17
3
7
Ensure that all audit steps are indexed to working papers or
that the PM or PLD has approved the step not being
performed before the working papers are closed.
Corrective
action
implemented
OA
OPE
6/21/17
4
8
Reinforce GAGAS and PMH requirements related to
indexing, and the role of indexing in the OIG's quality control
process, to the teams requiring additional training in this
area.
Corrective
action
implemented
OA
OPE
6/21/17
5
8
Reinforce the "indexing tools" section on the OIG intranet to
the teams requiring additional training in this area.
Corrective
action
implemented
OA
OPE
6/21/17
17-N-0295
15
-------