U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Audit Report
EPA Should Further Connect the
National Program Manager
Process With Federal Guidance
on Internal Control Risks
Report No. 11-P-0067
January 18, 2011
-------
Report Contributors: Patrick Gilbride
Erin Barnes-Weaver
Karen L. Hamilton
Mary Anne Strasser
Stephanie Wake
Abbreviations
EPA U.S. Environmental Protection Agency
FMFIA Federal Managers' Financial Integrity Act
FY Fiscal year
GAO U.S. Government Accountability Office
GPRA Government Performance and Results Act of 1993
NPM National program manager
OCFO Office of the Chief Financial Officer
OCSPP Office of Chemical Safety and Pollution Prevention
OMB Office of Management and Budget
OW Office of Water
PAR Performance and Accountability Report
-------
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
11-P-0067
January 18, 2011
Why We Did This Review
We conducted this review to
determine how EPA's national
program manager (NPM)
process relates to the internal
control framework under the
Federal Managers' Financial
Integrity Act (FMFIA). We
determined whether the U.S.
Environmental Protection
Agency (EPA) should
improve connections between
the two processes and whether
NPMs and regions coordinate
program management and
address risks and
vulnerabilities.
Background
FMFIA requires federal
agency managers to annually
evaluate and indicate whether
their agencies' internal
controls comply with
prescribed standards. NPM
guidance sets forth goals and
program priorities to support
compliance with the
Government Performance and
Results Act of 1993.
For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202)566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2011/
20110118-11-P-0067.pdf
Catalyst for Improving the Environment
EPA Should Further Connect the National
Program Manager Process With Federal
Guidance on Internal Control Risks
What We Found
EPA has not fully integrated FMFIA and the NPM processes. Activities conducted
per the NPM process support internal controls; however, EPA's Office of the
Chief Financial Officer did not connect these processes until midway through
fiscal year 2009 (in supplemental guidance) and in fiscal year 2010 guidance, and
integration efforts are still in their infancy. NPMs already conduct many activities
related to internal control, yet national program offices have separate processes
and staff responsible for each process. Having national program offices primarily
responsible for internal controls over national programs would streamline
reporting and lessen confusion among staff involved in both processes.
NPMs have not linked assessing and evaluating relevant risks associated with
achieving program objectives to internal control requirements. FMFIA requires
managers to define program goals and identify key programs, complete a risk
assessment based on their priorities, and then establish controls to mitigate
identified program risks. National program offices and regions do not appear to
completely understand the risk assessment internal control standard and how to
apply it to program operations. Without consistently conducting risk assessments,
EPA lacks a sound, documented basis for reasonably assuring that programs
implement effective internal controls consistent with federal internal control
standards. Additional training on risk assessment, including how to identify
weaknesses, determining how to manage risks, and how to conduct necessary
internal control reviews, should improve program management.
What We Recommend
We recommend that the Chief Financial Officer assign NPMs primary
responsibility for FMFIA reporting on internal controls for national programs and
rely on the lead regional coordinator process for input from the regions, and direct
regional personnel to report on administrative and financial internal control
activities along with unique geographic and programmatic issues in regional
assurance letters. We also recommend that the Chief Financial Officer develop a
training course on FMFIA and enhance the FMFIA intranet site by providing
links to risk assessment guidance and completed products that offices could use as
best practices. The Agency agreed with our recommendations and began taking
steps to address them.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
January 18,2011
EPA Should Further Connect the National Program Manager Process
With Federal Guidance on Internal Control Risks
Report No. ll-P-0067
FROM:
TO:
Arthur A. Elkins, Jr
Inspector General
Barbara J. Bennett
Chief Financial Officer
/
The U.S. Environmental Protection Agency (EPA) Office of Inspector General issued this report
on the subject audit. This report contains findings that describe problems we identified and
corrective actions we recommend. This report represents our opinion and does not necessarily
represent the final EPA position. EPA managers will make final determinations on matters in this
report in accordance with established audit resolution procedures.
The estimated cost of this report, calculated by multiplying the project's staff days by the
applicable daily full cost billing rates in effect at the time, is $472,472.
Action Required
On November 22, 2010, your office provided comments to our report, and we discussed your
planned corrective actions and milestone dates on December 15, 2010. We believe your planned
corrective actions address the intent of each of our recommendations. As such, we plan to close
this assignment upon issuance of this final report. We have no objections to the further release of
this report to the public. This report will be available at http://www.epa.gov/oig.
If you or your staff has any questions regarding this report, please contact Melissa Heist,
Assistant Inspector General for Audit, at (202) 566-0899 or heist.melissa@epa.gov: or
Patrick Gilbride, Director for Audit, Risk and Program Performance Issues, at (303) 312-6969
or gilbride.patrick@epa.gov.
-------
EPA Should Further Connect the National Program Manager 11 -P-0067
Process With Federal Guidance on Internal Control Risks
Table of Contents
Chapters
1 Introduction 1
Purpose 1
Background 1
Noteworthy Achievements 4
Scope and Methodology 5
2 FMFIA and NPM Processes Not Integrated 6
FMFIA and NPM Processes Have Common Elements 6
Program Office Interpretations Vary on Degree of Integration 8
Regional Personnel Unclear on Assurance Letter Content 9
Conclusion 10
Recommendations 10
Agency Comments and OIG Evaluation 10
3 Risk Assessment Internal Control Standard 12
Risk Assessments Not Informing Program Reviews 12
Conclusion 14
Recommendations 15
Agency Comments and OIG Evaluation 15
Status of Recommendations and Potential Monetary Benefits 16
Appendices
A Details on Scope and Methodology 17
B Agency Response to Draft Report 20
C Distribution 23
-------
Chapter 1
Introduction
Purpose
The U.S. Environmental Protection Agency's (EPA's) national program managers
(NPMs) develop annual guidance documents to define program priorities,
strategies, and performance measures in accordance with the Agency's strategic
plan, annual plan and budget, and the Administrator's priorities. We conducted
this review to determine how the NPM process relates to the internal control
framework under the Federal Managers' Financial Integrity Act (FMFIA), and
whether the Agency should improve connections between the two processes. We
also determined whether NPMs and regional personnel coordinate program
management and whether this coordination addresses program risks and
vulnerabilities.
Background
EPA annually issues the Performance and Accountability Report (PAR)1 to
describe to the President, Congress, and the public the Agency's environmental
program and financial performance during the fiscal year. The PAR also describes
progress in addressing management issues and accountability systems and
controls. The annual PAR satisfies a number of legislative reporting requirements,
including those of the Government Performance and Results Act of 1993 (GPRA)
and FMFIA. EPA's Office of the Chief Financial Officer (OCFO) develops,
manages, and supports a goals-based management system for the Agency, which
includes preparing EPA's strategic plan, annual budget and performance plan, and
the PAR. OCFO initiates both the FMFIA and NPM processes by providing
annual guidance to EPA managers. OCFO also reports results from each process,
such as information from FMFIA assurance statements and NPM performance
results, in the Agency's annual PAR.
National Program Manager Process
GPRA requires the PARs, strategic plans, and annual performance plans to
facilitate results-oriented management. GPRA also requires agencies to clarify
their missions, set strategic and annual performance goals, and measure and report
on performance toward these goals. NPMs for each of EPA's five national
1 Effective for the fiscal year 2010 reporting period, EPA now uses an alternate reporting approach to the PAR. The
Agency financial report summarizes EPA's financial results and presents its audited financial statements, and the
annual performance report presents detailed performance results as measured against targets established in EPA's
annual plan and budget. For the purposes of our report, we will refer to the PAR, as it was the reporting approach in
place during the time we conducted our audit.
11-P-0067
-------
program offices issue annual guidance documents to initiate program planning
and establish a relationship among annual operational measures, EPA's annual
budget, and long-term strategic goals. NPMs establish national goals for their
respective programs and then evaluate and adjust national priorities as new data
on emerging environmental issues become available. EPA uses this process that
NPMs undertake while developing their guidance documents (hereafter referred
to as the "NPM process") to support compliance with GPRA requirements. NPM
annual guidance focuses on three areas:
1. Developing NPM priorities, strategies, and associated measures
2. Reporting results for prior year performance commitments
3. Negotiating agreements for performance commitments
NPMs establish these priorities, strategies, measures, and commitments through a
process of coordination and negotiation with regional personnel. EPA adopted a
methodology in 1984 to provide regions an organized, consistent, and effective
role in all major phases of Agency decisionmaking through lead regional
coordinators. Lead regional coordinators act as conduits between the regional
personnel and the NPMs to ensure ongoing regional input to EPA's national
program offices. Lead regional coordinators consolidate information from
regional personnel on priorities, emerging issues, weaknesses, and other issues for
NPMs to consider during their process and for national program offices' FMFIA
assurance letters.
OCFO issues technical guidance for national program offices to follow as they
prepare annual NPM guidance on Agency priorities. OCFO's Technical Guidance
on FY 2010 National Program Manager Guidance and Annual Commitment
Process in Measures Central requires managers to establish program priorities
and performance measures in support of GPRA requirements and serves as an
overall program management tool. This NPM process aims to support Agency
program management and decisionmaking by:
• Improving the quality, consistency, and reliability of measures and related
data and reporting
• Analyzing progress toward results in midyear reporting to aid in
negotiating draft performance commitments
• Engaging with state and tribal partners and stakeholders
EPA's Management Integrity Process
FMFIA requires federal agency managers to establish internal accounting and
administrative controls in accordance with standards prescribed by the U.S.
Government Accountability Office (GAO) in Standards for Internal Control in
the Federal Government. FMFIA also requires federal agency managers to
annually evaluate their compliance with GAO's standards and issue a statement of
full compliance or noncompliance with FMFIA (an "assurance letter"). If the
11-P-0067
-------
Administrator determines that the Agency has not fully complied with GAO's
standards, the Administrator must report internal weaknesses and a corresponding
corrective action plan in the Administrator's assurance statement.
Office of Management and Budget (OMB) Circular A-123 describes federal
managers' responsibilities for internal control and provides guidance to meet
FMFIA requirements. The circular states that internal control should be "an
integral part of the entire cycle of planning, budgeting, management, accounting,
and auditing" and "provide continual feedback to management." It also advises
agencies to combine their FMFIA reporting efforts with other ongoing efforts to
improve effectiveness and accountability.
GAO's Standards for Internal Control in the Federal Government provide an
overall framework for establishing and maintaining internal control and for
identifying and addressing major performance and management challenges and
areas at greatest risk of fraud, waste, abuse, and mismanagement (table 1).
Table 1: GAO's Standards for Internal Control in the Federal Government
Control
environment
This standard establishes and maintains an environment throughout
the organization that sets a positive and supporting attitude toward
internal control and conscientious management. Internal control and
conscientious management includes establishing goals, objectives,
and performance measures at both the entity and activity levels.
Risk
assessment
A precondition to risk assessment is the establishment of clear,
consistent agency objectives. The internal control risk assessment
process includes assessing risks the agency faces from both internal
and external sources. Management should comprehensively identify
risks and should consider all significant interactions between the entity
and other parties, as well as internal factors at both the entity and
activity levels.
Control activities
Control activities are the policies, procedures, techniques, and
mechanisms that implement management's direction to achieving
goals. Internal control activities help ensure that management's
directives are carried out.
Information and
communications
This standard includes data and information (performance and
financial) to determine whether the organization meets its goals and
objectives and maintains accountability over resources.
Monitoring
Internal control monitoring should assess the quality of performance
overtime and ensure that audits and other review findings are
promptly resolved.
Source: OIG summary of GAO's Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1, November 1999.
According to OMB Circular A-123, risk assessment forms the foundation of any
effective system of internal controls. Risk assessment includes identifying and
analyzing relevant risks associated with achieving goals and objectives, such as
those defined in strategic and annual performance plans developed under GPRA.
After an organization identifies significant areas of risk, it should develop control
11-P-0067
-------
activities to minimize or eliminate those risks. Risk analysis generally includes
estimating the risk's significance, assessing the likelihood of its occurrence, and
deciding how to manage the risk and what actions to take.
OCFO initiates the FMFIA reporting process by providing annual guidance to
EPA managers. OCFO's FMFIA guidance specifies coordination and
communication between NPMs and regional offices. New for fiscal year (FY)
2010, OCFO issued the guidance in two parts: the first part focused on financial
activities and the second part focused on program operations. OCFO intended the
second part to achieve more systematic and rigorous reviews of internal controls
over program operations and establish clear regional and national program roles
and responsibilities for reviewing controls and sharing information between
offices.
Noteworthy Achievements
OCFO has taken a number of steps to improve EPA's management integrity
program. In FY 2010, OCFO issued separate FMFIA guidance for financial and
program operations. The program guidance addressed how to conduct internal
control reviews over program operations, and also clarified responsibilities
between program offices and regions. In addition, OCFO hired a contractor in
FY 2009 to conduct FMFIA program compliance reviews in a sample of offices
to determine necessary changes to improve FMFIA implementation. OCFO
completed additional reviews in FY 2010 in Regions 9 and 10, and will continue
these reviews on a rotating basis as part of OCFO's oversight of the management
integrity program.
OCFO has also made efforts to show the relationship between the NPM process
and FMFIA by cross-referencing them in each guidance document. The FY 2010
FMFIA program guidance mentions the NPM process, while the FY 2011
Technical Guidance on the NPM Guidance mentions the FMFIA process for the
first time. OCFO's Technical Guidance on FY 2012 National Program Manager
Guidance and Annual Commitment Process notes that annual NPM guidance
documents serve as an important internal control for Agency programmatic
operations because the documents set forth program priorities and key actions for
the upcoming year. OCFO also encouraged NPMs to discuss their annual program
guidance as a key internal control in preparing FY 2011 annual letters of
assurance to the Administrator. National program offices and regions in our scope
acknowledged connections between the two processes and have taken initial steps
to integrate them.
Finally, we observed strong communication and coordination between national
program offices and regions. They establish national program priorities through
an inclusive process involving states, tribes, and other stakeholders.
11-P-0067
-------
Scope and Methodology
We conducted this audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the review
to obtain sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our review objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and conclusions
based on our review objectives.
We focused our review on FMFIA and NPM process implementation by the
Office of Water (OW), Office of Chemical Safety and Pollution Prevention
(OCSPP), and Regions 5 and 9; we also reviewed Regions 3 and 6 because they
house, respectively, the OW and OCSPP lead regional coordinators. We also
focused our review on guidance provided by OCFO to EPA managers.
We reviewed OCFO's FMFIA guidance for FYs 2008, 2009, and 2010, and
OCFO's Technical Guidance on the National Program Manager Guidance and
Annual Commitment Process in Measures Central for FYs 2010, 2011, and 2012.
We also reviewed FMFIA assurance letters for FYs 2008 and 2009 for OW,
OCSPP, and Regions 3, 5, 6, and 9; NPM guidance documents for FYs 2008,
2009, and 2010; and program review strategies for FY 2010.
We interviewed OCFO, OW, OCSPP, regional, and other EPA personnel to
understand, document, and analyze EPA's FMFIA and NPM processes and
coordination between national program offices and regions.
We are issuing this report to bring to the Agency's attention findings that could
influence FMFIA and NPM reporting in FY 2011.
Appendix A provides additional information on our scope and methodology.
11-P-0067
-------
Chapter 2
FMFIA and NPM Processes Not Integrated
The Agency has not fully integrated FMFIA into the NPM process. FMFIA
requires federal agency managers to assess financial and programmatic
operations, establish controls, and ensure those controls are effective. NPM
guidance sets forth goals and program priorities to support compliance with
GPRA and align Agency long-term strategic goals and annual budgetary decisions
with detailed implementation instructions. Activities conducted per the NPM
process support internal controls. EPA has made efforts to improve management
integrity implementation for program operations and is currently working to
clarify links between the two processes. However, OCFO has segregated
information that offices should use in both processes and integration efforts are in
their infancy. Having national program offices primarily responsible for internal
controls over national programs would streamline reporting and lessen confusion
among staff involved in both processes.
FMFIA and NPM Processes Have Common Elements
OMB Circular A-123 provides guidance to federal managers on implementing
FMFIA to improve the accountability and effectiveness of federal programs and
operations by establishing, assessing, and reporting on internal control. The
circular states, "the requirements of FMFIA serve as an umbrella under which
other reviews, evaluations and audits should be coordinated and considered to
support management's assertion about the effectiveness of internal control." The
circular lists a number of statutory requirements and government-wide initiatives,
including GPRA, which should be considered as part of an agency's internal
control framework and should be integrated to meet the requirements of FMFIA.
Internal control is a major part of successful agency management and comprises
the plans, methods, and procedures used by an agency to meet its mission, goals,
and objectives. OMB Circular A-123 states that by incorporating internal control
into its planning and implementation activities, an agency fulfills federal
expectations for performance-based management.
EPA establishes and communicates goals and priorities through the NPM process
to support GPRA compliance. Functional statements note how national program
offices have responsibility for EPA's program integrity and performance:
• OW: The Assistant Administrator for Water serves as principal advisor to
the Administrator and provides Agency-wide policy, guidance, and
direction for the Agency's water programs. Primary responsibilities
include evaluating regional activities.
11-P-0067
-------
• OCSPP: The Assistant Administrator for Chemical Safety and Pollution
Prevention is responsible for establishing Agency strategies and
developing and operating Agency programs and policies for assessment
and control of pesticides and toxic substances. Responsibilities include
monitoring, evaluating, and assessing program operations in EPA
headquarters and regional offices.
While organizationally regional personnel report to Regional Administrators,
regional personnel are accountable to national program offices for negotiating
performance commitments and reporting performance results. Regional personnel
raise emerging issues to national program offices at periodic NPM planning
meetings. Regional personnel also provide FMFIA input to national program
offices through the lead regional coordinator process. Thus, NPMs manage national
programs and oversee regional programs through these existing mechanisms.
Our review found that NPMs already conduct many of the steps outlined in
OCFO's FY 2010 FMFIA guidance as shown in bold text and checkmarks in
table 2. Table 2 also notes internal control standards to which OCFO's guidance
steps pertain.
Table 2: FY 2010 FMFIA guidance outline and related internal control standards
Step
Internal control
standard
Task
NPM
activities
I. ESTABLISH A FOUNDATION FOR INTERNAL CONTROL REVIEWS
1
2
3
Control
environment
All GAO
standards
Control activities
Identify key programs and operations.
Develop a Program Review Strategy for each key program and
operation. Among other things, the strategy will identify potential risks
associated with the program; rank the risks; and outline the internal
controls (e.g. policies, procedures, or measures) in place to
mitigate the risks.
Prepare a Multiyear Plan for reviewing internal controls over program
operations. Based on risk levels assigned, prepare a Multiyear Plan that
establishes priorities for assessing the internal controls over
programmatic operations.
/
/
/
II. CONDUCT FY 2010 REVIEWS
1
2
3
Control activities
and monitoring
Control activities
Information and
communication
Conduct reviews, testing, or monitoring activities planned for FY
2010.
Determine corrective actions.
Document your findings.
/
III. REPORT FY 2010 FINDINGS
1
2
3
Monitoring and
information and
communication
Information and
communication
Information and
communication
Provide status updates for midyear Management Integrity Report and
the Agency "Update" meeting.
Develop FY 2010 assurance letter to the Administrator.
Prepare for end-of-year "Decision Meeting."
7
Source: OIG analysis of an outline provided by OCFO's associate staff director for accountability within the Office of Planning,
Analysis, and Accountability, on June 30, 2010.
11-P-0067
-------
Table 2 illustrates how the NPM process relates to the FMFIA process, and how
information from one process can inform the other. However, EPA has not fully
integrated FMFIA into the NPM process. OCFO's guidance documents only
recently referenced each process. OCFO has reoriented its FMFIA process to
include program operations, and OCFO, national program offices, and regional
personnel considered FY 2010 a building year in which to clarify links between
the two processes.
Program Office Interpretations Vary on Degree of Integration
While NPMs, management integrity advisors, regional personnel and planners,
and OCFO acknowledged links between the two processes, we found variations in
the extent to which regions and offices understand the relationship between the
FMFIA and NPM processes and confusion as to what to report. OW managers
have different views as to the linkages between the two processes—one saw no
link, and the other had fully integrated each process. However, most regional
program personnel continue to struggle with how FMFIA relates to the NPM
process. OCFO said this confusion derives largely from an acknowledged lack of
familiarity with FMFIA terms and framework, and OCFO is striving to improve
understanding.
Staff stated they had not considered the NPM guidance as a tool to identify
program vulnerabilities. However, OCFO believes the NPM process is the
primary control for program management. We agree and note the following
elements of the NPM process relevant to FMFIA:
• Final NPM guidance from the national program offices contains
information that could be included in the FMFIA Midyear Status Report to
the Administrator.
• Information published in national program offices' midyear reports on
commitments could be considered for input from the lead regional
coordinators to NPMs for FMFIA assurance letters.
• Managers' discussions of program priorities, vulnerabilities, and other
issues during the NPM process include issues that offices should assess for
internal control deficiencies and, if necessary, report in FMFIA assurance
letters. We noted two such examples: (1) a national water division
directors meeting in October 2009 addressed water quality monitoring,
new administration priorities, the Urban Waters Initiative, and surface
mining operations; and (2) OCSPP division directors discussed with us
significant management issues such as the Toxic Substances Control Act
and requirements for poly chlorinated biphenyls in caulk.
11-P-0067
-------
Regional Personnel Unclear on Assurance Letter Content
By not linking the FMFIA and NPM processes, regional personnel remain unclear
as to how to report certain issues in assurance letters. For example, our review of
FY 2009 regional FMFIA assurance letters found inconsistencies in how regional
personnel reported geographic initiatives. Geographic initiatives are programs or
activities unique to a particular EPA region (e.g., the Chesapeake Bay and the
Great Lakes programs are tasked with protecting and restoring large aquatic
ecosystems). OCFO said that reporting responsibilities on geographic initiatives
varies by NPM. Of the four regions' assurance letters we reviewed, we found that:
• Region 3 briefly mentioned the Chesapeake Bay.
• The Great Lakes National Program Office issued its own assurance letter
through Region 5's annual FMFIA process.
• Regions 6 and 9 did not mention initiatives within their purview: the Gulf
of Mexico, United States-Mexico Border Water Quality, or the Pacific
Islands Waters.
As we reported in 2009,2 because OCFO previously focused FMFIA primarily on
financial and administrative activities, staff were confused about FMFIA roles
and reporting. Beginning in FY 2009, OCFO expanded EPA's FMFIA reporting
from strictly financial and administrative activities to include program operations
and, in FY 2010, clarified regional and national program roles and
responsibilities. During this transition, regional offices remain confused as to
what to report on and how. Regional comptrollers noted improvements in this
year's OCFO guidance, but said that program personnel remain unclear on their
FMFIA responsibilities.
OCFO should require NPMs to summarize national program issues in their
assurance letters, including information NPMs obtain on regional program
implementation and performance. Regional FMFIA assurance letters would then
focus on administrative and financial internal control activities. OCFO has also
historically administered the FMFIA and NPM processes separately and has only
recently viewed the two as complementary. Program offices and regional
personnel have considered the NPM process a separate task distinct from FMFIA,
even though many NPM process activities support FMFIA.
Both the Administrator and Chief Financial Officer have issued statements on
how EPA should view the management integrity process as a year-long process
instead of a once-yearly exercise to complete assurance letters. In her February 2,
2010, memorandum, the Administrator stated that to improve management
integrity for FY 2010, everyone involved should view it as a year-long process—a
2 We issued two reports on the administrative focus of FMFIA guidance and the confusion regional and program
office personnel had with FMFIA requirements: EPA Should Use FMFIA to Improve Programmatic Operations,
Report No. 09-P-0203, August 6, 2009; and EPA's Office of Research and Development Could Better Use the
Federal Managers' Financial Integrity Act to Improve Operations, Report No. 09-P-0232, September 15, 2009.
11-P-0067
-------
significant departure from how EPA has traditionally carried out management
integrity activities. Lastly, the Agency has committed to adopting a "OneEPA"
approach to accomplishing its environmental protection mission.3 By making
NPMs responsible for internal controls and FMFIA reporting for national
programs (working through Lead Regional Coordinators to do so), the Agency
would support OneEPA and foster more communication between regional
personnel and NPMs.
Conclusion
EPA would increase its ability to maximize its resources, achieve its
commitments, and meet its goals by clarifying links between the NPM and
FMFIA processes. Separate activities and reporting related to the FMFIA and
NPM processes potentially result in duplicative activities under each. NPMs
already conduct many of the activities related to FMFIA in the NPM process, yet
national program offices have separate processes and staff responsible for each
process. Having NPMs primarily responsible for reporting on internal controls
over national programs would streamline reporting and lessen confusion among
staff involved in both the NPM and FMFIA processes.
Recommendations
We recommend that the Chief Financial Officer:
2-1 Assign NPMs primary responsibility for FMFIA reporting on
internal controls for national programs and rely on the lead
regional coordinator process for input from the regions.
2-2 Direct regional personnel to report on administrative and financial
internal control activities along with unique geographic and
programmatic issues in regional assurance letters.
Agency Comments and OIG Evaluation
In recommendation 2-1 of our draft report, we stated that OCFO should "Use
existing activities under the NPM guidance process to require that NPMs in National
Program Offices complete FMFIA reporting on program performance, risks, and
emerging issues (including those related to regional program performance and/or
feedback NPMs receive from regional program implementers)." In its response,
OCFO suggested that we revise the recommendation to "require NPMs to address
in their NPM Guidance, as appropriate, the vulnerabilities and weaknesses
identified through their FMFIA responsibilities." We disagreed with OCFO's
suggested revision to recommendation 2-1 because it did not incorporate using the
3 As described in EPA's Open Government Plan, the "OneEPA" tool is in place to promote transparency by
initiating discussion, capturing suggestions, and collecting reactions both within the Agency and from the public.
11-P-0067 10
-------
NPM framework along with the lead regional coordinator process. National
program offices should integrate evaluating internal controls into all program
management activities using these processes. After reviewing OCFO's response,
we clarified our report to focus on the entire NPM framework (i.e., developing
program priorities, strategies, and performance commitments linked to strategic
and budget planning), rather than specific annual NPM guidance documents. We
met with OCFO on December 15, 2010, and agreed upon the current
recommendation. We also discussed OCFO's planned corrective actions and
milestone dates, such as connecting FMFIA and the NPM processes in its
FY 2012 NPM guidance and upcoming FY 2011 FMFIA guidance. OCFO's
planned FY 2011 FMFIA guidance will reinforce the role of NPMs and the lead
regional coordinator process.
On recommendation 2-2, OCFO responded that it should continue to direct
regions to address regional aspects of key national programs to ensure Regional
Administrator-level accountability. We agree and discussed this with OCFO on
December 15, 2010, and added text to recommendation 2-2.
We believe our recommendations will provide national program offices a more
unified perspective and a means to gauge program priorities, weaknesses, and
emerging areas across all regions. This approach will also ensure regional
accountability on unique geographic and programmatic issues. OCFO will verify
its planned corrective actions to address recommendations 2-1 and 2-2 in FY 2011
program compliance reviews. We believe OCFO's planned corrective actions
address the intent of our recommendations. Appendix B includes OCFO's full
response.
11-P-0067 11
-------
Chapter 3
Risk Assessment Internal Control Standard
NPMs assess and evaluate relevant risks associated with achieving their program
objectives through communication within the national program offices and with
regional personnel and other stakeholders. However, offices have not linked these
activities to internal control requirements. FMFIA requires managers to define
program goals and identify key programs, complete a risk assessment based on
their priorities, and then establish controls to mitigate identified program risks.
National program offices and regional personnel do not appear to completely
understand the risk assessment internal control standard and how to apply it to
program operations. Additional training on risk assessment, including how to
identify weaknesses, determining how to manage risks, and how to conduct
necessary internal control reviews, should improve program management.
Risk Assessments Not Informing Program Reviews
Risk assessment—a fundamental element in internal control—identifies and
analyzes risks that might impede the achievement of organizational goals, such as
goals defined in strategic and annual performance plans developed under GPRA.
Agencies should analyze identified risks for their potential effect or impact and
implement controls to minimize or eliminate the risks to achieve the internal
control objectives of efficient and effective operations.
OCFO's FY 2010 FMFIA guidance requires all EPA programs to identify key
programs and develop program review strategies that list and rank potential risks
and related internal controls. The guidance also requires that all EPA programs
prepare schedules, or multiyear plans, describing when offices plan to review
internal controls for program operations. OCFO's FY 2009 FMFIA guidance
required all EPA programs to develop a multiyear review strategy (similar to the
FY 2010 multiyear plan) and complete a checklist based on GAO's Standards for
Internal Control in the Federal Government that includes risk assessment.
We found that not all national program offices and regions within our scope
conduct risk assessments in accordance with GAO's standards. Of the six
FY 2009 assurance letters we reviewed, only OW reported that it conducted a risk
assessment and used it to determine program reviews. Our interviews confirmed
that in the course of program management activities and coordinating with
regional personnel and other stakeholders, offices perform elements of risk
assessments. Offices we reviewed do not, however, analyze risks for potential
effects or impacts on the Agency, do not consider those assessments in the context
of internal controls, and do not incorporate activities into FMFIA. While OCFO
has taken steps to include programmatic operations in FMFIA reporting, Agency
11-P-0067 12
-------
personnel need training on internal control standards and terminology, and ways
to connect FMFIA to their program-level tasks and accomplishments. OCFO is
developing training but does not expect it to be ready until the FY 2011
management integrity reporting period.
OCFO's FY 2010 FMFIA program guidance instructs offices to prepare program
review strategies that list and rank risks and vulnerabilities. OCFO assumes that
offices conducted risk assessments prior to completing program review strategies
so that risk information could be included. Offices have begun submitting
FY 2010 program review strategies, but our analysis of eight strategies indicates
that completing a strategy itself does not meet the intent of GAO's standards. The
Agency cannot ensure that offices assess and analyze internal and external risks
simply because they submitted program review strategies, as shown in table 3.
Table 3: Program review strategy limitations in addressing risk assessment
OCFO's FY 2010 FMFIA guidance requires that program review strategies list and rank potential
risks and related internal controls.
OIG All eight OW and OCSPP strategies that were available by August 2010 identified and
comment ranked at least one risk, but did not include any risk analysis. For example, in OW's
biosolids strategy, the risk of "insufficient monitoring data" does not have attendant control
activities, monitoring, and information/communication relative to that specific risk. Instead,
the strategy includes a random collection of material (e.g., one information/communication
entry is to make its website more user friendly, but it is not clear what risk that addresses).
OW's biosolids strategy also includes a potential major risk related to the lack of exposure
and toxicity data, but no apparent control activity to address that risk.
Risk analysis generally includes estimating the risk's significance, assessing the likelihood of its
occurrence, and deciding how to manage the risk and what actions to take.
OIG OCSPP has not completed a risk assessment. Instead, OCSPP senior managers met to
comment discuss and prioritize key risks to include in their program review strategies, half of which
were administrative. OCSPP did not include backlogged chemical assessments under the
Toxic Substances Control Act even though GAOa and our officeb identified the backlog as a
major management challenge for EPA.
OW completed and included a risk assessment in its FY2009 assurance letter, and its
FY 2010 program review strategy included reviews that were already planned as a result of
its risk assessment (i.e., ongoing reviews). One regional management integrity advisor said
that regional water managers did not understand why OW selected the areas included in its
strategy.
Source: OIG analysis of a sample of submitted FY 2010 program review strategies.
a
GAO, Environmental Protection Agency Major Management Challenges, GAO 09-434, March 4, 2009.
b EPA OIG, EPA 's Fiscal Year 2010 Key Management Challenges, May 11, 2010.
These examples indicate that national program offices and regional personnel do
not fully understand FMFIA and risk assessment. This confusion stems from how
to apply internal control risk assessment to program management. Many EPA
personnel understand risk assessment as a scientific term used to assess risks to
human health and the environment. For example, one program office's review
11-P-0067 13
-------
strategy identified as a program risk that "human health and the environment may
no longer be protective"—a risk in a scientific sense. However, GAO's Standards
for Internal Control in the Federal Government defines internal control risk as
barriers that might inhibit a program from achieving its objectives. In this case,
protecting human health and the environment is the program objective, and the
office did not identify internal control risks that prevent the program from
achieving this objective. Offices also appear to not understand how to apply risk
assessment to program operations because they have not connected internal
control risk assessment and the NPM process. OCFO said it is providing more
guidance and training on risk.
Managers and staff we spoke with described a "language barrier" between what
program staff understands about FMFIA and what management integrity staff
understands about programmatic operations. While OCFO has taken steps to
include programmatic operations in FMFIA guidance, Agency personnel need
training on internal control standards and terminology, and ways to connect
FMFIA to their program-level tasks and accomplishments. OCFO believes that
offices conduct risk assessments and establish controls but do not identify them in
FMFIA terms, and OCFO said it is working to address this language barrier.
Offices could benefit from OCFO posting on its intranet tools such as the five-
page overview of risk assessment (including step-by-step instructions and
definitions) included in the contractor's report on program compliance reviews
(dated January 15, 2010) and highlighting on its intranet completed products
(such as OW's risk assessment) that other offices could use as examples. OCFO is
developing training on GAO's standards and how to incorporate FMFIA into
daily program operations, and expects it to be ready for the FY 2011 management
integrity reporting period.
In a February 5, 2010, memorandum to all EPA Assistant and Regional
Administrators, EPA's Chief Financial Officer said that without adequate and
effective internal controls integral to day-to-day activities, the Agency jeopardizes
its mission by placing at risk the resources and authority entrusted to it to protect
the nation's environment and health.
Conclusion
Without conducting risk assessments consistent with GAO's Standards for
Internal Control in the Federal Government, EPA cannot ensure it has
appropriate internal controls in place or that programs operate effectively and
efficiently. Additionally, EPA has not incorporated FMFIA into day-to-day
activities, which limits how well offices identify and address program risks.
Moreover, without adequate training, the learning curve for program staff on
FMFIA and, conversely, for management integrity staff on environmental
programs, could take time and resources from other Agency priorities.
11-P-0067 14
-------
Recommendations
We recommend that the Chief Financial Officer:
3-1 Develop a training course on FMFIA that describes:
a. what internal control standards are, including definitions
and terminology;
b. how management integrity relates to program operations;
and
c. how to conduct risk assessments.
3-2 Enhance its management integrity intranet site by providing links
to risk assessment guidance and completed products (such as risk
assessments and program review strategies) that offices could use
as best practices or examples when completing their own products.
Agency Comments and OIG Evaluation
OCFO concurred with recommendations 3-1 and 3-2 and expects to complete
activities to address each recommendation in FY 2011. For example, OCFO
sought and applied our feedback on FMFIA training (per recommendation 3-1)
for management integrity advisors in June 2010. We concur with OCFO's
planned actions to address these recommendations. Appendix B includes OCFO's
full response.
11-P-0067 15
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
Rec. Page
No. No.
Subject
Status1
Action Official
Planned
Completion
Date
POTENTIAL MONETARY
BENEFITS (in SOOOs)
Claimed Agreed To
Amount Amount
09/30/11
2-1 10 Assign NPMs primary responsibility for FMFIA 0 Chief Financial Officer
reporting on internal controls for national programs
and rely on the lead regional coordinator process
for input from the regions.
2-2 10 Direct regional personnel to report on 0 Chief Financial Officer
administrative and financial internal control
activities along with unique geographic and
programmatic issues in regional assurance letters.
3-1 15 Develop a training course on FMFIA that describes: 0 Chief Financial Officer
a. what internal control standards are,
including definitions and terminology;
b. how management integrity relates to
program operations; and
c. how to conduct risk assessments.
3-2 15 Enhance its management integrity intranet site by 0 Chief Financial Officer
providing links to risk assessment guidance and
completed products (such as risk assessments and
program review strategies) that offices could use
as best practices or examples when completing
their own products.
09/30/11
0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
11-P-0067
16
-------
Appendix A
Details on Scope and Methodology
We conducted our review to determine whether EPA links the FMFIA process with the NPM
process. We also reviewed whether NPMs and regional personnel coordinate program
management to address program risks and vulnerabilities. We chose two program offices for our
review: OW and OCSPP. OW is one of EPA's largest program offices, has the largest budget,
and interacts extensively with regional personnel. OCSPP represents a contrast to OW as it is a
smaller office with a smaller budget. Further, because OCSPP is more headquarters focused, it
interacts differently with the regional personnel. We also selected two regions for review:
Regions 5 and 9. Region 5 is EPA's largest regional office and has the largest budget. Region 9
is a midsized office with a midsized budget, and its staff played a significant role in the
development of EPA's management integrity policy. Regions 5 and 9 both include states that
have significant water and pollution concerns. We also reviewed Regions 3 and 6 because they
house, respectively, the OW and OCSPP lead regional coordinators.
To address our objectives, we did the following:
• Reviewed and summarized relevant laws, regulations, policies, and guidance on the
management integrity (FMFIA) and the NPM processes.
• Flowcharted the FMFIA and NPM process timelines to identify potential linkage
points and areas of efficiency.
• Analyzed information from the Office of Regional Operations (which oversees the
lead regional coordinator process), OCFO, OW, OCSPP, Regions 5 and 9, and
management integrity advisors for all offices in our scope
• Gathered and analyzed information from lead regional coordinators for OW and
OCSPP, located in Regions 3 and 6 respectively, to understand the lead region
process and its role in management integrity reporting.
• Conducted a literature search to review previous related audits and reports.
• Reviewed OCFO's FMFIA guidance for FYs 2008, 2009, and 2010.
• Participated in OCFO conference calls and interviewed OCFO staff to discuss the
FMFIA process and the FY 2010 requirement on program review strategies.
• Reviewed assurance letters for OW, OCSPP, and Regions 5 and 9 for FYs 2008 and
2009, and compared reporting between the headquarters program offices and regions.
We also reviewed assurance letters for Regions 3 and 6. Our assurance letter reviews
11-P-0067 17
-------
focused on program operations, as well as whether letters included evidence of
completed internal control risk assessments.
• Reviewed OW and OCSPP program review strategies, which OCFO required as part
of the FY 2010 FMFIA process.
• Reviewed OCFO's Technical Guidance on FY 2010 National Program Manager
Guidance and Annual Commitment Process in Measures Central, as well as OW and
OCSPP NPM guidance for FYs 2008, 2009, and 2010.
• Conducted interviews with NPM and regional planners, regional comptrollers, office
directors, and other program and regional staff to understand, document, and analyze
the FMFIA process, the NPM process, and coordination between national program
offices and regional personnel. These interviews included briefings with NPMs from
OW and OCSPP, who explained processes they use to develop annual NPM guidance
documents and stakeholders with whom they coordinate on performance targets.
In FY 2009, OCFO hired Industrial Economics, Inc., to assess the effectiveness of EPA's
management integrity program and to identify how EPA program and regional offices can
improve FMFIA implementation. We reviewed Industrial Economics, Inc.'s, final report, dated
January 15, 2010, as it included offices in our project scope as well as recommendations to
OCFO that were similar to those resulting from our own interviews with Agency personnel.
Prior Audit Coverage
The OIG reviewed the Agency's FMFIA implementation in two reports issued in 2009:
• In EPA Should Use FMFIA to Improve Programmatic Operations, Report
No. 09-P-0203, issued August 6, 2009, we determined whether EPA offices
integrated internal control standards under FMFIA into their programmatic
operations. We also determined whether EPA offices use available GAO guidance to
develop and monitor their internal controls. We found that because OCFO did not
require—and program and regional offices did not evaluate and report on—
compliance with GAO's standards in FY 2008, EPA risked not fully complying with
FMFIA. We also observed that the FMFIA process emphasized administrative and
financial reporting over programmatic performance. We made five recommendations
and are monitoring corrective actions OCFO has undertaken to address all
recommendations.
• In EPA 's Office of Research and Development Could Better Use the Federal
Managers' Financial Integrity Act to Improve Operations, Report No. 09-P-0232,
issued September 15, 2009, we determined whether the Office of Research and
Development had a systematic strategy to establish, review, and monitor internal
controls. We also determined what the Office of Research and Development's
internal control strategy should contain to account for risks in meeting program goals.
We found that the Office of Research and Development has several opportunities for
11-P-0067 18
-------
improving the accountability and effectiveness of federal programs and operations to
better accomplish FMFIA as intended. We made three recommendations and are
monitoring corrective actions the Office of Research and Development has
undertaken to address all recommendations.
11-P-0067 19
-------
Appendix B
Agency Response to Draft Report
November 19, 2010
MEMORANDUM
SUBJECT: Response to draft Audit Report EPA Should Further Integrate National Program
Manager Guidance with Federal Guidance on Internal Control Risks
(Project No. OA-FY09-1003)
FROM: Barbara J. Bennett //s//
Chief Financial Officer
TO: Melissa M. Heist
Assistant Inspector General for Audit
We appreciate the opportunity to respond to the draft Audit Report cited above.
Throughout this review OIG has kept OCFO involved and informed, and we believe this has
been very constructive. In particular, thank you for taking time to discuss this report with OPAA
managers on November 17.1 would like to provide several general comments on the conclusions
and recommendations presented in this draft audit report. In addition, I have asked Kathy
O'Brien to send you a copy of the report annotated with our more detailed comments on specific
statements. We appreciate your consideration of our comments and suggestions.
Overall Comments - Link Between NPM Guidance and FMFIA
As you know, OCFO has taken a number of steps over the past year to clarify and
strengthen the Agency's internal controls over programmatic activities, including highlighting
the connections between Management Integrity (FMFIA) and processes such as the annual NPM
guidance. In our technical guidance for both FMFIA and NPM Guidance, we call attention to the
importance of identifying program risks and vulnerabilities, including obtaining input through
the Lead Region process.
Our primary concern with this draft audit is the confusion regarding the purpose of NPM
Guidance and the link between FMFIA and NPM Guidance - a critical concept central to the
audit. The primary purpose of NPM Guidance is to operationalize the program priority decisions
made in developing the Agency's Strategic Plan and Annual Plan and Budget, thereby
supporting the Agency's compliance with GPRA. On the other hand, Annual Management
Integrity guidance is the primary means to facilitate communication between NPMs and regions
for identifying program risks, vulnerabilities, and controls. In this report, you urge OCFO to
more fully "integrate FMFIA into the NPM Guidance." We believe that FMFIA and the NPM
Guidance, while related, are separate, complementary processes. We do not view FMFIA as
something to be incorporated into the Guidance; on the contrary, we view the NPM Guidance
process as one control or mechanism by which the Agency implements FMFIA.
11-P-0067 20
-------
We agree with the observation that some staff and managers at the national and regional
levels still struggle with relating their day-to-day activities to complying with FMFIA. There are
a number of reasons for this, including lack of familiarity with FMFIA terminology and the
historical view of FMFIA as a financial administrative process. We have made progress and will
continue to address these gaps in understanding through on-site reviews, meetings with staff and
managers, and technical guidance to implement both Management Integrity and the NPM
process. Also, our online Management Integrity Training, to be released in FY 2011, will help
increase this understanding.
Comments on Recommendations
Recommendation 2-1. Use existing activities under the NPM guidance process to require that NPMs
in National Program Offices complete FMFIA reporting on program performance, risks, and
emerging issues (including those related to regional program performance and/or feedback
NPMs receive from regional program implementers).
Based on the November 17 discussion, we are suggesting revised language to clarify this
recommendation:
Require NPMs to address in their NPM Guidance, as appropriate, the vulnerabilities and
weaknesses identified through their FMFIA responsibilities.
We believe we have fulfilled the intent of this recommendation. OCFO's Management
Integrity Guidance currently requires that NPMs "complete FMFIA reporting on program
performance, risks, and emerging issues." Further, OCFO's guidance to NPMs for developing
their FY 2011 and FY 2012 annual program guidance instructs the NPMs to seek input from
regions, through the Lead Region process, on program risks, vulnerabilities, and actions to
mitigate program risks, and to incorporate these, as appropriate, in their annual letters of
assurance. We believe these steps make the appropriate connection between the two processes
and, therefore, address this recommendation.
Recommendation 2-2 Direct regional personnel to report on administrative and financial
internal control activities in regional assurance letters.
OCFO requires this reporting now, and also requires that regions discuss their internal
controls as implementers of national programs.
We agree that NPMs have responsibility for identifying and mitigating risks that threaten
programs at a national level. Through the Lead Region process, NPMs receive regions'
perspectives on national program risks. We continue to maintain, however, that regions have
clear roles for implementing national programs in their regions and, in certain cases,
implementation responsibilities specific to a region (e.g., a number of geographical initiatives).
OCFO believes we should continue to direct regions to address regional aspects of key national
programs in their internal control assessments, in addition to administrative and financial
controls. All relevant activities should be addressed in regions' assurance letters.
11-P-0067 21
-------
Recommendation 3-1 Develop a training course on FMFIA that describes (a) what internal
control standards are, including definitions and terminology; (b) how management integrity
relates to program operations; and (c) how to conduct risk assessments.
Recommendation 3-2 Enhance its management integrity intranet site by providing links to risk
assessment guidance and completed products (such as risk assessments and program review
strategies) that offices could use as best practices or examples when completing their own
products.
We agree in general that risk is a subject needing further clarification for Agency staff
unfamiliar with its application in FMFIA.
OCFO's online training for management integrity advisors and Agency managers
(currently in development) will address the topics enumerated in the recommendation, including
risk. These courses will be available to MIAs, managers, and all Agency employees in FY 2011.
We may determine that supplemental training in some areas, such as risk assessment, may be
warranted after an evaluation of the initial training offerings.
OCFO is in the process of reorganizing, updating, and enhancing its Management
Integrity website with links to guidance, policy, and tools on risk assessment. This work will be
completed in FY 2011. In addition, some of these aids/links are embedded in OCFO's online
training for the convenience of MIAs and managers.
In summary, I believe that OCFO is already taking actions to address the intent of these
four recommendations to strengthen the Agency's FMFIA program and to integrate the NPM
Guidance with FMFIA, and I would like to close this audit as expeditiously as possible. I
appreciate your consideration of our comments on the draft Audit Report. Please contact Debbie
Rutherford (202-564-1913), Director of OCFO's Accountability Staff, to discuss these comments
further.
cc: Patrick Gilbride, OIG
Maryann Froehlich
Joshua Baylson
Kathy O'Brien
Stefan Silzer
11-P-0067 22
-------
Appendix C
Distribution
Office of the Administrator
Chief Financial Officer
Director, Office of Regional Operations
Assistant Administrator, Office of Water
Assistant Administrator, Office of Chemical Safety and Pollution Prevention
Regional Administrator, EPA Region 5
Regional Administrator, EPA Region 9
Agency Followup Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Audit Followup Coordinator, Office of the Chief Financial Officer
Audit Followup Coordinator, Office of Water
Audit Followup Coordinator, Office of Chemical Safety and Pollution Prevention
Audit Followup Coordinator, EPA Region 5
Audit Followup Coordinator, EPA Region 9
11-P-0067 23
------- |