^tosrx

I V'-WK-y g U.S. ENVIRONMENTAL PROTECTION AGENCY

\	/ OFFICE OF INSPECTOR GENERAL

PROl^

Catalyst for Improving the Environment

Audit Report

EPA's Office of Research and
Development Could Better Use the
Federal Managers' Financial Integrity Act
to Improve Operations

Report No. 09-P-0232
September 15, 2009


-------
Report Contributors:

Patrick Gilbride
Erin Barnes-Weaver
Karen L. Hamilton
Bryan Holtrop
Alicia Mariscal
Mary Anne Strasser

Abbreviations

BOSC	Board of Scientific Counselors

EPA	U.S. Environmental Protection Agency

FMFIA	Federal Managers' Financial Integrity Act

FY	Fiscal Year

GAO	Government Accountability Office

GPRA	Government Performance and Results Act

IRIS	Integrated Risk Information System

NHEERL National Health and Environmental Effects Research Laboratory

OCFO	Office of the Chief Financial Officer

OIG	Office of Inspector General

OMB	Office of Management and Budget

ORD	Office of Research and Development

PART	Program Assessment Rating Tool

Cover photo: A photo montage of EPA Office of Research and Development National
Health and Environmental Effects Research Laboratory facilities
geographically dispersed across the United States (EPA photos).


-------
tftD STA^

U.S. Environmental Protection Agency	09-P-0232

^	Office of Inspector General	September 15,2009

kLi

%; At a Glance

Catalyst for Improving the Environment

Why We Did This Review

We conducted this audit to
determine whether the U.S.
Environmental Protection
Agency (EPA) Office of
Research and Development
(ORD) fully integrated the
Federal Managers" Financial
Integrity Act (FMFIA) into
program operations. We
asked whether ORD has a
systematic strategy to
establish, review, and monitor
internal controls, and what
ORD's strategy should contain
to account for risks in meeting
program goals.

Background

FMFIA requires federal
managers to improve the
accountability and
effectiveness of federal
programs by establishing,
assessing, correcting, and
reporting on internal control.
FMFIA also requires federal
managers to annually evaluate
their agencies' compliance
with Government
Accountability Office (GAO)
internal control standards.

For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202) 566-2391.

To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2009/
20090915-09-P-0232.pdf

EPA's Office of Research and Development Could
Better Use the Federal Managers' Financial Integrity
Act to Improve Operations

What We Found

ORD's management integrity program is inconsistent with Agency FMFIA
guidance. ORD approaches FMFIA as an administrative reporting activity rather
than an opportunity to evaluate and report on research program performance. As
a result, ORD has not:

•	Conducted a comprehensive risk assessment,

•	Included National Program Directors in the FMFIA process,

•	Developed and implemented a strategy to establish and evaluate the
effectiveness of internal controls over research programs,

•	Provided FMFIA training to managers and staff, and

•	Included relevant risk and program performance information in assurance
letters.

EPA Order 1000.24 requires all organizations to systematically review and assess
the effectiveness of internal controls consistent with GAO internal control
standards. The Order gives program managers flexibility in designing review
strategies. While ORD's largest lab, the National Health and Environmental
Effects Research Laboratory (NHEERL), informally identifies program risks,
neither ORD nor NHEERL conducts internal control risk assessments on which to
base a program review strategy. Applying FMFIA as intended would help EPA
achieve its mission and program results through improved accountability.

ORD's Administrative Efficiencies Project management integrity workgroup has
initiated actions that we believe will address our findings, such as developing a
draft multi-year review strategy. In developing its new strategy, ORD should
include programmatic elements, a training plan, pertinent results from peer
reviews, and best practices to ensure more effective FMFIA implementation.

What We Recommend

We recommend that ORD (1) conduct a risk assessment using GAO standards and
develop a comprehensive risk-based program review strategy; (2) develop
comprehensive, tiered FMFIA training for managers and staff; and (3) revise its
management integrity program to include programmatic operations. ORD agreed
with our recommendations and has initiated corrective actions that we believe
address the intent of our recommendations.


-------
^tos%

136/

VpRO^0

OFFICE OF
INSPECTOR GENERAL

September 15, 2009

MEMORANDUM

SUBJECT:	EPA's Office of Research and Development Could Better Use the

Federal Managers' Financial Integrity Act to Improve Operations
Report No. 09-P-0232

V/\	V

FROM:	Melissa M. Heist

Assistant Inspector General for Audits

TO:	Lek Kadeli

Acting Assistant Administrator
Office of Research and Development

The Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA)
conducted this report on the subject audit. This report contains findings that describe problems
we identified and corrective actions we recommend. This report represents our opinion and does
not necessarily represent the final EPA position. EPA managers will make final determinations
on matters in this report in accordance with established audit resolution procedures.

The estimated cost of this report - calculated by multiplying the project's staff days by the
applicable daily full cost billing rates in effect at the time - is $515,790.

Action Required

On September 4, 2009, your office provided comments to our report that included a corrective
action plan with milestone dates. We believe your planned corrective actions address the intent
of each of our recommendations. As such, we plan to close this assignment upon issuance of this
final report. We have no objections to the further release of this report to the public. This report
will be available at http://www.epa.gov/oig.

We appreciate the efforts of your staff in working with us during the course of our audit. If you
or your staff has any questions regarding this report, please contact me at (202) 566-0899 or
heist.melissa@epa.gov; or Patrick Gilbride, Director, Risk and Program Performance Issues, at
(303) 312-6969 or gilbride.patrick@epa.gov.

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

WASHINGTON, D.C. 20460


-------
EPA's Office of Research and Development Could Better Use the
Federal Managers' Financial Integrity Act to Improve Operations

09-P-0232

Table of C

Chapters

1	Introduction	 1

Purpose	 1

Background	 1

Noteworthy Achievements		4

Scope and Methodology		5

2	Opportunities Exist for ORD to Better Use the

FMFIA Process to Improve Programmatic Operations		6

Management Integrity Program Inconsistent with FMFIA Guidance		6

Management Integrity Strategy Should Include Program Elements		13

Conclusion	 16

Recommendations	 16

Agency Comments and OIG Evaluation		17

Status of Recommendations and Potential Monetary Benefits		18

Appendices

A Organization of EPA ORD		19

B Details on Scope and Methodology		21

C Agency Response to Draft Report		23

D Distribution		28


-------
09-P-0232

Chapter 1

Introduction

Purpose

The Office of Inspector General (OIG) reviewed implementation of the Federal
Managers' Financial Integrity Act (FMFIA) within the Office of Research and
Development (ORD), the scientific research arm of the U.S. Environmental
Protection Agency (EPA). We sought to determine whether ORD fully integrated
FMFIA into programmatic operations. We examined ORD using its largest lab,
the National Health and Environmental Effects Research Laboratory (NHEERL),
as our example. Our objectives were to determine:

•	Whether ORD has a systematic strategy to establish, review, and monitor
internal controls.

•	What ORD's internal control strategy should contain to account for risks in
meeting program goals.

Background

EPA's Office of Research and Development

ORD is EPA's lead office for the production, review, and integration of scientific
and technical knowledge into environmental protection policies and regulations.
ORD has seven laboratories and centers across the country, with ORD
headquarters in Washington, DC, and main research facilities in Ohio and North
Carolina. NHEERL is ORD's largest individual laboratory, accounting for
21 percent of ORD's Fiscal Year (FY) 2008 budget and 33 percent of its
authorized full-time staff. NHEERL has division and field office laboratories in
eight locations and ecological environments across the country.

To provide the leadership to accomplish ORD's strategic goals, ORD created an
Executive Council, consisting of senior management, to make corporate
decisions. ORD instituted a strategic multi-year planning process to guide the
direction of ORD's research to focus on EPA's highest priority science needs.
National Program Directors lead development of multi-year plans with
involvement by staff and managers. There are no direct lines of authority
between National Program Directors and lab, center, and office directors as both
positions report to the Assistant Administrator. ORD confirms the relevancy and
credibility of its science through program reviews by the Board of Scientific
Counselors (BOSC). ORD aligned BOSC reviews to meet the structure of
reviews conducted under the Office of Management and Budget (OMB) Program
Assessment Rating Tool (PART).

1


-------
09-P-0232

ORD issued a policy in November 2006 on how ORD implements FMFIA.

ORD's Assistant Administrator has responsibility for implementing FMFIA.
Additional responsibilities lie with lab and center directors and deputy directors.
An ORD Management Integrity Advisor coordinates activities such as the
assurance letter process. The Advisor works with designated Management
Integrity Coordinators within ORD's seven labs and centers. According to ORD's
policy, National Program Directors do not have a role in the management integrity
process. See Appendix A for more details on ORD's organizational structure.

Management Integrity Guidance

FMFIA requires federal managers to improve the accountability and
effectiveness of federal programs and operations by establishing, assessing,
correcting, and reporting on internal control. Federal managers must also
develop and maintain internal control to achieve: (1) effective and efficient
operations; (2) reliable financial reporting; and (3) compliance with applicable
laws and regulations per OMB Circular A-123, Management's Responsibility
for Internal Control (revised). Effective internal control is a key factor in
achieving agency missions and program results.

The Federal Government has implemented several initiatives, such as the
Government Performance and Results Act of1993 (GPRA) and PART, to
improve program management. Activities conducted as part of these initiatives
support an agency's overall internal control framework. Figure 1.1 illustrates
how FMFIA serves as an umbrella under which agencies should coordinate
internal control efforts.

Figure 1.1: FMFIA Internal Control Framework

Federal Managers' Financial Integrity Act (FMFIA)

4	4

OMB Circular A-1 23, Management's	GAO Standards for Internal Controls

Responsibility for Internal Control	in the Federal Government

4	4

U.S. Environmental Protection Agency
Delegations / Orders / Policies / Manuals / Guidance

4

AA/RA Assurance Letters

	4	

Administrator's Assurance Statement

4

EPA's Performance and Accountability Report

Source: EPA training, EPA Internal Control and Management Integrity: Make It
Second Nature, issued (via EPA's Intranet) on May 28, 2008 (slide 11 of 21).

2


-------
09-P-0232

FMFIA requires federal managers to annually evaluate their agency's compliance
with Government Accountability Office (GAO) Standards for Internal Control in
the Federal Government, shown in Table 1.1, and issue a statement indicating full
compliance or noncompliance. The standards provide the overall framework for
establishing and maintaining internal controls, and for identifying and addressing
major performance and management challenges and areas at greatest risk of fraud,
waste, abuse, and mismanagement. The standards comprise a major part of
managing an organization.

Table 1.1: GAO's Standards for Internal Control in the Federal Government

Control
Environment

This standard establishes and maintains an environment throughout
the organization that sets a positive and supporting attitude toward
internal control and conscientious management. This includes
establishing goals, objectives, and performance measures at the entity
and activity level.

Risk
Assessment

Once the goals, objectives, and measures have been defined, the risks
that could impede efficiently and effectively achieving those objectives
are identified. This includes assessing risks the agency faces from
both internal and external sources. Risk assessment includes
identifying and analyzing risks associated with achieving objectives
defined in strategic and annual performance plans developed under
GPRA, and form a basis for determining how to manage risks.
Management needs to comprehensively identify risks and should
consider all significant interactions between the entity and other parties
as well as internal factors at both the entity-wide and activity levels.

Control
Activities

These are the policies, procedures, techniques, and mechanisms that
implement management's direction to achieving goals. Internal control
activities help ensure that management's directives are carried out.

Information and
Communications

This standard includes data and information (performance and
financial) to determine whether the organization meets its goals and
objectives and maintains accountability over resources.

Monitoring

Internal control monitoring should assess the quality of performance
overtime and ensure that audits and other review findings are
promptly resolved.

Source: OIG's Summary of GAO's Standards for Internal Control in the Federal Government,
GAO/AIMD-OO-21.3.1 (November 1999).

To implement FMFIA and OMB Circular A-123, EPA issued Order 1000.24,

Management's Responsibility for Internal Control. The Order:

•	Prescribes policies, procedures, and standards for internal controls at EPA;

•	Outlines Agency senior managers' roles and responsibilities for developing,
implementing, assessing, documenting, improving, and reporting on internal
controls;

•	Incorporates specific requirements for assessing internal controls over
financial reporting; and

•	Provides tools to help managers monitor both overall program progress and
the effectiveness of day-to-day operations.

3


-------
09-P-0232

In accordance with the Order, the Office of the Chief Financial Officer (OCFO)
issues annual guidance to program and regional offices on complying with
FMFIA. This guidance includes a reporting template with specific instructions
for completing assurance letters. Assurance letters provide the results of the
internal control assessment and an overall statement to the Administrator on the
adequacy of controls for the organization. In 2008 OCFO also developed an
Intranet training to increase understanding of internal controls, titled EPA
Awareness Training for Internal Controls and Management Integrity, although
OCFO did not mandate that all EPA staff complete the course. OCFO annually
collects all program and regional office assurance letters and compiles a single
draft assurance letter for the Administrator to review and sign.

EPA's Order also requires managers to develop and implement a strategy that
defines how they use sources of program management information to provide the
basis for their annual assurance letters. The systematic review strategy should be
consistent with and integrate Agency-wide processes used to develop and report
on program performance measures and results required under GPRA. Examples
of sources of program management information include: OIG and GAO reports,
internal and external program evaluations, audits and reviews conducted under the
Chief Financial Officers Act and GPRA, PART, and other reviews. The Order
recommends that program managers use GAO's five internal control standards
when developing a review strategy as the basis for determining the need for and
design of an internal control and how well it functions.

EPA also issued a 1996 publication, Management Integrity at EPA - A Manager's
"How To " Guide for Program Reviews: Seeing the Forest and the Trees. The
guide introduced the Agency's 10 management integrity principles and noted that
managers often miss the essence of internal controls and FMFIA:

In complying with FMFIA, many Federal managers historically
never saw the "Big Picture. " Most focused on filling out
checklists and performing other routine compliance tests, rather
than considering management controls in light of broader program
issues and EPA 's overall mission. In short, they got lost in the
trees and never saw the forest!

Noteworthy Achievements

ORD re-engineered its management integrity function through its Administrative
Efficiencies Project. This effort focused on improving administrative and
financial internal controls, including how ORD might conduct a formal risk-based
assessment of those controls. This effort's management integrity workgroup is
also considering a separate "scientific" or programmatic track for assessing
internal control.

4


-------
09-P-0232

ORD engaged the National Academy of Sciences to evaluate its research program
effectiveness in a report, Evaluating Research Efficiency in the U.S.
Environmental Protection Agency (published in 2008), which significantly altered
the dialogue and approach to efficiency measurement.

ORD and NHEERL undergo many external peer reviews to maintain a high level
of credibility. The Science Advisory Board reviews the quality and relevance of
scientific and technical information used or proposed as the basis for Agency
regulations. BOSC evaluates and reviews scientific research programs, plans, and
laboratories (and related management practices) and recommends improvement
actions. Since FY 2008, NHEERL has conducted management systems reviews
in lieu of traditional divisional reviews as a cost saving effort.

Scope and Methodology

We conducted our audit from July 2008 through April 2009 in accordance with
generally accepted government auditing standards. Those standards require that
we plan and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our audit
objectives.1 We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.

We focused our evaluation on ORD's headquarters office in Washington, DC, and
NHEERL facilities located in Raleigh, North Carolina, and Corvallis and
Newport, Oregon. We reviewed and analyzed EPA and ORD management
integrity policies, procedures, and FMFIA guidance; ORD's budget and
expenditure data; and FYs 2007 and 2008 FMFIA assurance letters. We
interviewed ORD and NHEERL personnel at various levels of responsibility.
We conducted site visits to NHEERL and its Western Ecology Division, including
tours of several laboratories. We benchmarked risk assessment methods used by
others in the public sector, as well as the FMFIA process at eight other federal
agencies. We reviewed NHEERL-related internal/external peer reviews to
determine the extent to which they addressed internal controls. Appendix B
provides additional information on our scope and methodology.

1 In the course of performing our field work, we identified findings applicable outside of ORD-NHEERL. In
February and March 2009 we expanded our field work to include reviewing assurance letters and FMFIA processes
in four regions and two program offices. In August 2009, we issued a report to OCFO on the Agency's management
integrity program, summarizing examples from the regions and program offices we reviewed.

5


-------
09-P-0232

Chapter 2

Opportunities Exist for ORD to Better Use the
FMFIA Process to Improve Programmatic Operations

ORD's management integrity program is inconsistent with Agency FMFIA
guidance. Currently, ORD approaches FMFIA as an administrative reporting
activity rather than an opportunity to evaluate and report on research program
performance. As a result, ORD has not:

•	Conducted a comprehensive risk assessment,

•	Included National Program Directors in the FMFIA process,

•	Developed and implemented a strategy to establish and evaluate the
effectiveness of internal controls over research programs,

•	Provided FMFIA training to managers and staff to assess program
performance, and

•	Included relevant risk and program performance information in assurance
letters.

EPA Order 1000.24 requires all organizations to systematically review and assess
the effectiveness of internal controls consistent with GAO internal control
standards. The Order gives program managers flexibility in designing review
strategies. While NHEERL, ORD's largest lab, informally identifies program
risks, neither ORD nor NHEERL conducts internal control risk assessments on
which to base a program review strategy. Applying FMFIA as intended would
help EPA achieve its mission and program results through improved
accountability.

Management Integrity Program Inconsistent with FMFIA Guidance

ORD Has Not Conducted a Comprehensive Internal Control
Risk Assessment

ORD has not conducted a formal risk assessment for identifying and analyzing
risks for possible effects in program operations. OMB Circular A-123 states that
managers should perform risk assessments to identify significant areas within
which to place or enhance internal control. The Circular describes risk
assessment as a critical step in the process to determine the extent of controls.

While ORD has not assessed risk, NHEERL and its Western Ecology Division
have informally assessed their program risks, as shown in Table 2.1.

6


-------
09-P-0232

Table 2.1: Program Risks

Identified Program Risks

NHEERL

Western
Ecology
Division

1. Inability to quickly respond to changing priorities.

X

X

2. Imbalance of breadth and depth in research program.

X

X

3. Difficulty in building/maintaining research collaborations.

X

X

4. Inability to meet commitments in face of declining
resources.

X

X

5. Inadequate safeguards to ensure that Agency decisions are
supported by the highest quality science.

X

X

6. Unclear priorities.



X

7. Mismatch of skill mix.



X

8. Difficulty in building and maintaining partnerships with
program offices and regions.



X

Source: NHEERL and Western Ecology Division presentations to OIG in November 2008.

NHEERL conducts quality assurance, peer review, and accountability reviews
that it believes address three of the five risks identified. However, NHEERL
identified these risks based on management's judgment subsequent to initiating
these reviews and did not assess the effectiveness of internal controls.

ORD Does Not Include National Program Directors in the FMFIA
Process

ORD's process to evaluate risks and assign priority does not involve National
Program Directors. These directors lead development of ORD's multi-year plans
that tie to the strategic plan and EPA's mission, so internal control risk
assessments should focus on impediments to multi-year plans. Some of the
directors we interviewed said that individual lab research priorities did not
necessarily align with multi-year plan priorities. Further, even though directors
play a significant role in directing and ensuring that ORD achieves its mission,
ORD has not involved them in evaluating internal controls, implementing the
management integrity program, or preparing FMFIA assurance letters.

Lab and center directors told us ORD should involve National Program Directors
in the FMFIA process but were unsure how to do so given ORD's matrix structure.
This structure separates program performance aspects such as PART and GPRA
from FMFIA and provides no clear link between required annual reports. EPA
Order 1000.24 addresses program managers' responsibility for internal controls,
including GPRA performance measures. The Order also specifies that any review
strategy be consistent with Agency processes for GPRA reporting.

ORD's organizational structure sets boundaries for what National Program
Directors can do in regards to implementing research assigned to lab and center
directors. While National Program Directors develop research plans, reviews, and
budgets, they do not oversee day-to-day operations including spending and
staffing. Without additional involvement, National Program Directors cannot
evaluate a research program's internal controls. In our interviews, directors

7


-------
09-P-0232

described difficulties encountered while managing their research programs, such
as limited access to information on funding spent against the budget and staff time
charges to research programs. They also indicated they could benefit from
improved communication and coordination with labs, centers, and offices to
ensure consensus on prioritizing, implementing, and managing research programs.

ORD Has Not Developed a Program Review Strategy

ORD has not developed a strategy to systematically review and assess the
effectiveness of internal control for program operations. EPA Order 1000.24
states that program managers should develop a strategy for systematically
reviewing and assessing the effectiveness of internal controls; detecting
weaknesses and deficiencies; and providing a sound, documented basis for the
assurance letter to the Administrator. OCFO's FY 2008 management integrity
guidance requires that annual assurance letters describe the organization's review
strategy for assessing how well internal controls over program operations
(guidance, procedures, and policies) protect against fraud, waste, abuse, and
mismanagement.

ORD managers annually require labs and centers to design a review strategy that
meets program needs and conduct internal control reviews. However we did not
find any evidence that these activities took place for research programs. Our
interviews with ORD and NHEERL staff, as well as our review of ORD's and
NHEERL's FY 2008 assurance letters, confirmed this. ORD states in its FY 2008
assurance letter that "ORD conducted more than 38 management reviews of the
following areas: extramural (assistance agreements, interagency agreements,
contracts, simplified acquisitions), purchase cards, property, funds control and
flexiplace." Management reviews focused on administrative and financial
activities, not program operations.

ORD managers agreed that their FY 2008 assurance letter did not discuss a
program review strategy or describe how it reviewed principal research programs.
ORD stated it believed "Examples exist in the assurance letter of how ORD
approached the review of some of its programs, for example the approach for
addressing the Agency's (and ORD's) Biofuels Strategy." However ORD did not
base this process on a comprehensive risk assessment, did not report on internal
control effectiveness, and may not have provided a sound basis for the Assistant
Administrator to assert compliance with FMFIA.

In August 2008, ORD organized a management integrity workgroup as part of its
Administrative Efficiencies Project. ORD charged this workgroup with
developing a plan for conducting a management integrity line of business as an
ORD-wide function. ORD said the workgroup will coordinate various programs
that support management integrity into standard ORD operating principles. We
reviewed ORD's draft strategy and do not believe that it addresses programmatic
controls.

8


-------
09-P-0232

ORD Relies on Limited OCFO Guidance

ORD relies on OCFO guidance that does not focus on program operations for
reporting internal controls. Further, ORD did not change the composition of its
assurance letter between 2007 and 2008 to reflect changes in OCFO guidance.
OCFO's FY 2008 guidance:

•	Required a more rigorous review of the Agency's internal controls against
GAO's Standards for Internal Control in the Federal Government.

•	Required offices to document their approach to programmatic internal
control reviews in assurance letters.

•	Included an Internal Control Evaluation Checklist as an attachment to
provide a basis on which to evaluate internal controls and to use the
checklist to assess the effectiveness of programmatic internal controls.

OCFO believed its FY 2008 guidance improved reporting on internal control
effectiveness of program operations. However, the OCFO letter template focused
on administrative and financial reporting.2 ORD did not follow any strategy or
report additional information on internal controls beyond what OCFO specified in
its template.

ORD staff told us they found OCFO's 2008 guidance confusing in several areas.
For example, staff could not discern whether ORD should report the occurrence
or results of program reviews. Staff also said OCFO's guidance was not specific
and did not always tie in to EPA Order 1000.24.

ORD's lab and center Management Integrity Coordinators rely on FMFIA
guidance disseminated by ORD after it receives OCFO's annual guidance.
However, ORD did not disseminate all guidance it received from OCFO in
FY 2008. ORD did not disseminate the checklist until OCFO initiated its
FY 2009 assurance letter process; 83 percent of Management Integrity
Coordinators interviewed said they had not seen the checklist before this year.
We also noted that OCFO's FY 2008 guidance and ORD's assurance letter
contained the subject heading Internal Control Review Strategy while NHEERL's
assurance letter did not, because the guidance ORD provided to labs and centers
did not include that subject heading as a reporting requirement.

In addition, we found that ORD's 2006 Management Integrity Policy, a
supplement to EPA's guidance, was inconsistent with FMFIA guidance because it
did not cite GAO's Standards for Internal Control in the Federal Government.
The policy also referenced out-of-date information, such as older versions of both
OMB A-123 and EPA Order 1000.24. ORD staff indicated that its Management
Integrity Workgroup plans to revise ORD's policy to include updated guidance.

2 We issued a report to OCFO in August 2009 describing our concerns on the administrative focus of FMFIA
guidance.

9


-------
09-P-0232

ORD Managers and Staff Need Additional Training on Internal
Control Standards

ORD personnel gain knowledge of FMFIA and internal controls largely through
on-the-job-training and did not receive sufficient additional training on evaluating
internal controls. Inadequate understanding of the internal control process
resulted in ORD relegating FMFIA to a yearly administrative reporting activity.
ORD managers and staff responsible for FMFIA receive no training on GAO's
five internal control standards or how to ensure research programs meet standards.
GAO's standards provide the overall framework for establishing and maintaining
internal control, and for identifying and addressing major performance and
management challenges. None of ORD's lab and center directors could say that
their assurance letters addressed internal control standards. Three of seven
Management Integrity Coordinators said they addressed all five standards
throughout their letters while the remaining four acknowledged that their letters
did not do so. Only four of seven coordinators were aware of GAO's standards,
while only two of seven were familiar with GAO's evaluation tool/checklist.
Figure 2.1 illustrates coordinators' awareness of management integrity guidance.

Figure 2.1: Awareness by Management Integrity Coordinators of FMFIA Guidance

<
E

EPA "How To" Guide

GAO Tool/Checklist

GAO Internal Control
Standards

EPA Awareness
Training

OMB Circular A-123
EPA Order 1000.24

WL

J

D

5

0%

80%

20%	40%	60%

Percentage of MICs Aware of Guidance

Source: OIG analysis of interviews with Management Integrity Coordinators.

100%

Managers and staff interviewed said they did not consider results of program
reviews, such as GPRA performance measures, PART, and peer reviews relevant
for FMFIA purposes. However, FMFIA guidance, including OMB Circular
A-123, emphasizes the importance of integrating these reviews into the FMFIA
process. Several coordinators interviewed said their FMFIA reporting activities
focused entirely on administrative activities with no linkage between program
review information and FMFIA. All of ORD's coordinators - and several
managers - said that additional training on FMFIA would be helpful. One ORD

10


-------
09-P-0232

manager suggested that EPA develop tiered training, with one tier for senior
managers on understanding controls and FMFIA and another for management
integrity staff on the "nuts and bolts" of implementing and reporting on controls.
We believe ORD's FMFIA training should also include coverage of all key
guidance documents, such as EPA Order 1000.24.

ORD managers agree on the need to conduct internal control training at all levels
within the organization. However, ORD's Draft Multi-Year Program Review
Strategy did not elaborate on a specific training plan. ORD states that it will
periodically train all key personnel involved in the internal control process and
work with the Agency to identify appropriate training for staff. ORD should
identify areas of strength and weakness among its staff and, in turn, tailor its
training around those needs. Coordinators interviewed suggested ORD develop
training that includes:

•	Internal controls,

•	An ORD-specific template for reporting,

•	Best practices/lessons learned,

•	Risk assessment requirements, and

•	Training unique to managers on their FMFIA responsibilities and internal
controls.

Without comprehensive and up-to-date training, personnel may not be qualified to
assess performance of programmatic operations.

Assurance Letters Omitted Program Risk and Performance
Information

ORD, NHEERL, and the Office of Science Policy's FY 2008 assurance letters did
not document results of relevant program reviews conducted by organizations
external to ORD. EPA Order 1000.24 provides several examples of program
management information to incorporate into assurance letters, including
management reviews, OIG and GAO reports, program evaluations, and other
audits and reviews such as GPRA. Examples of program risk and performance
information omitted from assurance letters follow in Table 2.2.

11


-------
09-P-0232

Table 2.2: Examples of Information Omitted from Assurance Letters

ORD

•

ORD's FMFIA strategy and FY 2008 assurance letter did not address how ORD
developed and implemented performance goals and measures to comply with
GPRA and PART requirements. ORD described this internal control system in its
"Accountability Handbook for Performance Measurement" (dated August 2007) and
in Section 5.2 ("ORD Performance Measure Tracking") of ORD's Policies and
Procedures Manual.

•

ORD's letter did not discuss results of BOSC reviews on four research programs
and one center in FY 2007 and 2008.3 Also, ORD's 2008 assurance letter did not
mention completed NHEERL-relevant PART and BOSC reviews for two research
programs.4 ORD said it incorporates BOSC review results "into ORD management
decision-making and into the criteria used for budget decisions and related
documents." Several of these BOSC reviews addressed program management
issues and could serve to demonstrate ORD's compliance with two GAO internal
control standards (control activities and monitoring).

•

ORD's letter did not mention GAO's April and May 2008 testimonial reports where
GAO found that ORD's revised Integrated Risk Information System (IRIS) process
did not respond to GAO's March 2008 report recommendations and further
jeopardized IRIS database viability.5 In recommending that EPA not consider IRIS
as a management challenge (in an attachment to its 2008 letter), ORD cited its
revised IRIS process but did not elaborate on GAO's findings. ORD told us it
disagreed when GAO first identified IRIS as a management challenge. However,
ORD now agrees since GAO listed IRIS on its High Risk report.

•

ORD only included performance measures on IRIS and the Human Health Risk
Assessment program in its FY 2008 letter, excluding all other performance
measures. ORD said OCFO's FY 2008 guidance did not require reporting on
performance measures. However, the first page of the cover memo accompanying
OCFO's FY 2008 guidance stated explicitly that FMFIA requires the Administrator
to report on internal controls over programs, including performance measures.
OMB Circular A-123 also specifies that agencies consider GPRA and PART
requirements as part of their internal control structure. Consistent with this
Circular, EPA Order 1000.24 specifies this same requirement.

•

ORD's letter did not mention results of a National Academy of Sciences report
issued in February 2008, Evaluating Research Efficiency in the U.S. Environmental
Protection Agency, and NHEERL divisional peer reviews. In its FY 2008
assurance letter, NHEERL described completing the Atlantic Ecology Division peer
review and responding to the Mid-Continent Ecology Division's peer review. ORD
excluded these significant NHEERL items from the FY 2008 ORD assurance letter.
Our review of the Atlantic Ecology Division peer review report determined that it
addressed three internal control standards (risk assessment, control activities, and
monitoring).

3	BOSC reviewed the Science and Technology for Sustainability Research Program, Human Health Risk
Assessment Research Program, Particulate Matter/Ozone Research Program (mid-cycle). Endocrine Disrupting
Chemicals Research Program (mid-cycle), and the National Center for Enviromnental Research.

4	These included the Ecological and Safe Pesticide/Safe Product Research Programs.

5	IRIS provides toxic chemical assessment information to EPA's stakeholders.

12


-------
09-P-0232

NHEERL and Office of Science Policy	

•	NHEERL did not identify results of relevant BOSC reviews (reports issued in July
and August 2007) in which NHEERL's Gulf Ecology Division participated. Our
analysis found that these reviews identified issues relating to four internal control
standards and all five of NHEERL's self-identified risks.	

•	NHEERL did not discuss the contents or results of a detailed Atlantic Ecology
Division peer review in its FY 2008 letter. NHEERL disclosed that it had completed
a peer review and that the committee issued a written report "which identifies
strengths and challenges and offers recommendations for improvement."	

•	ORD's Office of Science Policy, which manages BOSC efforts, listed in its FY 2008
assurance letter final reports completed for five research programs but did not
discuss report contents or results. The office included information on review
accomplishments, but this information only described the report title, procedural
activity (e.g., meeting, conference call), and final report. Additionally, its assurance
letter did not discuss a review strategy to systematically evaluate internal controls.

Source: OIG analysis.

ORD managers said the assurance letter "must attest to the soundness of internal
controls for programs, functions, and financial activities" for labs and centers.
Completing a risk assessment and developing a review strategy would support
decisions regarding the relevance of these reviews and, as a result, determining
whether to include review results in the assurance letter. However, because they
did not conduct a formal risk assessment nor follow a systematic review strategy,
ORD, NHEERL, and the Office of Science Policy omitted from assurance letters
external review results pertinent to management integrity. We found review
results directly addressed GAO's five internal control standards. Such omissions
could impact the accuracy of information ORD reports in its assurance letters, and
may render invalid any assurance ORD makes as to the integrity of its programs.

Management Integrity Strategy Should Include Program Elements

As noted above, ORD has not developed a program review strategy to
systematically review and assess the effectiveness of internal control as required
by EPA Order 1000.24. ORD viewed FMFIA as an administrative exercise and
did not consider external program review results as relevant to its management
integrity approach. ORD has taken recent steps to develop a draft Multi-Year
Program Review Strategy - a requirement of OCFO's 2009 FMFIA guidance.
We commend ORD for developing a formal strategy and encourage ORD to
include specific details on how it plans to address strategy recommendations in
EPA Order 1000.24. In addition, per our second objective, we believe ORD's
strategy should also include information on its extensive peer review program as
well as best practices we identified from other public sources.

External Peer Program Reviews Conducted by ORD and NHEERL

ORD's Strategy should explain how it plans to use external program reviews
conducted by the Science Advisory Board, BOSC, peers, GAO, and OIG as
program management elements required by EPA Order 1000.24. ORD initiates

13


-------
09-P-0232

program reviews at several levels within its complex matrix structure. ORD's
strategy should include a schedule for reviews and describe how ORD will use
and report review results as part of its FMFIA process. Results of these reviews,
in addition to other program evaluations, should form the basis for any assertions
ORD makes in its annual assurance letter to the Administrator. ORD should also
evaluate the scope and frequency of external reviews. The potential impact of any
risk should include both quantitative and qualitative costs:

•	Quantitative costs include the cost of property, equipment, or inventory,
cash dollar loss, and damage and repair costs.

•	Qualitative costs include loss of public trust, loss of future funding,
increased legislation, violation of laws, not achieving organizational goals,
and decreased credibility. Such costs, while more difficult to assess, are
equally important.

We found that ORD identified completed peer reviews but did not discuss review
results. Figure 2.2 lists Science Advisory Board, BOSC, and divisional NHEERL
peer reviews.

Figure 2.2: Number of ORD Peer Reviews for 2007-2009

* Note: ORD suspended its divisional peer review program in 2008 and, instead,
initiated a pilot Management Systems Review (first in the Gulf Ecology Division
in 2008, and planned for the Mid-Continent Ecology Division in September 2009).

Source: Data provided by ORD during the course of field work.

We analyzed select program reviews and identified internal control aspects in
questions reviewers asked as well as review results and recommendations. While
each review had different objectives, we found that several reports addressed, to
varying degrees, GAO's standards. Some examples include:

• In its mid-cycle review of the Global Change Research Program in September
2008, BOSC asked, "How responsive has the Global Change Research Program

14


-------
09-P-0232

been to the recommendations made in the April 2006 BOSC program review
report?" This question addresses the "monitoring" internal control standard.

• In its review of the Human Health Risk Assessment Program in April 2008,
BOSC asked, "How consistent are the Long Term Goals of the Program with
achieving the Agency's strategic plan and the Human Health Risk
Assessment's MYP (Multi-Year Plan)?" This question is similar to several
items GAO included in its Internal Control Management and Evaluation Tool
under the "risk assessment" heading.

A risked-based strategy that prioritizes systematic reviews can help determine
whether there are redundancies in a program and the programs at greatest risk are
being reviewed. ORD managers acknowledged they may have redundancies in
the peer reviews they conduct, and BOSC came to the same conclusion in a
report. In addition to the burden of being over-reviewed, ORD does not know if it
focuses reviews on the highest risk areas that warrant most attention.

Best Practices ORD Could Implement

We identified several best practices on management integrity used at public
organizations that ORD could use (with modifications) in its strategy:

•	The State of Minnesota's risk management plan provides an example of
steps that any risk assessment methodology should include. This plan,
shown in Table 2.3, becomes the overall basis for developing, evaluating,
and maintaining internal control.

Table 2.3: Elements of Minnesota's Risk Assessment Methodology

1.	Identify risk.

2.	Categorize risk.

3.	Assess likelihood and effect.

4.	Prioritize risks.

5.	Develop a plan to reduce risks (response).

6.	Document dates and actions taken to reduce risks.

7.	Establish systematic reviews and track responses.

8.	Control risk - use above process, update based on results, and revise.

Source: State of Minnesota

•	The Department of Defense requires its components to: (1) determine
high risk areas and establish written plans for testing those areas, and
(2) develop a written strategy for program reviews based on those risks.
The Defense Acquisition University identifies and describes risks by
reviewing strategic and other planning documents and communicating
with stakeholders to assess: (a) deliverables and work processes,

(b) milestones and schedule dates, (c) resource needs and sources, and
(d) performance requirements.

15


-------
09-P-0232

In addition, ORD could use GAO's Internal Control Management and Evaluation
Tool, which outlines steps for identifying, assessing, and analyzing
internal/external risks and effects. One step to identify internal risk factors
includes identifying "any potential risks due to a highly decentralized program
operation" - a step relevant to ORD given its matrix organization. We believe the
tool provides a sound starting point that offices can tailor as appropriate,
particularly since EPA Order 1000.24 affords program managers flexibility in
designing review strategies.

ORD could also conduct benchmarking similar to what it did on efficiency
measures for research organizations (see Chapter 1 "Noteworthy Achievements").
ORD finds these organizations more analogous to it and could ask for
management integrity best practices these organizations apply. Also, four of eight
federal agencies we reviewed separate FMFIA into two tracks - a program track
and a financial track. ORD's Administrative Efficiencies Project workgroup has
recently considered developing a "scientific," or programmatic, track, and ORD
should thoroughly consider this approach.

Conclusion

FMFIA requires federal managers to improve the accountability and effectiveness
of federal programs and operations by establishing, assessing, correcting, and
reporting on internal controls. Internal controls are key factors in achieving
agency missions and program results and improving accountability. We
recognize efforts ORD has made. However, ORD has several opportunities for
continued improvement. Through its proposed Multi-Year Program Review
Strategy, ORD could define elements of its training program, consider all
performance measures and peer review results for FMFIA reporting, and
incorporate internal control best practices. By doing this, ORD will better
accomplish FMFIA as intended - the umbrella under which ORD should form its
internal control framework.

Recommendations

We recommend that the Assistant Administrator for Research and Development:

2-1 Conduct a risk assessment using the GAO internal control standard for
risk assessment and EPA Order 1000.24 and, based upon the results,
develop a comprehensive risk-based program review strategy.

2-2 Train managers and other management integrity staff on FMFIA and

internal controls. For senior managers, offer training designed to provide
an overall understanding on internal controls and a manager's
responsibilities under EPA Order 1000.24. For Management Integrity
Coordinators, offer training designed to describe how to implement and
report on internal controls.

16


-------
09-P-0232

2-3 Revise the Management Integrity Policy to include programmatic
operations. The policy should include a role for National Program
Directors, integrate performance measures, reference current FMFIA
guidance, and include a training plan. The program should incorporate
public sector best practices and a two-track approach to address
administrative and programmatic elements.

Agency Comments and OIG Evaluation

ORD agreed with our draft report findings and concurred with our
recommendations. ORD noted, and we agree, that the FMFIA process is not the
only opportunity to evaluate and report on research program performance, and
ORD's comments provided additional information on other activities it conducts.
ORD included in its report comments a table listing planned corrective actions
and completion dates to address our recommendations. We believe ORD's
planned corrective actions address the intent of each of our recommendations.
Appendix C includes ORD's full response.

17


-------
09-P-0232

Status of Recommendations and
Potential Monetary Benefits

RECOMMENDATIONS

POTENTIAL MONETARY
BENEFITS (In $000s)

Rec. Page
No. No.

Subject

Status1

Action Official

Planned
Completion
Date

2-1 16 Conduct a risk assessment using the GAO internal
control standard for risk assessment and EPA Order
1000.24 and, based upon the results, develop a
comprehensive risk-based program review strategy.

Assistant Administrator
for Research and
Development

September
2010

2-2 16 Train managers and other management integrity staff O Assistant Administrator Within 12

on FMFIA and internal controls. For senior managers,
offer training designed to provide an overall
understanding on internal controls and a manager's
responsibilities under EPA Order 1000.24. For
Management Integrity Coordinators, offer training
designed to describe how to implement and report on
internal controls.

2-3 17 Revise the Management Integrity Policy to include

programmatic operations. The policy should include a
role for National Program Directors, integrate
performance measures, reference current FMFIA
guidance, and include a training plan. The program
should incorporate public sector best practices and a
two-track approach to address administrative and
programmatic elements.

for Research and months of
Development	course

development

Assistant Administrator January
for Research and	2010

Development

Claimed
Amount

Agreed To
Amount

1 O = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress

18


-------
09-P-0232

Appendix A

Organization of EPA ORD

ORD has facilities geographically located across the country, as shown in Figure A.l, with its
headquarters in Washington, DC, and main research facilities in Ohio and North Carolina.

Figure A.1: Location of ORD Labs, Centers, and Offices

Corvallis, OR

Newport, or K	Duiuth, mn Cincinnati, OH

Las Vegas, NV

Ada, OK

Source: ORD presentation to OIG in October 2008.

Narragansett, Rl
Edison, NJ

Washington, DC

Research Triangle
Park, NC

Athens, GA

Figure A.2 depicts ORD's matrix structure.

Figure A.2: ORD's Organizational Structure

Chief of Staff

Assistant Administrator
for Research and Development

Deputy Assistant Administrator for Management
Deputy Assistant Administrator for Science

Office of
Resources
Management

and
Administration

Office of
Science
Policy

Office of
Science
Information
Management

X

National
Exposure
Research
Laboratory

National



National

Health and



Center for

Environmental



Environmental

Effects



Assessment

Research





Laboratory





I

National
Program
Directors

EPA Science Advisor

Office of the Science Advisor

National Risk
Management
Research
Laboratory

National
Center for
Environmental
Research

National

National

Homeland

Center for

Security

Computational

Research

Toxicology

Center



X

Office of
Administrative
and Research
Support

Source: ORD (organizational chart as of April 2009).

19


-------
09-P-0232

Descriptions of duties corresponding to ORD organizational components are in Table A.l.

Table A.1: ORD Organizational Responsibilities

Assistant
Administrator
for Research
and

Development

•	Signs ORD's annual FMFIA assurance letter.

•	Provides oversight and accountability for ORD's management integrity program
and internal controls over program operations and financial reporting.

•	Implements the internal control framework and fosters an organizational
environment that supports continuous awareness of internal controls at all levels.

National
Program
Directors

•	Responsible for Multi-Year Plans that establish priorities and goals.

•	Serve as primary contacts in PART reviews/GPRA measurement.

•	Coordinate with BOSC regarding Multi-Year Plan program peer reviews.

Lab, Center, and
Office Directors

•	Responsible for managing resources allocated to labs, centers, and offices to
implement research in support of Multi-Year Plans.

•	Sign annual FMFIA assurance letters for their labs, centers, and offices.

Office of
Resources
Management
and

Administration

• Conduit between ORD's Assistant Administrator, OCFO, and ORD labs, centers,
and offices for the assurance letter process, including developing and providing
management integrity guidance for the organization and consolidating annual
assurance letters for labs, centers, and offices into ORD's annual letter.

Source: ORD presentation to OIG, and OIG's February 2009 interviews with National Program Directors.

ORD has developed numerous Multi-Year Plans to administer key research programs and outline
annual performance goals and associated measures. Multi-Year Plans provide an overview of
the direction of ORD's research, present significant research accomplishments, and communicate
ORD's research program to stakeholders. Key research programs include:

•	Clean Air

•	Human Health

•	Human Health Risk Assessment

•	Drinking Water

•	Pesticides and Toxic Substances
(Safe Pesticides/Safe Products)

Ecological Research
Water Quality
Global Change
Land

Several years ago, ORD began to focus on the importance of independently confirming that it
conducts the right science and does it well. Concurrently, OMB indicated, in conjunction with
PART, the importance of independent expert reviews of federal research programs. ORD
instituted BOSC reviews of its programs and aligned them to meet the structure of PART. In
2006, to improve its external review process and better ensure the relevancy and credibility of its
research programs and science, ORD developed three specific charge questions for use in
BOSC's summary assessment of each research program's long-term goals:

1.	Relevance: How appropriate is the research used to achieve each long-term goal?
Is the program still asking the right questions, or have they been superseded by
advancements in the field?

2.	Quality: How good is the technical quality of the program's research products?

3.	Performance: How much are the program results being used by environmental
decision-makers to inform decisions and achieve results?

20


-------
09-P-0232

Appendix B

Details on Scope and Methodology

We conducted our audit to determine how ORD implements FMFIA. During our audit, we
identified concerns with ORD's implementation of internal control standards prescribed by the
Comptroller General as required by Section 2 of FMFIA. Our findings only address ORD's
implementation of Section 2 of FMFIA (internal control over programs), and not Section 4
(financial accounting systems) or Appendix A of OMB's Circular A-123 (internal control over
financial reporting). Our audit focused on ORD's headquarters office, in Washington, DC, and
its NHEERL facilities in Raleigh, North Carolina, and Corvallis and Newport, Oregon.

NHEERL is ORD's largest laboratory in terms of its budget and number of personnel employed.

To address our first objective on whether ORD had a systematic strategy to establish, review,
and monitor internal controls, we did the following:

•	Gathered and analyzed FMFIA regulations, policies, and guidance related to GAO's
Standards for Internal Control in the Federal Government, and OMB Circular A-123.

•	Gathered and analyzed EPA and ORD policies, procedures, guidance documents, and
budget data related to FMFIA implementation, including EPA Order 1000.24.

•	Attended briefings by ORD managers regarding ORD's organization, resource
utilization, annual planning, approach to FMFIA implementation, systematic strategy for
reviewing internal controls, near- and long-term laboratory studies, and the review
process used by BO SC.

•	Conducted site visits to five NHEERL facilities (three collocated in Raleigh, North
Carolina, and one each in Corvallis and Newport, Oregon) and attended briefings on
organization, resource utilization, annual planning, and FMFIA implementation. (Site
visits in Raleigh also included tours of several other ORD laboratories.)

•	Reviewed ORD's, NHEERL's, and ORD's Office of Science Policy FMFIA assurance
letters to determine whether they addressed all five GAO standards as specified in
OCFO's FY 2008 guidance. We also reviewed letters to determine whether ORD and
NHEERL documented and used program review results to establish and assess the
effectiveness of internal controls.

•	Participated in OCFO conference calls and interviewed OCFO staff to understand the
FMFIA process, particularly concerns regarding programmatic review elements.

•	Interviewed ORD's seven Management Integrity Coordinators and their supervisors, and
ORD's eight National Program Directors about roles and responsibilities in implementing
ORD's FMFIA process, focusing on FMFIA time and training requirements and needs.

•	Developed summary working papers on each set of interviews to obtain quantitative data.

•	Identified and analyzed program reviews of ORD research programs for FY 2007 and
2008 to determine the extent review questions, results, and recommendations addressed
the five GAO standards.

•	Conducted interviews with ORD and NHEERL staff and managers on reasons for including
and excluding certain information from the assurance letter development process.

21


-------
09-P-0232

To address our second objective on what ORD's internal control strategy should contain to
account for risks in meeting program goals, we did the following:

•	Flowcharted ORD's calendar of external reviews to determine the number of reviews
conducted annually.

•	Benchmarked FMFIA assurance letters and policies used by other federal agencies to
determine best practices ORD could use in its own systematic strategy.

•	Conducted follow-up interviews with OCFO staff on their understanding of the internal
control review strategy as required by EPA Order 1000.24.

•	Reviewed internal control review strategies from other EPA program offices.

•	Benchmarked other sample risk assessment methodologies available on-line and
reviewed how others established controls based upon the Council of Sponsoring
Organizations requirements. We also contacted GAO for sample methodologies.

•	Obtained and reviewed ORD's draft strategy to determine any improvement areas and/or
whether it affected our strategy recommendations.

•	Determined the effect of not developing a review strategy by documenting the
relationship between EPA Order 1000.24 and OCFO assurance letter guidance,
determining how assurance letters could have referenced prior internal/external reviews
to demonstrate compliance with internal control standards, and reviewing
internal/external reviews and how ORD and NHEERL might redirect review resources.

We did not find any prior audits or evaluations of ORD's implementation of FMFIA.

22


-------
09-P-0232

Appendix C

Agency Response to Draft Report

September 4, 2009

MEMORANDUM

SUBJECT: ORD Response to OIG Draft Report EPA's Office of Research and Development
Could Better Use the Federal Managers' Financial Integrity Act to Improve
Operations Project No. OA-FY08-0323

FROM: Lek G. Kadeli/s/

Acting Assistant Administrator (8101R)

TO:	Patrick Gilbride

Director, Risk and Program Performance Audits (801G)

This memorandum responds to the Office of Inspector General (OIG) draft audit report,
EPA 's Office of Research and Development Could Better Use the Federal Managers' Financial
Integrity Act (FMFIA) to Improve Operations (Project No. OA-FY08-0323), dated August 6,
2009. The recommendations provided in the report will help the Office of Research and
Development (ORD) continue to improve its FMFIA process.

As the scientific research and assessment arm of EPA, ORD maintains a strong
management integrity program that systematically reviews and assesses the effectiveness of
internal controls consistent with GAO Standards and OMB Circular A-123. As required by the
Federal Managers' Financial Integrity Act (FMFIA), we annually evaluate our internal controls
over programs and administrative systems and provide assurance on the integrity of our controls.
ORD is committed to ensuring that our science is of the highest quality, our programs are
managed effectively and efficiently, and that we aggressively prevent fraud, waste, and abuse.

In contrast to the report's conclusion, the FMFIA process is not the only "opportunity to
evaluate and report on research program performance." As you correctly noted in the report,
ORD has "focused on the importance of independently confirming that it conducts the right
science and does it well." ORD instituted a strategic multi-year planning process to guide the
direction of ORD's research to focus on EPA's highest priority needs for science and promote
coordination of research across laboratories, centers and offices to achieve its goals. ORD has
engaged other agencies and scientific experts in an effort to determine the most effective
approach(es) to evaluate and measure the efficiency of its research programs through reviews by
Board of Scientific Counselors, Science Advisory Board, and the National Academy of Sciences;
quality assurance programs, which include peer reviews and self inspections; and Government
Accountability Office and OIG audits. Thus far in FY 2009, more than 70 reviews of ORD

23


-------
09-P-0232

programs, functions and operations have been completed. Based on the results of these reviews,
we are continually improving the science and research we provide to the Agency.

The OIG provides three recommendations to strengthen ORD's FMFIA process. In
general we agree with the recommendations and I am pleased to say that ORD has been actively
working on revisions to its FMFIA process. ORD will continue to include information in its
assurance letter that it deems to be of significant importance to the Administrator. ORD remains
committed to management integrity and maintaining effective internal controls throughout our
organization.

Attached please find: (1) our response to each of the three recommendations contained in
the draft report and 2) a summary table of ORD's corrective actions and associated projected
completion dates. If you have any questions, please contact me or Deborah Heckman at (202)
564-7274.

Attachment

cc: Donna Vizian
Hal Zenick
Amy Battaglia
Jim Morant
Deborah Heckman

24


-------
09-P-0232

ORD Response to OIG Recommendations Contained in Draft Report

"EPA's Office of Research and Development Could Better Use the
Federal Managers' Financial Integrity Act to Improve Operations"

Project No. OA-FY08-0323
August 6, 2009

Recommendation 2-1 - Conduct a risk assessment using the GAO internal control standard for
risk assessment and EPA Order 1000.24 and, based upon the results, develop a comprehensive
risk-based program review strategy.

Response: ORD generally agrees with this recommendation.

Recognizing the complexity of conducting a comprehensive risk assessment6 for a research
organization, ORD is developing an ORD-wide approach to the risk assessment. By December
2009, ORD senior leaders will be designated to serve on ORD Executive Assessment Team
(ORDEAT) to: ensure consistency in ORD's corporate approach to internal controls; review
internal control information in order to make corporate decisions; concur on the ORD three-year
program and management review schedule; and make recommendations to the DAA for
Management and AA regarding the ORD high risk areas. By August 2010, ORD will review its
processes, test key internal controls related to ORD activities, and assess programmatic and
administrative risks. By September 2010, after completing the risk assessment activities, ORD
will revise its multi-year program review strategy as necessary.

Recommendation 2-2 - Train managers and other management integrity staff on FMFIA and
internal controls. For senior managers, offer training designed to provide an overall
understanding on internal controls and a manager's responsibilities under EPA Order 1000.24.
For Management Integrity Coordinators, offer training designed to describe how to implement
and report on internal controls.

Response: ORD generally agrees with this recommendation.

ORD is committed to training managers and employees involved with administering ORD's
management integrity program. However, OCFO agreed to "complete development of an
Agency-wide strategy for comprehensive, tiered FMFIA training by the end offiscal year 2009 "
in its July 16, 2009 response to the OIG draft audit report titled EPA Should Use FMFIA to
Improve Programmatic Operations (Project No. 08-FY08-0323). In order not to duplicate
OCFO's efforts, ORD will collaborate with OCFO on developing and implementing an Agency-
wide training program which ensures compliance with FMFIA and proper reporting of internal
controls. ORD will assess the applicability of the newly developed training for senior ORD
managers and, if necessary, initiate additional course development. ORD will then ensure that
its managers and integrity staff are trained within 12 months of completion of the course
development.

6 As defined by GAO Standards for Internal Control in The Federal Government

25


-------
09-P-0232

Recommendation 2-3 - Revise the Management Integrity Policy to include programmatic
operations. The policy should include a role for National Program Directors, integrate
performance measures, reference current FMFIA guidance, and include a training plan. The
program should incorporate public sector best practices and a two-track approach to address
administrative and programmatic elements.

Response: ORD generally agrees with this recommendation.

By January 2010, ORD will revise the ORD Management Integrity Policy to include
programmatic operations, appropriate integration of performance measures and outcomes and
reference current FMFIA guidance. As recommended, ORD will devise a two-track approach to
address administrative and programmatic elements as required by GAO and Agency guidance.
The new policy will define the roles of management and will include National Program Directors
responsibilities or other matrix managers we may have in the future under ORD's programmatic
operations. The Management Integrity Policy will reference Agency training requirements for
all managers and ORD integrity staff.

26


-------
09-P-0232

ORD Corrective Actions and Projected Completion Dates

Rec.

No.

OIG Recommendation

Lead
Responsibility

ORD Corrective Action

Planned
Completion
Date

2-1

Conduct a risk assessment using the
GAO internal control standard for
risk assessment and EPA Order
1000.24 and, based upon the results,
develop a comprehensive risk-based
program review strategy

Assistant
Administrator
for Research

and
Development

ORD is currently finalizing a strategy that
examines and reports on internal controls
covering programmatic and administrative
operations and financial activities. Once
finalized, ORD's multi-year program
review strategy will help ORD identify
high-risk areas, detect weaknesses and
deficiencies, and identify best practices in
our internal controls.

September
2010

2-2

Train managers and other
management integrity staff on
FMFIA and internal controls. For
senior managers, offer training
designed to provide an overall
understanding on internal controls
and a manager's responsibilities
under EPA Order 1000.24. For
Management Integrity
Coordinators, offer training
designed to describe how to
implement and report on internal
controls.

Assistant
Administrator
for Research

and
Development

Collaborate with OCFO

Within 12
months of
Course
Development

2-3

Revise the Management Integrity
Policy to include programmatic
operations. The policy should
include a role for National Program
Directors, integrate performance
measures, reference current FMFIA
guidance, and include a training
plan. The program should
incorporate public sector best
practices and a two-track approach
to address administrative and
programmatic elements

Assistant
Administrator
for Research

and
Development

ORD will revise the ORD Management
Integrity Policy to include programmatic
operations, a definition of the National
Program Directors' role in the process and
integration of performance measures and
outcomes.

January 2010

27


-------
09-P-0232
Appendix D

Distribution

Office of the Administrator

Acting Assistant Administrator, Office of Research and Development
Agency Follow-up Official (CFO)

Agency Follow-up Coordinator
General Counsel

Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs

Audit Follow-up Coordinator, Office of Research and Development
Audit Follow-up Coordinator, Office of the Chief Financial Officer
Acting Inspector General

28


-------