Djo5
c,9-
UNITED STATES ENVIR
AGENCY
M Street, SW
Washington, DC 20460
SFP 28
OFFICE OF
THE INSPECTOR GENERAL
MEMORANDUM
LT)
CFJ
SUBJECT:
FROM:
TO:
Report of Audit - EPA's Integrated Financial Management
System (IFMS)
Audit Report No. ElNMF3,r 15-0073-4100561
Kenneth A. Konz/V+*^% &y/
Assistant Inspector General $br Audit
Jonathan Z. Cannon
Assistant Administrator for Administration
and Resources Management
Attached is our final report entitled "EPA's Integrated
Financial Management System {IFMS}." The primary objectives of
the audit were' to: assess current progress of IFMS implementation
and interfaces of IFMS; determine whether generally accepted
system development practices were employed throughout IFMS;
identify any systems duplicating IFMS functions; verify the
implementation of controls and recommendations addressing the
IFMS problems identified in prior Financial Managers Financial
Integrity Act reports and the 1991 Office of Inspector (OIG) IFMS
audit report; and assess user satisfaction.
This audit report describes problems and recommended
corrective actions the OIG has identified. This report
represents the opinion of the OIG. Final determinations on the
matters in the report will be made by EPA managers in accordance
with established EPA audit resolution .procedures. Accordingly,
the findings described in this report do not necessarily
represent the final EPA .position.
In accordance with EPA order 2750, you, as the action
official are required to provide this office a written response
to the audit report within 90 days of the final report date. For
corrective actions planned but not.completed by your response
date, reference to specific milestone dates will assist this
office in deciding whether to close this report. In addition,
please track all action plans and milestone dates in the
Management Tracking System.- Information Resources Center
US EPA (3*04)
401 M Street, SW
Washington, DC 2
R«cycl«d/R»cyclabls
PrtrnsOwtth Soy/Cand* Ink on paper ihai
eonUina 4) Mwt 50% r«cyd*d fiber
-------
-------
We appreciate the positive response by the Assistant
Administrator, for Administration and Resources Management to our
recommendations presented in the report and the many actions he
and his staff have initiated to fully implement and improve IFMS.
We have no objection to the further release of this report
to the public. Should you or your staff have any questions about
this report, please contact Craig Silverthome, Director, ADP .
Audits and Assistance Staff on (202) 260-7603.
Attachment
-------
-------
EPA'8 IFMS
EXECUTIVESUMMARY
PURPOSE
In response to the Federal Government's increased commitment to
improve the quality and effectiveness of financial management
system, Environmental Protection Agency (EPA) has taken action to
develop and implement a -modern, efficient, and integrated financial
management system. Consequently, the Office of Inspector General
(DIG) performed an audit of the development and implementation of
EPA's IFMS.
BACKGROUND
Since the mid 1980's, EPA has been involved in the development of a
major system, IFMS, in order to integrate EPA's financial systems
which track and control over $6.9 billion annually. In 1987, the
Agency planned for $7.7 million for software and related services to
implement IFMS.
RESULTS IN BRIEF
EPA has taken a number of significant steps to implement an
integrated financial management system and has made progress in
overcoming previous management problems. However, significant
additional efforts are needed to ensure that the system will fully
meet the Agency's financial management needs. Without such efforts,
the planned completion of IFMS implementation scheduled in fiscal
1995, is still questionable (original implementation was scheduled
for 1989). Until full implementation is attained, the return on the
planned IFMS investment over the remaining system life will be
limited.
The IFMS problems were primarily due to deficiencies related to top
management leadership and direction, organizational structure,
system development methodology, cost tracking, and policies and
guidance. However, during the course of the audit, EPA initiated
corrective actions in many of these areas. In addition, over the
past year EPA top management involvement with and oversight over
IFMS development has increased substantially.
The IFMS deficiencies identified above meet EPA's materiality
criteria for reporting to the President and Congress in conjunction
with Office of Management and Budget (OMB) Circular A-123 and the
Federal. Managers' Financial Integrity Act (FMFIA). As a result, we
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
also reviewed the FMFIA evaluation process for the Office of
Administration and Resources Management to determine why these
weaknesses were not identified internally (see Chapter 5).
At the exit conference on September 21, 1994, Agency officials
generally agreed with the overall findings and recommendations in
this report. In addition, the Comptroller stated that this audit
contributed to expediting and improving the IFMS development and
implementation process.
PRINCIPAL FINDINGS
Costs And Financial Management Risks Are Continuing To Escalate Bv
Not Fully Implementing IFMS
EPA has taken a number of significant steps to implement an
integrated financial management system and to overcome previous
management problems, which include such actions as: (1) implementa-
tion of a single general ledger; (2) appointment of a Chief
Financial Officer (CFO), Deputy Chief Financial Officer (DCFO), and
Director, IFMS Project Management Staff; (3) completion of the IFMS
Strategy and Master Work Plan; (4) implementation of the newest off-
the-shelf software version which upgraded IFMS capabilities,
training and documentation; and (5) updating the IFMS cost study,
requirements analysis, and Charter.
Despite these accomplishments, the Agency still needs to take
significant actions to fully implement an integrated, comprehensive
financial management system critical to the financial management of
EPA. Target dates for completing IFMS implementation have slipped
to, at least, fiscal 1995 (originally implementation was planned for
1989). Until full implementation is attained, the return on the
planned IFMS investment over the remaining system life will be
limited. Specifically, EPA needs to implement critical modules;
eliminate their dependency on the legacy financial systems1, which,
in part, duplicate IFMS functions and capabilities; and fully
interface IFMS to financial subsystems and other administrative
systems.
Historically, EPA has incurred cost overruns and other costs as a
result of ineffective IFMS implementation. For instance,
development and implementation costs have escalated from $7.7
million to $17.2 million. These costs include, at least, $148,100
1The legacy systems include the Financial Management System (FMS); Resources Management Information
System (AMIS); and Automated Document Control Register (ADCR).
11
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
EPA spent or will .spend, on software and maintenance costs for
modules that the Agency has not used. Also, EPA will incur
additional costs to operate and maintain the legacy systems with an
estimated total cost of $11.2 million from 1990 to 1995a--less
unknown offsetting processing costs (no Agency estimate available)
which would be incurred until IFMS is fully implemented.3
Moreover, as full implementation of IFMS is attained, financial
management functions critical to EPA can be significantly improved.
For example, implementation of the'project and cost allocation
accounting module and the accounts receivable module will increase
Agency managers' abilities to effectively manage the financial and •
budgetary aspects of their programs or, in the case of Superfund, to-
assemble cost recovery packages. In addition, full implementation
can significantly reduce or eliminate the use of obsolete systems •
which are susceptible to failure, potentially making critical
financial management data unaccessible for an indefinite period of
time. For example, two of the legacy systems {FMS and RMIS), which
are not scheduled to be eliminated until fiscal 1995, were developed
over 20 years ago, and written in outdated languages which are no
longer supported by industry.
The IFMS problems discussed in the report were primarily due to:
• insufficient top management leadership and direction in
relation to IFMS development and implementation;
• fragmented managerial authority and unclear lines of authority;
• . IFMS decisions made without a comprehensive plan based on a
system life cycle approach;
• not adequately following a generally accepted System
Development Life Cycle (SDLC) approach;.
• the over-customization of the off-the-shelf software;
• the need for a comprehensive process/system to accumulate
costs; and
The legacy systems were originally planned to be eliminated in September 1969. The current estimate
for elimination ie September 1995. Therefore, we presented the operations and maintenance costs for those
systems to show other costs not originally planned for in the initial estimate.
He recognize some of these costs would be incurred when processing is shifted from the legacy systems
to IFMS. However, the Agency was unable to provide us with cost information to determine those offsetting
processing costs.
iii
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
• unclear Federal and EPA policies and guidance on life cycle
costing.
However, during the course of the audit, EPA initiated corrective
actions in many of these areas. In addition, over the past year,
EPA top management involvement with and oversight over IFMS
development have increased substantially. In our opinion, top
management and steering committees need to continue to take the
strong, decisive, and sustained actions that are required to follow
through on IFMS implementation. ' .
Life Cycle Costs NotUsed To Manage IFMS
Recently, EPA revisited efforts to create a working capital fund
(WCF), so that it could more cost effectively administer services,
including Automated Data Processing (ADP) and telecommunications
services. The Agency is also investigating a project and cost
allocation accounting module modified by another agency; this will
be coordinated with the WCF initiative. However, EPA did not
develop, review, and update costs throughout the IFMS system life
cycle* to effectively control the development and operation of the
system. As a result, EPA management was not in a position to make
informed system and budget decisions regarding the design,
development, operation, and maintenance of the system with an
estimated life cycle cost of approximately $202 million over 12
years. EPA deficiencies in tracking life cycle costs for IFMS are
primarily due to: (!) a need for a comprehensive process/system to
accumulate costs; and (2) unclear Federal and EPA policies and
guidance on life cycle costing.
Results Of IFMS Customer Satisfaction Survey
Overall,.IFMS user satisfaction has improved since 1991. The
Customer Satisfaction Survey statistically projected averages for
IFMS were all in or above the acceptable/satisfactory range.
Questions on IFMS availability and usefulness received the two
highest response averages of 7.8 and 7.6 out of a 10 maximum
positive rating, respectively. However, the survey indicated a need
for improvement in training and documentation. This occurred
because of the inability of the IFMS managers and coordinators to
communicate with all of the identified users about.available
training and documentation. Consequently, some users were not aware
Our analysis concentrated on gathering life cycle coat figures for IFMS over an estimated system
life. However, the figures also include estimated coats of the other core financial systems--Management and
Accounting Reporting System (MARS); FMS; ADCR; RKIS; Combined Payroll and Redistribution and Reporting System
(CPJWS)--incurred within the IFMS estimated system life.
iv
Report No. E1NMF3-15-0073-4100561J
-------
EPA'8 IFMS
of available training and documentation that would enable them to
use the system efficiently and effectively.
RECOMMENDATIONS
We are recommending that the Assistant Administrator (AA), Office of
Administration and Resources Management (OARM) as the Designated
Senior Official (DSO) for Information Resource Management (IRM) and
the CFO: (1) provide continued top management involvement.and
leadership in IFMS development and implementation; (2) continue to
reassess the plans, goals, alternative solutions, cost and benefits
of the remaining IFMS phased implementation; (3) limit customization
to the off-the-shelf software modules; (4) revise the IFMS Charter
and IFMS Strategy and Master Work Plan to reflect the IFMS system of
accountability and establish priorities and realistic target dates;
and (5) establish an accurate and effective methodology and process
to accumulate, track, and monitor all IFMS life cycle costs.
AGENCY COMMENTS AND OIG EVALUATION
In a memorandum dated June 15, 1994, the Deputy Assistant
Administrator (DAA) for Finance and Acquisition responded for the
Agency to our draft report. Her response (see Appendix I) consists
of three parts: a transmittal memorandum summarizing the Agency
response and concerns; Attachment 1 - Response to Recommendations
and Proposed Corrective Actions; and Attachment 2 - Specific
Comments on the Draft Audit Report. To provide a balanced
understanding of the issues, we have summarized the Agency's
position in appropriate locations throughout our report. We are
addressing the transmittal memorandum below and the recommendations
at the end of each chapter. We have also commented on the detailed
Agency response in Appendix II.
In summary, the Agency agreed with 15 of the 17 recommendations in
our draft report, partially agreed with 1 recommendation, and
disagreed with 1 recommendation. In addition, the Agency has
completed action on four of the recommendations and initiated action
on seven recommendations. However, in three of the recommendations,
where the proposed actions did not fully meet the intent of our
recommendations, we addressed our concerns in the individual
chapters. In addition, we eliminated the one recommendation that
the Agency disagreed with.
The Agency response expressed concern that our report did not fully
reflect all additional actions taken to improve IFMS since the end
of our fieldwork in December 1993. We updated our report to include
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
all relevant actions taken by the Agency in areas addressed in our
report; The response also indicated disagreement with several of
the opinions expressed in* our draft report. For example:
• The response stated that as evidence of IFMS data integrity
problems our draft report cited other OIG audits that had-
identified accounting or reporting findings and that the Agency
was already responding to those reports with appropriate
action. As a result, we deleted references to those prior
reports as evidence of data integrity problems.
• The response also noted that our draft report left the
impression that IFMS was "dis-integrated." As a result, we
clarified our remarks to present the instances where IFMS is
electronically connected to other systems and where it is not.
*!
• Further, the response opposed the intimation that expenditures
(i.e., hundreds of millions of dollars) discussed in the report
were somehow "wasted" or "unnecessary." In our report where we
discussed the total life cycle costs, we adjusted the wording
to clarify that our point is not that the money is being
wasted, but rather that EPA management was -not in a position to
make informed system decisions.
We addressed all of the Agency concerns in Appendix II of this
report to either clarify our opinions or to change our position when
we agreed with the Agency comments. At the exit conference on
September 21, 1994, Agency officials generally agreed with the
overall findings and recommendations in this report. In addition,
the Comptroller stated that this audit contributed to expediting and
improving the IFMS development and implementation process.
VI
Report No. E1NMF3-15-0073-4100561
-------
EPA.'a ZFMS
TABLE OP CONTENTS
Page
EXECUTIVE SUMMARY i
CHAPTERS
1 INTRODUCTION 1
PURPOSE "... 1
BACKGROUND 1
SCOPE AND METHODOLOGY , 3
PRIOR AUDIT REPORT COVERAGE . ' 6
2 COST AND FINANCIAL MANAGEMENT RISKS ARE CONTINUING
TO ESCALATE BY NOT FULLY IMPLEMENTING IFMS 7
FINANCIAL INFORMATION SYSTEM CRITERIA WELL
ESTABLISHED 9
IFMS NOT FULLY IMPLEMENTED 11
ADVERSE EFFECTS OF NOT FULLY IMPLEMENTING IFMS 1.8
LEADERSHIP, ACCOUNTABILITY, PLANNING, AND POLICIES
NEED TO BE IMPROVED 24
RECOMMENDATIONS , 33
AGENCY COMMENTS AND O.IG EVALUATION 34
3 LIFE CYCLE COSTS NOT USED TO MANAGE IFMS • . . . 36
SYSTEM LIFE CYCLE.COST CRITERIA 36
LIFE CYCLE COSTS NOT -USED THROUGHOUT IFMS 38
IMPACT OF NOT USING LIFE CYCLE COSTING . . . 39
COST ACCUMULATION PROCESS AND POLICY FOR APPLICATION
.SYSTEMS NEED TO BE IMPLEMENTED . . 41
RECOMMENDATIONS 42
AGENCY COMMENTS AND OIG EVALUATION 43
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 ZFHS
4 RESULTS OF IFMS CUSTOMER SATISFACTION SURVEY ... 44
USER SATISFACTION SURVEY CRITERIA .... 44
IFMS TRAINING AND DOCUMENTATION NEEDS IMPROVEMENT 45
BETTER COMMUNICATIONS NEEDED 46
RECOMMENDATIONS 48
AGENCY COMMENTS AND OIG EVALUATION . . 48
5 OARM'S FMFIA PROCESS DOES NOT SUFFICIENTLY ADDRESS
THE RISKS ASSOCIATED WITH CRITICAL IFMS PROCESSES . . . .'. 49
RECOMMENDATIONS ". 50
AGENCY COMMENTS AND OIG EVALUATION . 50
i
APPENDICES
APPENDIX I: AGENCY COMMENTS _. . 51
APPENDIX II; OIG EVALUATION OF AGENCY COMMENTS 80
APPENDIX III: ACTIONS TAKEN ON 1991 OIG IFMS REPORT
RECOMMENDATIONS 92
APPENDIX IV: EPA'S FINANCIAL MANAGEMENT SYSTEMS
IMPLEMENTATION PLAN IN 1988 96
APPENDIX V: EPA'S FINANCIAL MANAGEMENT SYSTEMS
OF DECEMBER 1993 97
APPENDIX VI: IFMS MANAGEMENT STRUCTURE 98
APPENDIX VII: SUMMARY OF IFMS SYSTEM LIFE
CYCLE COSTS 99 j
APPENDIX VIII: CUSTOMER SATISFACTION SURVEY
ANALYSIS 104|
APPENDIX IX: PLANNED AND ACTUAL IMPLEMENTATION OF IFMS
COMPONENTS AS OF 12/93 1271
APPENDIX X: UNCORRECTED AND CORRECTED LONG-
STANDING FINANCIAL MANAGEMENT
WEAKNESSES REPORTED UNDER FMFIA
(FISCAL 1989 - 1993) 1281
Report No. E1NMF3-15-0073-41005611
-------
EPA's IFMS
APPENDIX XI: . IFMS AND LEGACY SYSTEM COSTS
APPENDIX XII: ESTIMATED ANNUAL AND TOTAL COST OF
MAINTAINING LEGACY SYSTEMS (1990 - 1995)
APPENDIX XIII: STATUS OF IFMS CHANGE REQUESTS
APPENDIX XIV: ACRONYMS
APPENDIX XV: REPORT DISTRIBUTION - .
129
130
131
132
136
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 ZFHS
This page intentionally left blank.
Report No. E1NMF3-15-0073-41005611
-------
EPA'8 IFHS
CHAPTER 1
INTRODUCTION
PURPOSE
EPA tracks and controls over $6.9 billion dollars annually in
support of its mission. To accurately account for such a large sum
of money requires an effective system of accounting and internal
controls to monitor outlays and ensure that government assets are
adequately safeguarded. In addition, in the last several years, the
Federal Government has increased its commitment to improve the
quality and effectiveness of financial management systems. As a
result, EPA has taken action to implement a modern, efficient, and
integrated financial management system. This report describes the
status of the implementation of this system.
The objectives of the audit were to: (1) assess the current progress
of IFMS implementation and interfaces in accordance with OMB
Circular A-127; (2) verify the implementation of controls addressing
the IFMS problems in prior FMFIA reports; (3) determine whether
generally accepted system development practices were employed
throughout IFMS; (4) assess whether maintenance activities were
economically and efficiently managed; (5) identify systems which may
be duplicating functions of IFMS; (6) assess user satisfaction with
IFMS through the issuance of a user satisfaction questionnaire; and
(7) verify the adequacy of actions to implement the recommendations
in the March 1991 IFMS audit report--"Integrated Financial
Management System: Managing Implementation Of The New Accounting
System" (Report No. E1AMFO-11-0029-1100153). (See Appendix ill.)
BACKGROUND
Since the mid 1980's, EPA has been involved in the development of a
major system, the Integrated Financial Management System, in order
to integrate and modernize the Agency's accounting and budgeting
systems which track and control over $6.9 billion annually. In
September 1987, EPA purchased a commercially available off-the-shelf
software package (i.e., Federal Financial Systems (FFS) developed by
American Management Systems, Inc. (AMS)) which contains a variety of
core accounting and financial management modules. Appendix IV
provides a schematic view of EPA* s financial systems configuration
based on the 1988 IFMS. implementation plan. The Agency originally
planned to implement IFMS at an estimated cost of $7.7 million in a
3-phased approach by October 1990. Phase 1 involved bringing up the
accounting and budget modules contained within IFMS and eliminating
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
the Agency's dependency on three systems: FMS, RMIS, and ADCR.
Phase 2 involved bringing up additional project/cost accounting and
budget preparation modules within IFMS and implementing a new
account code structure. Phase 3 involved the integration/inter-
facing of IFMS with other financial subsystems and administrative
systems. However, in 1990, because of the implementation and
conversion problems, EPA deferred major implementation tasks and
focused on stabilizing the operations of the software and on the
development of the ad hoc reporting system--Management and
Accounting Reporting System (MARS).- EPA planned to complete these
tasks before working on other system capabilities. To date, only
parts of the three phases have been completed.
Appendix V shows a schematic view of the systems as of December 1993
which make up EPA's financial management systems as defined in OMB
Circular A-127. These include the "core financial systems" (i.e.,
IFMS, FMS, RMIS, ADCR, MARS, and CPARS). Other finaneial.and
administrative systems depicted in Appendix V include EPA Payroll
System (EPAYS), Grants Information and Control System (GICS),
Contract Payment System (CPS), Personal Property and Accounting
System (PPAS), Contract Information System (CIS)/Automated
Procurement Documentation System (APDS), and Comprehensive
Environmental Response, Compensation, and Liability System
(CERCLIS).
In September 1990, the Agency established, through an IFMS Charter,
two committees (the Executive Management Group (EMG) and the System
Management Group (SMG)) for setting the direction of IFMS and
monitoring progress. These groups provided information to the
Administrative System Council (ASC) on IFMS issues. Currently, the
responsible division offices (Financial Management Division (FMD),
Budget Division (BD), Administrative Systems Division (ASD), and
National Data Processing Division (NDPD)) manage day-to-day IFMS
activities (see Appendix VI). The IFMS Charter also established the
IFMS Project Manager position (currently, the Director, IFMS.Project
Management Staff), in which the incumbent serves as the principal
point of contact for Agency senior management on issues connected
with IFMS and for monitoring and evaluating IFMS progress towards
meeting its goals. In response to the CFO Act of 1990, the
Administrator designated the AA, OARM as the CFO. In addition, the
Administrator issued a delegation of authority which delegated the
management and operation of the agency systems to the AA, OARM. The
AA, OARM further redelegated this authority through the DAA for
Finance and Acquisition to the Comptroller.
IFMS runs on two International Business Machines (IBM) ES/9000
mainframes with a Multiple Virtual Storage/Enterprise Systems
Architecture (MVS/ESA) operating system. IFMS operates in both
"online" and "batch" mode. It operates in a Customer Information
Control System (CICS) environment using Adaptable Data Base System
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
(ADABAS), which is a hierarchial data base management system. IFMS
was developed using AMS's.FFS (which is an off-the-shelf software
package used to meet standard, Federal Government financial
requirements) along with interfaces to other systems. IFMS
currently processes over two million transactions on an annual
basis. EPA is currently running FFS version S.le software. EPA's
financial systems including IFMS have approximately 2,500 users in
headquarters, regions, and laboratories.
SCOPS AND METHODOLOGY
The primary focus of this review was on EPA's core financial systems
including IFMS. The audit fieldwork was performed from November
1992 to December 1993, primarily at EPA Headquarters, Washington, DC
and the NDPD, Research Triangle Park (RTP), Nortph* Carolina (NO. We
did not perform a detailed review of Agency actions subsequent to
our fieldwork, but have recognized management actions based on
documents provided.
To accomplish our objectives, we reviewed the following:
•• The Budget and Accounting Procedures Act of 1950 (as amended);
the FMFIA of 1982; the CFO Act of 1990; the Paperwork Reduction
Act (PRA) of 1980 (as amended); the Government Performance and
Results Act of 1993; the National Performance Review (NPR); the
detailed requirements of OMB Circulars A-11, A-109, A-123,
. A-127, and A-130 and revisions or draft revisions; General
Accounting Office's (GAO) Title 2; GAp's Critical Factors in
Developing Automated Accounting and Financial Management
Systems; Joint Financial Management Improvement Program (JFMIP)
requirements; Controller's Performance Measurements; Federal
Information Resource Management Regulation (FIRMR); applicable
Federal Information Processing Standards (FIPS) Publications;
applicable OMB bulletins and guidance; and General Services
Administration (GSA) guidance entitled "A Guide For
Requirements Analysis And Analysis of Alternatives."
• EPA Directive 1100 on mission and functions; 'EPA Directive 1200
'on delegation of responsibilities; EPA Directive 2100 on IRM
policies; EPA Resources Management Directive (RMD) 2580 on
financial management systems; EPA RMD 2560 on internal
controls; EPA temporary directives on System Design and
Development Guidance (1989); EPA Operations and Maintenance
Manual (1990); IRM Steering Committee minutes since 1985; ASC
meeting minutes since 1990; EMG and SMG meeting minutes; IFMS
Charter; and EPA organizational charts.
• IFMS Analysis of Alternatives and Action Plan for
Implementation of System Improvements (or Feasibility Study)
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFHS
(dated 1985); EPA IFMS System Alternatives Implementation
Analysis (dated December 1986); IFMS Report Of The Current
System (dated July 1986); IFMS Requirements Report (dated
August 1986); Conceptual Design for IFMS (dated September
1986); System Decision Papers and correspondence; draft
Strategy and Master Work-plans (dated October 1992 and January
1993); Final Strategy and Master Work Plan (dated July 1993);
Financial Management Status Report and Five Year plans (1987-
1990 and 1992-1993); IFMS Feasibility and Cost-Benefit Task
Order; Requirements Analysis and Options Definition for an
Integrated Financial Management System (dated January 1994);
EPA Account Number Structure studies (dated 1988, 1992, and
1993); FMS/RMIS Replacement Studies (dated May 1991 and March
1992); Working Capital Fund Feasibility Study (dated April
1991); Key Financial Management Weaknesses Memorandum (dated
October 1992); and EPA's 1989-1993 FMFIA Reports to the
President and Congress.
EPA's Information System Inventory (ISI) ,- A-127 Review Report
regarding IFMS; IFMS system and user documentation; Change
Management System (CMS), Action Request Tracking System (ARTS),
and NDPD Problem Management Detailed Record; Online Timeshare
Utilization System (OTUS) system documentation and samples of
these systems' data; FFS versions 4.0, and 5.0 software
documentation; IFMS and MARS subrelease documentation; IFMS
Online Statistics reports; AMS Hotline status reports; NDPD
Operational Policies Manual; "Applied Software Measurement,"
Capers Jones, copyright 1991; "Software Engineering," Roger E.
Pressman, copyright 1992; "Foundations of Business Systems,"
Arthur Anderson and Co., copyright 1989; "Managing The Software
Process," Watts Humphrey, copyright 1989; IFMS FFS data
dictionary; MARS Data Element Dictionary; and PREDICT data
dictionary.
Five contracts related to the core financial system and
interfaces from AMS (three contracts), Booz-Allen and Hamilton,
and Science Applications International Corporation (SAIC); a
sample of related task orders, delivery orders, work
assignments deliverables, and progress reports; EPA Letters of
Interest; EPA Acquisition Procurement Request (dated July
1987); IFMS Delegation of Procurement Authority (dated July
1987); and AMS Best And Final Offer (dated September 1987).
Cost information of Agency systems through the EPA OMB Circular
A-11 submissions; and NDPD Billing Report Distribution System
(INFOPAC) reports. Using this information we estimated the
IFMS system life cycle costs. (See Appendix VII for detailed
methodology).
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
• Prior 1991 DIG report on IFMS and supporting workpapers; follow
up documentation supporting report recommendations; other
applicable OIG reports; and OMB Report on IFMS dated September
1993.
• FMFIA documentation for 0'f.fice of Information and Resources
Management (OIRM), NDPD, and selected Office of the Comptroller
(OC) divisions including .management control plans; assessable
unit assurance letters; event cycle listing; OARM 1993 FMFIA
assurance letter; GAO's Standards For Internal Controls; and
GAO report "Meeting the Government's Technology Challenge"
(dated February 1990).
We interviewed senior officials including the Acting DAA for Finance
and Acquisition; Deputy Director, NDPD; the Acting Comptroller; the '
Comptroller; and the Acting Directors for OIRM and FMD. We also
interviewed officials and requested documentation from IFMS Project
Management Staff, FMD, BD, ASD, Information Management Services
Division, Management and Evaluation Staff, and NDPD. Position
descriptions and performance standards were reviewed, as
appropriate. We coordinated our efforts with OIG's Financial Audit
Division (FAD) and -their ADP subcontractors; and OIG auditors on the
President's Council on Integrity and Efficiency (PCIE) software
maintenance audit regarding IFMS issues. In addition, we conducted
a statistically based user satisfaction survey of users of the core
financial systems--IFMS, RMIS, ADCR, FMS, and MARS. See Appendix
VIII for methodology.
We spoke to GSA and OMB officials to obtain clarification of policy
issues and information on IFMS. Additionally, we interviewed:
(1) AMS officials on issues regarding the off-the-shelf software and
implementation of financial systems at other Federal agencies';
(2) an OGDEN Government Services official regarding the IFMS
requirements and cost analysis study; and (3) Martin Marietta
officials regarding the data dictionary and system billing.
information. We also interviewed or reviewed documentation from the
Securities and Exchange Commission (SEC), Department of
Transportation (DOT), Internal Revenue Service (IRS), and Department
of Veteran Affairs (VA) on matters regarding their integrated
financial management systems.
We conducted this audit in accordance with Government Auditing
Standards (1988 revision) issued by the Comptroller General of the
United States. Our audit included tests of management and related
internal controls, policies, and standards. We did not perform a
detailed review of the sources and systems which generated the cost
figures for accuracy and reliability as it was outside the scope of
the audit. Additionally, we did not perform a data integrity audit
of IFMS. No other issues came to our attention which we believe
were significant enough to warrant expanding the scope of the audit.
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFKS
PRIOR AUDIT REPORT COVERAGE
A March 1991 OIG audit report, entitled "Integrated Financial
Management System: Managing Implementation Of The New Accounting
System" (Report No. ElAMFO-llr0029-1100l53), reviewed the IFMS
implementation up to October 1990. The report cited that: (1) the
Agency planned to spend more than double its initial estimate to
implement the system; (2) the project had been extended three years;
(3) users were dissatisfied with the system; and (4) many key
requirements had not been met. The report also cited that EPA did
not update critical decision-making documentation (e.g., such as the
cost-benefit analysis), or fully comply with generally accepted
system test practices. The report made 13 recommendations to
address the problems identified by the audit. The Agency
implemented nine of the recommendations and partially.implemented
the remaining four recommendations. See Appendix III for a
discussion of the status of each 'of the recommendations.
Report No. E1NMF3-15-0073-4100561J
-------
EPA's IFMS
CHAPTER 2
COSTS AMP FINANCIAL MANAGEMENT RISKS ARE CONTINUING
TO ESCALATE BY MOT FULLY IMPLEMENTING IFMS
EPA has taken a number of significant steps to implement an
integrated financial management system and has made progress in
overcoming previous management problems. For example, EPA
(1) implemented a single general ledger, which effectively
consolidated 14 Servicing Finance Offices' (SFO) individual general
ledgers that were previously managed separately and decentrally;
(2) implemented some functional modules, system interfaces, and
software subreleases; (3) implemented MARS; (4) appointed a CFO;
(5) reorganized OARM financial management functions including
assigning a DCFO; (6) issued financial management"plans;
(7) appointed a Director, IFMS Project Management Staff (formerly
the. IFMS project manager); (8) issued a IFMS Charter which outlines
the project structure; (9) completed the IFMS Strategy and Master
Work Plan; (10) implemented the off-the-shelf software version 5.1e
which upgraded IFMS capabilities; (11) approved decisions on the
direction of IFMS over the next seven years; (12) completed a cost-
benefit analysis for the IFMS fixed assets subsystem; (13) updated
the IFMS cost study and requirements analysis;- (14) established a
plan to implement an enhanced fixed asset subsystem in IFMS and to
reconcile data in PPAS; (15) revised its IFMS Charter setting out
IFMS roles and responsibilities; and (16) implemented a new CMS.
Despite these accomplishments, the Agency has yet to fully implement
an integrated, comprehensive financial management system critical to
EPA's mission. Target dates for completing-IFMS implementation
(Phases 1-3) have slipped more than 6.5 years to at least fiscal
1995. Until full implementation is attained, the return on the
planned IFMS investment over the remaining system life will be
limited. Specifically, EPA has not: implemented critical modules;
eliminated their dependency on the legacy financial systems, which,
in part, duplicate IFMS functions and capabilities; and fully
interfaced IFMS to financial subsystems and other administrative
systems. Also, EPA has taken several years to upgrade to more
current software versions, and has not completed many significant
changes in the maintenance process in a timely manner. In addition,
EPA has not taken timely corrective action regarding its
longstanding financial management problems reported under the FMFIA,
many of which IFMS was to correct.
EPA has incurred cost overruns and other costs as a result of
ineffective IFMS implementation. For instance:
• development and implementation costs have escalated from $7.7
million to $17.2 million. These costs include, at least,
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 XFMS
$148,100 EPA spent or will spend on software and maintenance
costs for modules that the Agency did not implement;
• EPA will incur additional costs to operate and maintain the
legacy systems with an estimated total cost of $11.2 million
from 1990 to 1995--less unknown offsetting processing costs (no
Agency estimate available) which would be incurred until IFMS
is .fully implemented. The legacy systems were scheduled for
elimination in September 1989; and
• EPA may incur additional expenditures ranging from an estimated
. $415,000 to several million dollars over the next couple years
to update the account code structure and convert historical
data from financial and administrative systems.
Moreover, as a result of delays, management functions critical to
EPA's mission can be significantly improved. For example, the
delays in implementing the project and cost allocation accounting
module and the accounts receivable module have impaired Agency
managers' abilities to effectively manage the financial and
budgetary aspects of their programs or, in the case of Superfund, to
assemble cost recovery packages. In addition, obsolete systems are
being maintained which are susceptible to failure, potentially
making critical financial management data unaccessible for an
indefinite period of time. For example, two of the legacy systems
(FMS and RMIS), which are not scheduled to be eliminated until
fiscal 1995, were developed over-20 years ago, and are written in
outdated languages which are no longer supported by industry.
While IFMS has electronic interfaces with the EPAYS and CPS, it is
not linked with other important EPA systems--GICS; PPAS; CIS/APDS;
CERCLIS; and others--in a way that permits Agency managers to make
sound program and financial management decisions and has resulted in
a highly complex and unwieldy financial management environment.' For
example, complex manual reconciliations are necessary where
automated interfaces are not in place, and lack of interfaces causes
duplicate data entry--a material nonconformance reported in the 1993
EPA FMFIA report.
Over the past year, EPA top management involvement with and
oversight over IFMS development have increased substantially.
However, top management and steering committees need to continue to
take the strong, decisive, and sustained leadership that is
necessary to follow through on IFMS implementation. In addition,
EPA has fragmented managerial authority and has not clearly defined
lines of authority among the four organizations primarily involved
in IFMS. Decisions were made without an adequate plan that used
costs based on a system life cycle approach. EPA also did not
follow a generally accepted SDLC approach and over-customized the
off-the-shelf software.
8
Report No. E1NMF3-15-0073-410056!
-------
EPA'8 IFHS
FINANCIAL INFORMATION SYSTEM CRITERIA WELL ESTABLISHED
The Budget and Accounting Procedures Act of 1950 (as amended)
requires the Comptroller General to establish accounting principles
and standards, and internal control standards. This law requires
that these systems conform to the prescribed requirements and that
the head of each Federal agency be responsible for maintaining
adequate systems of accounting and internal controls. The FMFIA
also requires that internal accounting and administrative controls
be implemented in accordance with the standards established by the
Comptroller General. In addition, the PRA requires that agencies
perform their information management activities in an efficient,
effective, and economical manner.
The CFO Act of 1990, in part, provides for the establishment of a
CFO who reports to the agency head on financial management matters.
The CFO's responsibilities include developing and maintaining an
integrated financial management system; directing, managing, and
providing policy guidance and oversight of all financial management
personnel, activities, and operations; and approving and managing
financial management operations and improvements. One of the major
intents of the CFO Act was to provide a strong centralized
leadership to solving financial management system problems,.
Additionally, the House of Representatives Bill ,(H.R. 3425) to
create a cabinet level Department of the Environment, addresses the
need for central direction and control of IRM activities--through
the establishment of a Chief Information Officer (CIO). This
official's responsibilities would include, among other things,
developing comprehensive processes for controlling information
systems development and operations life cycle. It also specifically
requires, in part, that this official, in cooperation with the CFO,
ensure that financial systems are effectively designed, developed,
and implemented and identify opportunities to redesign business
practices and supporting information systems to improve agency
performance.
To implement the above Acts, the OMB, GAO, JFMIP, and Treasury
Department have issued various policies and procedures. OMB
Circular A-127, entitled "Financial Management Systems," prescribes
policies and procedures for establishing and maintaining a single
integrated financial management system. This Circular, requires
that: the agencies establish and maintain a single, integrated
financial management system which may be supplemented by subsystems;
data entered into the system and other systems is to be entered only
once and transferred automatically; and the expenditure of funds be
limited to only those systems that meet the requirements of the
Circular. The Circular also requires that' financial systems which
are excessively costly be identified and phased out and that systems
which overlap or duplicate other systems be eliminated. Further,
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
the revision of OMB Circular A-127 (dated July 1993) defines a
single, integrated financial management system as a unified set of
financial systems and the financial portion of mixed systems. Also,
it defines "unified" in terms of systems which are planned for and
managed together, operated in .an integrated fashion, and linked
together electronically in an efficient and effective manner. The
Circular also requires that agencies comply with the functional
system requirements as defined by JFMIP.
OMB Circular A-130, entitled "Management of Federal Information
Resources," among other things, requires that: (1) information
systems operate efficiently and accurately; (2) existing or planned
information systems do not unnecessarily duplicate information
systems; and (3) agencies establish management controls. These
management controls should document the requirements that each major
system is intended to serve, provide for the periodic review of
those requirements to determine whether the requirements continue to
exist, and ensure the system continues to meet the purposes for
which it was developed.
OMB Circular A-123, entitled "Internal Control Systems," requires
that .agencies establish, maintain, evaluate, improve, and report on
internal controls. OMB Circular A-109, entitled "Major System
Acquisitions," requires that systems be acquired and developed on a
system life cycle basis.
GAO's Title 2, entitled "Policy and Procedures Manual for Guidance
of Federal Agencies," prescribes accounting system standards and
requirements that the agencies' heads must observe in establishing,
maintaining and reporting on their systems of accounting and
internal controls. Appendix 3 of Title 2 prescribes standards and
related guidance for the development and operations of accounting
systems. These include: general system requirements for systems in
operation, reporting system requirements, standards for system
development and the maintenance process, and requirements for
documentation of system development projects and accounting systems.
In addition, under an interagency project, the JFMIP has developed
uniform requirements for the core financial systems of Federal
departments or agencies. These uniform requirements are contained
in the January 1988 publication entitled "Core Financial System
Requirements" and are incorporated by reference into the accounting
system standards prescribed by the Comptroller General. The
Treasury Department has also issued requirements 'on the Standard
General Ledger to facilitate standardized government accounting and
the preparation of standard external reports. These.requirements
represent minimum standards for all financial activities, financial
system operations and support functions, and system hardware
configuration.
10
Report No. E1NMF3-15-0073-4100561J
-------
EPA'8 IFHS
The FIRMR is the primary regulation governing acquisition,
management and use of Federal information processing resources by
agencies. This regulation, among other things, sets forth
requirements for conducting a requirements analysis and an analysis
of"alternatives. The Commerce.Department has also issued FIPS
Publications which prescribe standards and guidelines on system
development processes (e.g., FIPS Publications 38, 64, 101, and
106}. For instance, FIPS 64 identifies a number of documents
critical to the system development process including feasibility
study, cost-benefit analysis (including an analysis and comparison
of alternatives), and project request. GSA's "Guide for
Requirements Analysis and Analysis of Alternatives" (dated January
1990) defines the steps involved in performing a requirements
analysis.
EPA's RMD 2580 prescribes policies and procedures to be followed in
developing, operating, evaluating, and reporting on components of
its financial management system. Additionally, EPA Directive 2100
provides policy statements on software management, ADP resources
management, and data standards. EPA Directive 2100 requires that
significant technology investments be evaluated periodically through
an analysis of information requirements studies and cost-benefit
studies. Also, EPA has issued guidance, entitled "EPA System Design
Development Guidance" and "Operations and Maintenance Manual." This
guidance has been issued as temporary EPA Directives which provide
specific requirements throughout a system's life cycle. The
guidance prescribes that cost-benefit analyses are evolving
documents, which should change as the project progresses and becomes
better defined. Therefore, for those efforts which proceed to the
design, development, and implementation phases, these documents will
require updating. The Operations and Maintenance Manual also
defines operational baseline as the completely implemented and
tested software system. This operational baseline is the basis for
future maintenance changes and enhancements and is established when
software has been placed into production and/or turned over to the
user.
IFMS NOT FULLY IMPLEMENTED
Kev Modules Not Implemented
EPA planned to implement 11 FFS modules provided in the off-the-
shelf package from January to October 1989. Implementation of these
modules would allow EPA to meet the standard, Federal Government
financial management requirements. We found that as of December
1993, five of the modules were implemented; one was partially
implemented; one was entirely customized; and four were not
11
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
implemented as planned.(see Appendix IX). In addition, the accounts
receivable module was implemented with technical shortfalls in the
debt servicing area regarding calculation and posting of interest,
penalties, and handling charges. We also found that another module
for the budget formulation (also called the budget preparation
module) was customized in its entirety to meet EPA's needs.
The four modules that were not implemented as planned were the
project and cost allocation accounting module, the payroll module,
the fixed asset module, and the reporting module; which were to
provide critical functionality to the financial management system
and satisfy Federal requirements. The first module (project and
cost allocation accounting) was never purchased, but annual
maintenance fees were being paid. The second module (payroll) was
purchased and annual maintenance fees were paid,.but it was not
installed. In June 1994, the Agency indicated that they were
investigating the project and cost allocation accounting module
modified by another agency; and installed CPARS in lieu of the AMS
payroll module. Further, the third module (fixed assets) has been
installed and annual maintenance fees are being paid, but.the module
has not been implemented. In July 1994, the Agency indicated they •
were moving forward with developing and pursuing a implementation
plan for an enhanced fixed asset module being developed by another
agency. In addition the forth module (off-the-shelf reporting) did
not meet EPA's financial reporting requirements. As a result, the
Agency developed a separate reporting system (i.e., MARS) which was
never intended in its original plan and still does not always meet .
the Agency reporting needs. For instance, MARS does not provide
reports necessary to support some day-to-day accounting operations
such as transactions input listings and external financial reports
submitted to Treasury and OMB. The chart on the next page
summarizes costs for software and/or maintenance on modules the
Agency did not implement.
Additionally, EPA may be paying annual maintenance for software
modules not implemented. These include two modules--Report Painter,
and IMAGINE--which were identified on an August 1992 GSA contract.
Report Painter was originally part of the IFMS basic system
purchased in fiscal 1988. In April 1993, EPA's ASD reported a
significantly different inventory of IFMS modules than the original
contract. We attempted to verify some evidence of recordation of
these modules as a fixed asset in IFMS. However, FMD does not have
a capitalized fixed asset inventory of IFMS software and
enhancements, nor does it have a net present value of these costs in
IFMS. Therefore, we could not reconcile EPA's IFMS modules with the
contract records or the accounting system's records.
12
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
Identified Potential Coats For
Modules8 Not Implemented
Module
Project
Accounting
Cost
Allocation
Property/Fixed
Assets
Payroll
Report Painter
Imagine
total
Original
Purchase
Price
$ 0
$ 0
$15,000
$40,000
$ 4,500
Could Not
Determine
$59,500
Maintenance
Fees
Fiscal 1989-95
$10,800
$22,950
$16,800
$16,000
$ 1,800
$20,250
$88,600
Total Cost
$ 10,800
$ 22,950
$ 31,800
$ 56,000
$ 6,300
$ 20,250
$148,100
Leaacv Financial Systems Not Eliminated
Contrary to EPA plans, OMB Circulares A-127, and A-130 on duplicate
and obsolete systems, we found that EPA continues to use legacy
financial systems that duplicate, in part, the functions and
capabilities of IFMS. EPA originally planned to eliminate their
dependency on these systems--FMS, ADCR, and RMIS--by September 1989
(see Appendix IX). Based on a review of Agency acquisition
documents, one of the major benefits and justifications for
implementing IFMS was the replacement of FMS (i.e., the legacy
accounting system) which was not capable of meeting the OMB Circular
A-127 requirements and the goal of providing a modern, efficient
financial management environment. However, FMS is still being used
for various functions -such as ad hoc reporting of historical data
and processing detailed payroll information. This system is over 20
years old; batch-oriented; and written using now obsolete data base
management systems. Similarly, RMIS (used for budget preparation)
is an old, outdated system, which is not easily maintained. While
ADCR (used for commitments, some obligations such as travel, and
FFS Basic System was purchased in September 1987 for $377.764 and maintenance of $48,372 a year tor
four years. The financial reporting software was included as part of these costs', and the costs were not
computed separately. The Imagine module was not listed in the original contract. Module costs that could be
identified during the original and the follow-on maintenance support service contract were included.
13
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
status of funds) is relatively new and maintainable, it duplicates
functional capabilities in IFMS and is costly to operate and
reconcile with other systems. We estimate the annual cost.of
operating and maintaining these legacy systems to be as much as $2
million (see Appendix XII). As of December 1993, EPA planned to
eliminate these three systems by September l995--over 6.5 years
after IFMS was put into production.
Interfacing Of EPA's Financial Systems Not Fully Addressed
Contrary to OMB Circular A-127, JFMIP requirements, and EPA's plans,
EPA has not fully interfaced its financial subsystems and other
administrative systems with IFMS. EPA planned to interface IFMS ' -
with EPAYS, GICS, CPS, PPAS, and CIS/APDS by 1990. While IFMS is
currently interfaced with EPAYS and CPS, we found that PPAS and GICS
have not been fully interfaced with IFMS and are not scheduled to be
completed until September 1995 (See Appendix IX). Additionally, EPA
officials have deferred work on CIS/APDS because they plan to
replace these systems, and have not yet interfaced IFMS with
CERCLIS.
Moreover, contrary to its plans, EPA did not standardize key
processes during the initial phases of development which are vital
to integrating its financial systems and establishing a foundation
for Agencywide financial processing. For instance, EPA did not
develop and implement a standard accounting classification code
which would provide more than 10 digits to accommodate its financial
operations. Currently, EPA has a 10 digit account code which does
not meet the Agency's needs. Internal and external studies6 and
internal correspondence have shown that the 10 digit account code
structure is limited, because available coding elements are
exhausted for certain data elements and the account code structure
cannot accommodate new user requirements and new congressional
appropriations. . In March 1993, the Acting Comptroller announced a
project to build capacity for.an expanded account code structure.
We agree with this decision because without an expanded standard
account code structure in place, efficient and effective integration
of existing Agency financial systems into IFMS will be very costly
and difficult to achieve.
Additionally, a single, comprehensive data dictionary for IFMS and
other financial systems is critical to the management of an
integrated system. The OIG March 1991 IFMS report noted that the
IFMS data dictionary was not adequate. The 1986 IFMS Requirements
Analysis also noted the users need for an on-line IFMS data
Studies on the redesign of EPA's account code structure have been performed by Arthur Young, Inc.
(1989); SPA'S Financial Management Division (1988); Booz-Allen and Hamilton, Inc. (1992); and DynCorp-Viar,Inc
(1993).
14
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFHS
dictionary. However, EPA still has not adequately implemented such
a data dictionary. Instead, we found an inadequate IFMS data
dictionary and multiple data dictionaries (e.g., IFMS and MARS data
dictionaries). ASD officials provided a "Table Definition Report"
which they referred to as their "data dictionary." This document
lists the IFMS tables showing all data element names in each table
and six descriptive fields (e.g., starting and ending position for
each data element, length, and type). However, we concluded that
this is not an adequate and comprehensive data dictionary, because
it does not provide: a definitional statement for1 each of the data
elements; information on data element usage, sources, and formats;
descriptions of the relationships to other data; or descriptions of
files, data bases, and programs that use those files and data bases.
A useful data dictionary will also .support several versions of the
same data. In our*view, the Agency should have a.single,
comprehensive data dictionary for all IFMS related systems because
multiple data dictionaries undermine the above stated goals of a
data dictionary. Also, EPA has an on-line data dictionary for
ADABAS known as PREDICT which could include a single, comprehensive
IFMS data dictionary. However, as of November 1993, the PREDICT
data dictionary was not comprehensive, and completion of it was riot
scheduled.
Delays in Software Upgrades
EPA did not upgrade IFMS in a timely manner to the more current FFS
software versions 4.0 and 5.0. EPA was running a customized FFS
version 3.0 since implementation in 1989. Subsequently, EPA
initiated two major IFMS projects to develop and implement the
reporting system MARS, and upgrade to the latest customized FFS
version 4.0. However, for over 15 months, EPA attempted to upgrade
to a customized version 4.0 (which was initially scheduled for
October 1991) without successf This upgrade offered improved
functionality in the accounts receivable and travel modules.
Instead, EPA decided in 1992. to implement a consolidated version 4.0
and 5.0 (eventually referred to as EPA's version 5.1e) and scheduled
the combined release for February 1994. The upgrade from FFS
version 3.0 to version 5.0 offered increased functionality in the
accounts receivable, budget execution, accounts payable, and
automated disbursement modules, as well as improved security
features; and takes advantage of enhancements designed by EPA as
well as other Federal agencies. After the completion of our
fieldwork, the Agency implemented the off-the-shelf software version
S.le which upgraded IFMS capabilities.
Software Maintenance Requests Not Implemented For Years
Many software maintenance/enhancement requests which impact the core
financial management systems were not implemented, completed, or
closed. We analyzed ASD's CMS--the primary system which tracks IFMS
15
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IPMS
development, maintenance, operations activities, or other types of
contractor support--and found that out of 237 work requests7, only
25 (10.5 percent) were implemented/completed/closed. The remaining
212 {89.5 percent) work requests were not implemented, completed, or
closed. The majority of these requests dealt with software
modifications and were in the -high priority category. Also, several
requests dated back to 1991 (see Appendix XIII). We recognize that
some of the older requests were overtaken by the recent upgrade to
version 5.1e, which made some requests obsolete. However,
management still needs to reexamine the status of the outstanding
requests.
In addition, we identified three other systems (ARTS at Financial
Systems Branch, Problem Management Detailed Record at NDPD, and the
IFMS Hotline run by AMS) which track and/or manage system problems
that can result in software changes to IFMS and fche core financial
management systems within different divisions. The problems, change
requests, and issues identified in each system are either tracked to
implementation, or passed to CMS for implementation. Once a
proposed change has been approved by the Change Control Board, it
would clearly fall under the configuration management process in'
CMS. It should be clear when the change control board approves a
change, that it becomes a system change request controlled under
CMS. Subsequent to our fieldwork, the Agency implemented a new
version of CMS.
Critical System Development Life Cycle Documents Not Updated
Despite longstanding requirements for system development and our
1991 IFMS audit report recommendation, EPA had not updated critical
decision-making documents over the life of IFMS. For instance,
while an initial requirements analysis was prepared, it was not
updated to reflect current requirements. The originally identified
"key requirements" from 1985 were primarily used as a basis for
IFMS's system planning documents, and the current requirements were
neglected. For instance, requirements were not updated to reflect
the need for a standard account code structure, elimination of the
legacy systems, and implementation of a project and cost allocation
accounting module, a fixed asset module, automated system
interfaces, an on-line comprehensive data dictionary, and workload
and performance specifications. The Director, IFMS Project
Management Staff agreed that the system has changed-and they have
not measured or kept track of these requirements.
Since our IFMS report in March 1991, EPA had scheduled an update to
the original requirements and cost-benefit study, but, as of
. CMS contained 237 work requests (a work request may require one or multiple program change*). EPA
officials indicated approximately 3000 modifications baaed on these work requests are made per year to IFMS.
16
Report No. E1NMF3-15-0073-4100561
-------
SPA'S IFMS
December 1993, had not updated these documents due to other
priorities and la'ck of funding. Without these documents, EPA can
not be assured the system will meet its intended objectives, and
user needs. These documents are also critical for current and
future financial management system planning decisions. For example,
the system manager also needs these documents for developing
alternative, cost-beneficial technical and contractual solutions.
Subsequent to our fieldwork, EPA updated its 1985 requirements
analysis in January 1994 and completed a cost analysis in April
1994. • . .
LonQStandingmgMFIA Weaknesses And Nonconformances8 Not
Corrected
.*
Contrary to OMB Circular A-123 and EPA Directive 2560, EPA has not
taken timely corrective action regarding its longstanding financial
management system weaknesses and nonconformances. Implementation of
IFMS was supposed to eliminate many of these weaknesses and
nonconformances, one of which has existed for over a decade (i.e.,
reconciliation between property and accounting records). EPA
submits a plan to correct these weaknesses every year; however,
target dates continually slip and the weaknesses and nonconformances
remain. We found that: (1) IFMS has remained on OMB's high risk
list since 1989; and (2) four of eight nonconformances identified
between 1983 and 1989 are still not reported as corrected. Further,
in 1992, EPA identified two additional material weaknesses--the IFMS
and the accounts receivable (see Appendix X). For instance,
excerpts from the FMFIA report showed that:
• . OMB's high risk list includes IFMS because of the reported
nonconformances and weaknesses in EPA financial systems and the
customizing of IFMS to meet EPA's needs has not been completed.
• IFMS weaknesses (referred to as accounting system-related
financial management problems) were reported as material,
because of specific system related -problems which in the
aggregate, continue to impair EPA's ability to provide
complete, reliable, and timely data for Agency decision-making
and asset control.
• Accounts receivable was reported as a material weakness because
of technical shortfalls in the IFMS accounts receivable module,
noncurrent policies and procedures, inaccurate and incomplete
reports, and insufficiently trained personnel.
8,
Non-conformance items are those which do not comply with OMB A-127 objectives and the JFMIP Federal
Financial Management System Requirements.
17
Report No. E1NMF3-15-0073-4100561
-------
EPA'8
In addition, a June 1993 report, entitled "Report of Financial
Audit" on the Fiscal 1992 Financial Statement Audit of the Superfund
Trust Fund, Leaking Underground Storage Tank Trust Fund, and
Asbestos Loan Program, referred to many of these nonconformances
that have not been corrected since the 1980's. Among these are the-
(1) lack of centralized accounts receivable recbrdation; (2)
inadequate reconciliation of the property system to the general
ledger; and (3) inattention to automation and interface
opportunities.
ADVERSE EFFECTS OF NOT FULLY IMPLEMENTING IFMS
Cost?Overruns. Other Costs. And Lack OfFunctionality
EPA will spend, at least, an estimated $17.2 million (see Appendix
XI) to fully implement IFMS, which is a cost overrun cost of almost
$10 million. These costs include, at least, $148,100 EPA spent or
will spend on software and maintenance costs for modules that the
Agency did not implement.
By delaying the elimination of the legacy systems, EPA continues to
spend its limited resources every year on maintenance and operations
for duplicate functions in FMS, RMIS, and ADCR. EPA will incur
additional costs to operate and maintain the legacy systems. These
costs are estimated to be $11.2 million from 1990 to 1995--less
unknown offsetting processing costs (no Agency estimate available).
when IFMS is fully implemented. The legacy systems were scheduled
for elimination in September 1989 (see Appendix IX). Because EPA
did not eliminate its legacy systems, additional interfaces for FMS,
ADCR, and RMIS,'not previously scheduled, were required. This has
added to the IFMS complexity and the cost of maintenance and
software .upgrades.
Additionally, not'eliminating these legacy systems and not creating
a standardized expanded account code structure may result in other
significant unplanned expenditures. EPA has performed various
studies on the account code limitations and the data conversion
since 1988. In fact, FMD's November 1993 impact study concluded
that 47 EPA software systems would be impacted by the new account .
code structure. Of these 47 systems, 9 were scheduled to be
replaced through the IFMS implementation (including FMS, ADCR, and
RMIS). Further, the study concluded that the remaining 38 systems
would be directly impacted by the change in the account code. The
study estimated that 20 systems would cost $415,000 to modify; and
did not include estimates for all the remaining systems. The study
also did not fully include estimated costs for data conversion.
However, the FMD officials estimated in 1992 that modification to
the account code structure and conversions of data in the systems
18
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 ZFMS
relying on the 10.digit account code structure could cost up to
several million dollars.
CostsTo EPA Programs Incurred Due To Lack Of Functionality Of
IFMS
Because of not fully implementing 6 of the 11 modules, EPA has not
maximized the benefits of IFMS which has had significant effects on
EPA programs. For instance, not implementing the project and cost
allocation accounting module/process has impaired the ability of
Agency managers to effectively manage the financial and budgetary
aspects of their programs or, in the case of Superfund, to assemble
cost recovery packages. In an October 13, 1992, letter to the AA,
OARM, the Director, FMD addressed the effects of not having a
project and cost allocation accounting process:
*
With respect to Superfund, costs are not tracked to the level
of detail needed for some projects, and manual efforts, and
alternative systems are required to provide that level of
detail. Cost recovery cases may be put at risk if this data is
not available. [Office of Research and Development] ORD's
managers are unable to manage the financial aspects of their
widely dispersed research projects as well as they would like
without project cost detail. We may simply be unable to
implement in any satisfactory way the Agency initiatives on
themes, certain CFO requirements or the WCF without this
capability.
In another case, because EPA implemented the accounts receivable
module with technical shortcomings, an independent accounting firm
could not determine the completeness of EPA's accounts receivable
balance for its fiscal 1993 Report of Financial Audit of the
Superfund Trust Fund, Leaking Underground Storage Tank (LUST) Trust
Fund, and Asbestos Loan Program.
Also, as of December 1993, EPA had no plans^to implement certain
other modules (e.g., fixed asset, project/cost allocation, and
payroll), resulting in reduced system functionality and not
achieving the original intended system objectives. The Agency was
in the process of determining the requirements and options for fixed
assets and project/cost allocation accounting, which included re-
examining the off-the-shelf software modules. Subsequent to our
fieldwork, the'Agency provided documentation to show that they are
investigating an enhanced project and cost allocation accounting
module and pursuing implementing an enhanced fixed assets module
being developed by another agency.
19
Report Mo. E1NMF3-15-0073-4100561
-------
EPA'a ZFM£I
System Is High Risk. Inefficient. And Does Not Meet
Reou i rement s
Ineffective implementation of IFMS has resulted in the following:
• Leoacv systems are costly and difficult to maintain. While the
average life of a financial system is 12 years9, PMS and RMIS
were developed over 20 years ago. These systems do not employ
modern data base technology, and major portions of the program
code are written in outdated languages, which make it
increasingly difficult to obtain technical support. Also, the
. technical support that is available is shrinking. As a result,
recovery would be extremely difficult and costly to promptly
fix these critical systems, and thus could result extensive
system operational downtime.
The Director, FMD, October 13, 1992, memorandum (cited above)
regarding these systems indicated that:
...technical support for FMS is becoming increasingly
tenuous because of the age of the system and decreasing
numbers of EPA or contractors employees familiar with its
operations. As a result, faith in the integrity of data
in FMS may be called into question. Further, if there
were a catastrophic failure to FMS, EPA would be unable to
access critical data for an indefinite period of time,
restoration would be difficult and costly and major EPA
functions could not be performed. Risk of a catastrophic
failure increases the longer we rely on the system.
Also, the "Integrated Financial Management System Strategy and
Master Work Plan" dated July 16, 1993, indicated that:
We have a system that consumes considerable resources -
drawing away from other Agency needs. We are still
operating redundant, "legacy" systems; we must invest in
tedious reconciliation across systems to ensure data
integrity; we have not taken full advantage of improved
computer technology to meet user needs;... and we face
challenges of funding and of management ... Continuing to
operate the historical systems in parallel with IFMS is
costly ..., prone to error..., not timely..., and
programmed in outdated computer languages....
9 He estimated a 12-year ayatem life for IFMS baaed on an Institute of Electrical and Electronics
Engineers (IEEE) publication entitled "Software Lifetime and its Evolution Process over Generations' (dated
1992) .
20
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
The Acting Comptroller announced projects in March 1993 to
eliminate duplicate legacy systems and to build capacity for an
expanded account code. While we are aware of the various
studies and work-groups involved in these areas since 1988, we
believe that formally establishing these projects is critical
for developing a single integrated financial management system.
Data integrity problems. Since 1989 EPA has been unable to
reconcile all payroll transactions from EPAYS, FMS, and CPARS
to IFMS. Headquarters Accounting Operations Branch (HAOB)
payroll officials indicated in 1993 that during the transfer of
data from each of the systems to another, some data is rejected
by each system and put in a reject file, while other data "just
disappears ..." Thus, data processed plus the rejected data
does not equal the original data that was input, or as one
official put it, "A plus B does not equal C." Additionally,
EPAYS, FMS, CPARS, and the related reject files all have
detailed level data; whereas, IFMS has summary level data.
HAOB officials could provide no consistent overall figure
regarding the positive and negative variances between the
systems.
Since IFMS has not been fully interfaced with some other
systems--PPAS, GICS, and CIS/APDS--EPA has duplicate data entry
which increases the risk of errors and omissions in the data
resulting in inconsistencies and inaccurate financial
information. For instance, EPA's Financial Management Status
Report And Five-Year Plan dated August 1993- indicated that the
reconciliations have been burdensome and inaccurate between
PPAS and IFMS. The 1993 FMFIA report identified the lack'of .
automated interfaces between IFMS and -key Agency administrative
systems as a material non-conformance area which causes
duplication of data entry. Internal studies have shown that
some property items in PPAS are not in IFMg and vice versa; and
that EPA is unable to track items from one system to another.
One of the reasons for these weaknesses is the absence of key
common data elements in the two systems. • The Director, FMD,
October 13, 1992, memorandum (cited above) regarding this
interface issue between systems indicated that: .
. . . Agency programmatic (e.g., CERCLIS)' and
administrative systems (e.g., PPAS, GICS) include data
elements for financial information. Without interfaces
with IFMS this data must be entered manually, and thus,
there is a significant chance for major differences with
the official Agency financial data in IFMS. In addition,
duplicate data entry is wasteful in terms of the Agency
administrative resources needed to perform the
keypunching. . . Where interfaces are not available, the
21
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
value of the financial data in those systems for program
managers and administrative staff is impaired, and there
is a risk that inaccurate financial data would be reported
and purported to be "official" financial information.
These potential data discrepancies could cause Congress,
OMB, and the public to lose confidence in EPA reports.
IFMS has not kept up with new technology and changes in Federal
accounting policies and standards. Without being able to
implement timely software upgrades and maintenance changes, EPA
has not kept up with the JFMIP requirements. For,example,
JFMIP requires that the accounts receivable function include
provisions to calculate interest, penalties, and overhead on
overdue receivables, which was not performed under IFMS
software version 3.0. In another example, JFMIP requires the
aging of receivables, but IFMS does not have an acceptable
means for automatically setting up or aging installment
receivables or calculating compound interest under the current
software version.
Additionally, OMB Circular A-127 requires an integrated
financial management system, and the revision to the Circular
clarifies this definition to include common data elements.
Without an adequately implemented data dictionary, EPA can not
comply with these requirements. Also, without fully defining
common data elements in its data dictionary, EPA cannot
effectively deal with data redundancy or enhance the sharing of
data among different systems, JFMIP requires that an account
code classification structure accommodate fund, program,
organization, project, and object classification structures.
EPA's 10 digit account code structure does not meet this
requirement.
Off-the-shelf software benefits reduced. The FFS software has
been customized to the extent that its benefits (i.e., minimal
development costs and ease of implementation) have been
significantly reduced, and the ability to maintain IFMS in a
timely and cost effective, manner is in question. For example,
EPA has used significant resources over several years to
upgrade to version 4.0 and S.le of FFS because EPA had to
reestablish its baseline and do significant retrofitting.
IFMS off-the-shelf software has been very difficult and time
consuming to implement. EPA's target date for completing IFMS
modules is September 1995, which is about 6.5 years after the
system's planned implementation. In marked contrast,. AMS
officials indicated that other agencies (e.g., IRS, VA, and
SEC) have implemented AMS' off-the-shelf software in 18 months
or less.
22
Report No. E1NMF3-15-0073-410056!
-------
EPA'8 IFMS
• IFMSmanagerscannot make informed decisions or adequately
prepare contractworkstatements. Without periodically
updating and monitoring requirements, and performing cost-
benefit analysis,* management does not have complete information
to make informed decisions and prepare IFMS related contracts.
Updating these documents will provide information regarding
changes in user needs, requirements, developmental approach and
alternatives, and cost benefits which may seriously impact the
success of IFMS. Further, without updating these documents,
management has no assurance that all the requirements .have been
met, and that those that are being addressed are the most
critical. By updating these .documents, EPA will have a
mechanism to measure whether the system is meeting its'
original objectives and user needs. Subsequent to our
fieldwork, EPA completed a requirements analysis and a cost
study for IFMS. . . »
Moreover, updating requirements allows EPA to more effectively
prepare IFMS contract Statements of Work (SOW) and
specifications. During the audit, we reviewed contracts' SOW,
Delivery Orders, work assignments, and modifications on five
contracts related to the core financial system and interfaces.
We found SOWs under many of the tasks were not clearly defined,
and in some cases, were not defined at all. For instance, one
of the task orders for IFMS stated that over 40 operational
areas required software fixes and modification. None of these
operational areas were clearly defined, 22 operational areas
were listed, and the remaining 18+ areas were not identified.
{Task 4.3, GSA Financial Management Systems Software (FMSS)
Multiple Award Schedule (MAS) Contract GSOOK92AFD2500, DO 26-
0476-NBLX). Additionally, CMS work orders for software
modifications or enhancements do not provide a crosswalk (audit
trail) to the contract's SOW task orders. GSA MAS contracts
require that requirements be firmly established and clearly
defined prior to awarding contracts and subsequent delivery
orders for software purchases, software maintenance, training,
documentation, and technical assistance activities. Subsequent
to 'our review, a management official said the new CMS
implemented in May 1994 provided a crosswalk to tasks in the
SOWs.
Financial Management Environment Is Verv Complex And Unwieldy
By not fully implementing IFMS, EPA has created a very complex and
unwieldy financial management environment. From a system
configuration viewpoint, the current IFMS environment is more
complex than what was originally envisioned in the 1988
implementation plan. For example, complex manual reconciliations
are required where automated interfaces are not in place. Also,
maintaining the legacy accounting and financial systems which
23
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
interface with IFMS increases the complexity of the total system
design and makes the management process cumbersome. For example,
IFMS uses MARS, Reporter, Special Package for Unique Reports (SPUR),
and Budget and Accounting Reporting System (BARS) for financial
reporting instead of one system. These legacy systems also require
additional alternative control mechanisms, such as reconciliations
and additional automated edits to verify the integrity of data
within IFMS. Further, the development of custom modules and the
development of MARS to meet EPA's environment has been costly,
complex, and difficult to manage. - In our opinion, this has also led
to additional retrofitting problems resulting in not being able to
upgrade from 3.0 to 4.0 and 5.1e in a timely manner.
This complex and unwieldy environment, coupled with the current
schedule for developing and operating IFMS, may result in
diminishing returns on IFMS functionality. We estimate that IFMS
system life will end in the year 200110. By 1995, EPA's IFMS will
have exceeded about half of its estimated life. Target dates to
satisfy remaining functions are as follows: implement critical
modules (one in March 1995 and two to be determined); eliminate
legacy systems (September 1995); implement additional interface -
(September 1995); and solve other IFMS FMFIA weaknesses (September
1995). Until full implementation of IFMS, the return on the planned
IFMS investment over the remaining life of the system will be
limited. Moreover, we are particularly concerned that the Agency
may not successfully eliminate the legacy systems by September 1995
because of the dependency on the systems and the magnitude of the
projects. Subsequent to our review, the Agency implemented IFMS
version 5.1e which upgraded the accounts receivable module, but
acknowledged that further changes to the accounts receivable module
need to be made. Also, a plan for a enhanced fixed asset module to
be implemented by July 1996 was approved by the Deputy AA for OARM.
LEADERSHIP. ACCOUNTABILITY. PLANNING. AND POLICIES
NEED TO BE IMPROVED
Top Management Leadership And Direction Needs To Continue
Over the past year, we noted that top management involvement with
and oversight over IFMS has increased substantially. Prior to that,
top management officials were selectively leading and directing IFMS
activities, rather than managing IFMS on a system life cycle
approach and on a set of priorities based on a cost-benefit
While we are aware that upgrades are available to IFMS which should result in a longer system
life, we based our estimated system life on a survey published by IEEE of 95 systems (see Appendix IX). Me also
are aware that AMS is developing a client-server based system (i.e.. System 2000) which should replace FFS in
the future. Additionally, the 12-year system life is conservative since FIPS Publication 106 and contractor
studies related to IFMS have estimated system lives between 5-7 years.
24
Report No. E1NMF3-15-0073-4100561J
-------
EPA'a IFMS
analysis. Additionally, top management did not used the committees
effectively as a vehicle to provide leadership and direction on IFMS
and longstanding financial management issues. Continuation of the
current level of leadership and direction is critical to ensure the
successful implementation of IFMS.
One of the most critical factors in determining the success or
failure of a major system initiative is the degree of top
management11 support- Only with top management support can a major
system become an accepted, integral part of the organization. GAO,
in their report entitled "Meeting the Government's Technology
Challenge" dated February 1990, discussed the importance of top
management direction and leadership for successful automation
efforts. The report indicated that without clear direction and
support from the top, modernization initiatives tend to degenerate
into loose collections of independent systems. Moreover, the CFO
Act of 1990 re-emphasized and required a sound leadership structure
and consolidated responsibility to ensure that one person, who is
part of top management, has overall responsibility for establishing
and implementing effective financial management policies, internal
•controls, and financial management systems. Along these lines, it
is essential that top management be involved in planning systems,
allocating resources, making key decisions, establishing priorities,
and reviewing critical system activities throughout the system life
cycle. OMB's 1991 guidance further emphasizes that the CFO should be
a' full participant in Agency IRM decisions.
The AA, OARM has been designated by the Administrator as the CFO.
In this capacity, this official is responsible for serving as EPA's
CFO in accordance with the CFO Act of 1990. The CFO is responsible
for overseeing all financial management activities relating to the
programs and operations of EPA and for managing the Agency's
financial systems in accordance with applicable laws and
regulations. Among other activities, this official has submitted a
CFO Five Year Plan; initiated actions to resolve longstanding FMFIA-
reported weaknesses; established a formal IFMS Charter; participated
in yearly IFMS budget decisions; concurred on the decision to
implement a new account code structure; and demonstrated awareness
of chronic IFMS financial management weaknesses in the Agency.
However, although various high-level planning documents were
prepared and certain actions with milestones initiated from year to
year, the critical longstanding financial issues presented will
remain uncorrected until stronger and more decisive actions are
taken by top management, to1 follow through on IFMS implementation and
integration.
Me define "top management' as the Deputy Assistant Administrator level and above.
25
Report NO. E1NMF3-15-0073-4100561
-------
EPA'B IFMS
The CFO has been .appointed as the DSO for IRM for the Agency, who is
responsible for chairing the Executive Steering Committee for IRM.
The IRM Steering .Committee is an accepted method for top management
to provide leadership and direction to ensure efficient and
effective use of information resources. However, based on a review
of the IRM Steering Committee minutes and documents of meetings
prior to May 1994, we identified that IFMS has not been a topic of
discussion since 1987, We have reported similar deficiencies
regarding this Committee's involvement in the IRM program, in our
September 1992 report on longstanding IRM problems. (Report No.
E1NMF1-15-0032-2100641). '
The DAA for OARM was not involved in major decisions regarding IFMS
until in August 1991 when he officially approved the creation of the
IFMS project management functions. In July 1992, the DAA for
Finance and Acquisition was created and designated DCFO for the
Agency. This official is responsible for establishing and
maintaining an Agency wide integrated accounting and financial
management system. From 1989 to 1991, we found one major decision,
which was made by the Comptroller. In 1990 the Comptroller deferred
major development, and agreed to focus on efforts to stabilize the
operations of the software and develop MARS.
We examined eight IFMS system decision papers since 1991. Five of
the eight were major decision papers and four received concurrence
at the DAA level or above. Additionally, in 1993, the DAA for
Finance and Acquisition was briefed on the status of issues left .
over from system development (e.g., the elimination of the
predecessor systems and the account code structure issue);
subsequently, one formal decision paper was prepared by the
Comptroller to establish priorities or establish responsibilities
for implementation. Since December 1993 subsequent to our
fieldwork, the top management officials have made several key
decisions. 'These included decision memorandums (1) on the direction
of IFMS over the next 7 years; (2) to adopt and implement -a fixed
assets subsystem; (3) approving a revised IFMS Charter and (4) a
decision to eliminate ADCR a predecessor system. Although, the
revised IFMS charter did address many of our concerns, it did not
address our concerns that the chairmen of the SMG and EMG be in the
CFO chain-of-command per the delegation.
The ASC, established in 1990, developed a draft charter which
includes responsibilities to advise the AA on policy, priorities,
and plans for major administrative systems. The charter also
required that this Council provide oversight for the implementation
of IFMS. The ASC meeting minutes dating back to 1990 showed that
the DCFO never attended a Council meeting, and that the former DAA
(who was the designated chairperson) had not attended a meeting
since November 1990. Although the ASC was to meet monthly, since
1990 the Council only met eight times. Our review of the meeting
26
Report No. E1NMF3-15-0073-4100561J
-------
EPA's IFMS
minutes showed that IFMS was discussed; however, no recommended
actions or decisions were documented by this Council regarding the.
FMFIA problems and implementation of IFMS. Furthermore, subsequent
to our fieldwork, the revised IFMS Charter omits all references to
the ASC and establishes oversight responsibilities to the EMG, which
is a lower level group (e.g.. Division Directors and below).
The Comptroller has been designated by the CFO as responsible for
establishing and maintaining an Agency wide integrated accounting
and financial management system. Our discussions with the Acting
Comptroller in June 1993 .indicated involvement in: developing IFMS
high-level plans; reviewing the annual budget documents for IFMS;
and coordinating these documents with the CFO. This official told
us that he was involved in decision-making for MARS and operational
software fixes based on available resources. Also, the Acting
Comptroller initiated actions in May 1993 to address some issues
left over from system development (e.g., elimination of legacy
systems) which have yet to be resolved. While this official
indicated that he was aware of the tradeoffs in their approach, a
comprehensive plan and/or an updated cost-benefit study was not used
to support the decisions.
Fragmentation And Unclear Lines Of Authority
On September 14, 1990, EPA revised its management approach towards
IFMS by introducing an IFMS Charter which established roles and
responsibilities for the IFMS Project Manager, EMG, SMG, and four
.OARM divisions that work on IFMS. The charter did not: (1) clearly
outline the decision-making authority of the IFMS project staff; (2)
reflect the lines of authority as outlined in the CFO delegation;
and (3) clearly define individual line authority and decision-making
power among the four organizations or the groups involved in the
operation and maintenance of IFMS.
• Our March 1991 audit report recommended that the AA for OARM
formally establish an independent organization with authority,
responsibilities, and resources to successfully complete the
objectives of the IFMS implementation and that this
organization directly report to senior management. During our
review, OARM established an IFMS Charter which set forth that
the Director, IFMS Project Management Staff served as the
principal point of contact for Agency senior management on .IFMS
and was responsible for monitoring and evaluating the progress
of IFMS. This official reported directly to the Comptroller.
Major activities of this IFMS staff included: preparing a draft
implementation plan; managing IFMS through the budget process;
recommending approvals of software sub-releases; and
coordinating with users. However, this staff did not assure
that key IFMS milestones were met; essential requirements were
met; and needed resources were obtained to satisfy the 1988
27
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
IFMS implementation plan. Moreover, a proposal to formalize
the IFMS staff showed that the staff's primary function was to
aid in the management of the system through a coordination
role. OMB's September 1993 "Report on Financial Systems
Planning Process" for EPA also recognized that responsibility
for financial systems was fragmented among several
organizational entities and that the Director, IFMS Project
Management Staff did not have line authority over the project
working groups. Thus, OMB concluded that no clearly
established office or individual was accountable for ensuring
that decisions from the planning process were effectively
carried out. The report recommended that EPA establish clearer
accountability for financial systems development and
implementation.
• T
While the September 14, 1990, IFMS Charter identified roles and
certain responsibilities, it did not clearly state the
individual line authority and responsibilities as delegated
under the CFO Act, nor did the IFMS Charter explicitly state
the decision-making power vested in specific individuals. For
instance, the IFMS Charter did not address the CFO, DCFO, or
IRM Steering Committee and the relationship to the other
management groups and staff outlined in the IFMS Charter. The
1990 IFMS Charter's lines of authority and decision-making
roles were unclear. For instance, the charter provided that
the EMG was chaired by the IFMS Project Manager. This official
had limited authority in practice over the EMG members because
the recognized12 CFO designate is the Director, FMD, who was
one of the group members. In our opinion, the IFMS Charter
should reflect the Comptroller as the EMG chairperson, because
this would be consistent with the CFO delegation and promote a
hierarchical structure for decision-making. Similarly, the
charter provided that the SMG was chaired by the Chief,
Financial Systems Branch (FSB). This official had equivalent
organizational status to the other members of the SMG, and thus
did not have the requisite authority over the members. In our
view, the SMG chair should be the Director, FMD who was the
recognized official under the CFO delegation. The Director of
FMD should establish decision items, a voting process, and a
method to escalate items to upper management in cases of '
disagreement. Moreover, the IFMS Charter should explicitly
define the decision-making powers vested in' the individuals, as
well as the roles of the supporting IRM Steering Committee,
ASC, EMG, and SMG. Subsequent to our fieldwork, the IFMS
12 The Delegation 1-16 dated October 1992 designates the CFO authorities from the Administrator to the
AA for Administration and Resources Management who has redelegated some of the responsibilities regarding
financial management functions through the Deputy CPO to the Comptroller. Although the delegation recognizes
that these functions "may be redelegated* to the Director, FMD, it has not been formalized.
28
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
Charter (May 1994) was revised to reflect new roles and
responsibilities of .individuals and organizations.
IFMS Decisions Made Without An Adequate Plan Based On A System
Life Cycle Approach
In 1987-88, EPA planned a 3-phased approach to implement IFMS.
Major initiatives under this approach included implementing FFS
modules; eliminating legacy accounting and financial systems;
implementing a project and cost allocation accounting module;
modifying the account code structure; and interfacing with other
systems. Because of. significant operating problems in implementing
the FFS modules and data conversion problems, the original
implementation plan was abandoned. Instead, various parts of the
system were to be implemented individually.
However, since the time of the original 1988 implementation plan,
EPA had not issued a comprehensive strategy and master work plan for
over five years to complete the system implementation effort. In
July 1993, the OC issued the "Strategy and Master Work Plan." This
plan has conceptually sound goals (e.g., deals with integration and
elimination of legacy system issues), but does not match resources
(dollars, staff, and contracts) with tasks; has a limited priority
structure; is not sufficiently detailed; and is not specific
regarding the IFMS direction and user needs over the next five
years. For instance, many of the requirements (e.g., project and
cost allocation accounting, fixed assets, and account code
structure) in the original, plan have not been defined to a
sufficient level to realistically estimate cost and target
completion date. Further, while the plan lists ongoing projects, it
does not prioritize these projects in a comprehensive fashion to
show the interdependence of activities (i.e., activities that can be
carried out in parallel and what must be done in sequence). This is
needed to realistically estimate target completion dates and
minimize the risk and cost of the projects. Thus, in our view, the
plan as written is essentially a "wish list" and provides no
assurances that it can be fully implemented in the time frames
indicated. The September 1993 OMB report indicated that EPA needs
to continue to focus adequate personnel and financial resources
towards integrating and enhancing its financial management system
beyond the conversion to IFMS version 5.1. The report stated that
EPA's strategic planning has not been timely nor has it driven the
budget process, identify realistic, achievable goals, and the
necessary action plans to achieve these goals.
The absence of a comprehensive plan has resulted in IFMS being
implemented on a piecemeal basis through the budget process (i.e.
primarily the SMG IFMS spending plan). This process is short term
and only provides detailed costs for the current year, rather than a
structured, well thought-out, SDLC approach over an estimated system
29
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
life. ' The Acting" Director of FMD indicated that the budget.
decisions on IFMS are made on a yearly basis in the context of
resources available, and are not always documented or are only
minimally documented. Documentation provided by FMD indicated that
top management (i.e., the CFO -and DCFO) were aware of chronic system
problems left over from initial IFMS system development. However,
these documents did not provide sufficient and objective rationale
to support the decisions. Thus, we are concerned that decisions are
being made without considering cost effectiveness and efficiency.
(See Chapter 3 on IFMS life cycle costing.)
Further, based on discussions with management and a review of the
EMG and SMG meeting minutes, we concluded that management has
largely focused on upgrades to the software package and MARS, at the
expense of critical activities which were not previously performed
in the early stages of development. Without a formally structured
system development approach and timely comprehensive plan,
management has no assurance that the decisions, tasks and work
assignments are cost-effective, prioritized logically, and in the
best interest of EPA financial management systems.
System Development Not Following A Generally Accepted System
Development Life Cycle Approach
System development was not accomplished through a structured process
and one standard system development methodology was not used
throughout the IFMS life cycle. Critical system documents were not
prepared, updated, or used, such as a comprehensive data dictionary
and cost-benefit analysis. EPA's unique requirements as well as
other requirements were not well-defined. Testing of the system was
not adequate prior to system implementation in 1989. Conversion of
data from FMS to IFMS was incomplete upon system implementation in
1989. The Director, IFMS Project Management Staff indicated that
EPA did riot successfully implement IFMS Phase 1 activities and did
not keep up with requirements because they failed in 1989 to do
proper system testing. This official also indicated that for two
years (1990-91) they "shut down" (i.e., did not proceed forward)
because they were still working on Phase 1. Moreover, Director
indicated that budget cuts slowed progress in fiscal 1992.
One comprehensive system development methodology was not used
throughout IFMS. For example, we identified 11 different system
development methodologies (EPA and contractor-developed) cited in
contracts' SOWs or contractor deliverables. To illustrate, one
delivery order specified four different methodologies under the GSA
AMS contract. (GSA Contract GSOOK92AFD2500, DO 26-0476-NBLX).
Further, we found that these methodologies were not always stated in
each of the IFMS contracts and delivery orders. While we did not
attempt to determine the full effects of using these many different
methodologies, we believe that the lack of a comprehensive and
30
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
standard methodology was a contributing factor to ineffective IFMS
implementation. Had one comprehensive methodology been employed and
followed, a more structured approach would have been ensured for
solving EPA's financial management system needs. Although EPA
issued its SDLC guidance in April 1993 as a temporary EPA directive,
it has yet to update this directive to establish minimum mandatory
requirements and address more current software tools and techniques.
EPA Over-Customized The Off-The-Shelf Package
Since IFMS implementation, EPA has essentially layered the IFMS
systems over the old processes and customized the modules, in part,
to preserve the old financial system processes. This has
essentially led to the over-customization of the software and the
need to re-establish a baseline13 system. EPA needs to examine how
the organization carries out its financial management process and
what they expect from the financial management system. By doing
that, EPA will determine the opportunities to implement a more
standard system, but also opportunities to streamline the
organization and eliminate inefficiencies based on old technology
and ways of doing business. OMB's September 1993 report also •
addressed this issue of extensive customization of FFS off-the-shelf
software; and how it continues to hinder financial system
initiatives.
We interviewed AMS officials to determine the degree of
customization in terms of lines of code of the existing EPA IFMS
application compared to the original off-the-shelf version 3.0. One
of these officials indicated that this precise a comparison could -
not be performed because FFS version 3.0 is no longer available in
the AMS baseline libraries. However, she indicated that EPA's
implemented version of 3.0 was 50 percent greater in terms of lines
of code then the off-the-shelf package version 3.0. She further
indicated that this percentage of lines of code appeared reasonable
for providing insight into the degree of customization. This
strategy to customize when IFMS was first implemented was consistent
with EPA's managements philosophy. For example, AMS officials
stated that EPA management wanted IFMS processes to be similar to
the Agency's operating process at the time.
An AMS official said that EPA had changed its philosophy since the
time of IFMS implementation and have moved towards t-he generic FFS
baseline version. For example, an AMS official told us that EPA's
planned implemented version of 5.1e will be 29 percent greater in
13 A baseline is a software configuration management concept defined aa software which i» used as
a basis for future maintenance changes and enhancements. The baseline helps ensure that only authorized changes
are made and establishes a foundation for configuration and change management.
31
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
terms of lines of code" (customization) then the off-the-shelf
package 5.1 version. This percentage of change is significantly
less than .the changes to the version of 3.0, but is nonetheless
still excessive.
We discussed with AMS senior management officials the experience of
other agencies in implementing the FFS off-the-shelf software.
Generally, the smaller agencies implemented the off-the-shelf
software package in approximately 6-8 months. The AMS officials
attributed the success of these agencies to implementing "vanilla
packages" with little modification. AMS officials indicated that in
larger agencies (such as IRS. and the VA), FFS software was
successfully implemented in 12 to 18 months. They added that IRS is
comparable to EPA's accounting system in terms of complexity of
operations.
In another case, the SEC, which is the only other Federal agency to
implement the ADABAS version of AMS' FFS software, has successfully
implemented FFS, eliminated its predecessor accounting system within
8 months, and upgraded new versions of the software quickly and at
little cost. We recognize that the SEC is a much smaller
organization and has a much smaller operating budget. However, the
SEC approach to implementing the commercial off-the-shelf software
package maximized the advantages of the package at least risk.
Based on interviews with the SEC's Office.of the Comptroller's Chief
for Fiscal Operations, we determined that the SEC management made a
concerted effort to implement the FFS commercial software with only
a limited number of changes and interfaces. Essentially, we were
told that the SEC implemented a "vanilla package" to maximize the
benefits of the off-the-shelf software and to minimize the costs of
software conversion for the acquired system, future upgrades, and
software maintenance. This official indicated that the SEC adopted
the new system processes, rather than retrofitting the new system to
the predecessor system process. Since implementation, the SEC has
successfully implemented Version 5.01.
According to Capers Jones, a leading authority on software quality
management:
...Package modification is a high-risk activity with a
significant chance of failure and disaster. Not only is
productivity normally low for modifying purchased packages, but
if the vendors are reluctant to provide support, modification
can easily be more expensive than custom development. As a
rule of thumb, package modification is feasible only if the
An AMS official indicated that this percentage determination waa based on a comparison of the FFS
baseline release S.I, which consists of 866,7SO lines of code, with EPA'o planned IFMS S.le release, which
consists of 1,216.069 lines of code.
32
Report No. E1NMF3-15-0073-4100561I
-------
EPA's IFMS
vendor is cooperative and supportive and if the package is well
structured and well documented. Even then, changes that equal
more than about 15 percent of the volume of the original
software should be avoided.
In our view, EPA should minimize the customization of the off-the-
shelf package and reassess their direction to take full advantage of
the available technology at the least overall cost over the life-
cycle of the system. As a consequence of not implementing the off-
the-shelf software with minimal changes, EPA did-not upgrade from
3.0 to the more current software versions 4.0 and 5.1e in a timely
manner.
RECOMMENDATIONS
We recommend that the AA for OARM:
1. Continue to:
a. Reassess and periodically update the requirements, plans,
goals, alternative solutions, costs and benefits of the
remaining IFMS phased implementation and determine whether
the existing approach (i.e., system implementation and
software management activities) should continue or be
revised. The analysis of alternative solutions should
evaluate IFMS and other alternatives against prescribed
OMB requirements and EPA's financial management (user)
needs as well as involve a comparison of costs, risks, and
scheduled completion dates. Provide a decision paper
supported by this reassessment to the CFO and the
Executive Steering Committee for IRM for approval.
b. Prepare decision papers with appropriate approvals for
major IFMS projects.
c. Eliminate the legacy systems according to scheduled
milestone dates.
d. Update the IFMS Strategy and Master Work Plan once the
reassessment is completed and approved. At a minimum,
this plan should also match resources with tasks, contain
priorities and realistic target dates, and be sufficiently
detailed to identify responsible action officials. Update
and revise this plan at least on annually throughout the
remaining system life cycle.
e. Update and require one generally accepted systems
development life cycle methodology to be followed by EPA
and contractors for IFMS. This methodology should require
33
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFHS
a structured approach to system development and systematic
framework for management control. This methodology should
be referenced in all applicable EPA IRM contracts, require
minimum standard documentation (e.g., data dictionary,
cost-benefit analysis, requirements analysis, project
plan), and be prepared and updated throughout the life of
the system.
f. limit the extent of customization to the off-the-shelf
software modules to the maximum extent-possible and
examine ways to change and re-engineer how EPA does
business to move towards a more standard financial system.
Require increased involvement by the Executive Steering
Committee for IRM in recommending and approving actions for
IFMS decisions commensurate with their defined roles and
responsibilities. This increased involvement should be
formalized and include intermediate reviews when planned cost
or time thresholds are exceeded or performance criteria are not
met.
Revise the IFMS Charter to reflect the IFMS system of
accountability, including the delegation of authority under the
CFO; and the line of authority among individuals, staffs,
committees, and management control groups. Formalize the
operational role of the FMD Director under EPA Delegation 1-16.
Utilize project management and scheduling techniques (program
evaluation and review techniques or critical path method) in
managing IFMS on an overall basis to aid in planning and
controlling interrelated activities and establishing realistic
milestones.
Incorporate the appropriate ARTS functions into CMS. Re-
examine the outstanding work requests and eliminate any that
are no longer necessary.
Perform an off-the-shelf software inventory; document the
modules purchased, installed,and implemented; and capitalize
costs over $5,000. Stop paying maintenance on modules which
will not be implemented. Consult with General Counsel about
the possibility of recouping original costs and maintenance
fees from AMS for modules purchased, but not implemented.
AGENCY COMMENTS AMP PIG EVALUATION
Based on our review of the AA for OARM's June 15, 1994, response to
our draft report and other supplementary documents, OARM agreed with
six out of seven of our recommendations. However, OARM did not
34
Report No. E1NMP3-15-0073-4100561
-------
EPA'8 XFKS
adequately address our concerns in two recommendations.
Additionally, OARM disagreed with one recommendation. In addition,
OARM has completed action on one recommendation and initiated action
on four recommendations.
Although OARM has agreed with Recommendation I.e., the corrective
actions do not address our concern that IFMS contractors were not
following the Agency's official SDLC methodology. We are primarily
concerned about the preparation and periodic updating of critical
system documentation, such as a comprehensive data dictionary and
functional requirements analysis. The intent of bur recommendation
is that the Agency establish procedures to ensure that all future
delivery orders under the GSA AMS contract for IFMS meet the
requirements of Chapter 17 of EPA Directive 2100.
OARM agreed with recommendation 4, but the corrective actions did
not address our concern that a more comprehensive project planning
process is needed. The implementation of version 5.1e is an example
of detailed project planning. The existing Strategy and Master Work
Plan is not detailed enough to estimate project costs or establish
target dates. With several major projects being performed
concurrently, detailed planning is crucial to minimizing costs,
establishing realistic target dates, and" maintaining projects on
schedule. For example, the project to establish an expanded account
code structure is currently scheduled for completion in October
1994. However, this target date is unrealistic because the project
cannot be fully implemented until after the elimination of the
legacy systems. Without a structured SDLC approach and a
comprehensive plan, the plan is not realistic. In addition, the
current approach does not ensure projects will avoid project
resource contention or identify critical project interdependences.
OARM disagreed with our draft report recommendation to revise the
position descriptions and performance standards of key officials to
reflect duties in the IFMS charter and incorporate performance
standards for IFMS. The OARM response stated that "Key IFMS
managers are well aware of their responsibilities for IFMS, and
their performance standards reflect those responsibilities and
desired outcomes at the level of detail deemed appropriate for their
positions and levels of responsibility...." We recognize that it is
difficult to determine what is the appropriate level of detail in
position description and performance standards and that our
recommendation may have been too prescriptive. In addition, we
noted other compensating controls (e.g., the updated SDLC
requirements in Chapter 17 of EPA Directive 2100 and the revised
IFMS Charter). Therefore, we eliminated that recommendation.
35
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
CHAPTER 3
LIFE CYCLE COSTS NOT USED TO MANAGE IFMS
EPA has initiated efforts to create a WCF, so that it can more cost
effectively administer services, including ADP and telecom-
munications services. The Agency is also investigating a project
and cost allocation accounting module modified by. another agency;
this will be coordinated with the WCF initiative.
However, EPA is not developing, reviewing, and updating costs
throughout the IFMS system life cycle to effectively control the
development and operation of the system. As a result, EPA
management was not in a position to make informed- system and budget
decisions regarding the design, development, operation, and
maintenance of the system with an estimated life cycle cost of
approximately $202 million over 12 years (see Appendix VII),1S EPA
deficiencies in tracking life cycle costs for IFMS are primarily due
to: (1) a need for a comprehensive process/system to accumulate
costs; and (2) unclear Federal and EPA policies and guidance on life"
cycle costing.
SYSTEM LIFE CYCLE COSTS CRITERIA
The Paperwork Reduction Act of 1980 (as amended) establishes a broad
mandate for agencies to perform their information management in an
efficient, effective and economical manner. Pursuant to the Act,
the April 1976 OMB Circular A-109, entitled "Major System
Acquisitions", provides policies to ensure the effectiveness and
efficiency of the process of acquiring major systems. Specifically,
it requires each agency to: (1) maintain the capability to predict,
review, assess, negotiate, and monitor costs for the system on a
life cycle basis; (2) make new assessments where significant costs,
schedules, or performance variances occur; and (3) estimate life
cycle costs16 from system design through production to ensure
appropriate trade-offs among costs, schedules, and performance.
The CFO Act of 1990 requires OMB to submit a 5-year financial
management plan which includes cost estimates for implementing the
plan. The Act further requires agency CFO's to prepare and annually
Our analysis concentrated on gathering life cycle coat figures for IFMS over an estimated system
life. However, the figures also include estimated eosta of the other core financial systems (i.e., MARS, FMS,
ADCR, RMIS, CPARS) incurred within the IFMS estimated system life.
16OMB circular A-109 defines life cycle cost as the Bum total of the direct, indirect, recurring,
nonrecurring, and other related costs incurred, or estimated to be incurred, in the design, development,
production, operation, maintenance and support of a major system over its anticipated useful life span.
36
Report No. E1NMP3-15-0073-4100561
-------
EPA'8 ZFHS
update, revised agency plans to implement OMB's 5-year plan. In
implementing this act, the 1993 OMB Circular A-127, entitled
"Financial Management Systems11, requires that financial management
systems development and implementation efforts seek cost effective
and efficient solutions; and develop annual financial management
system plans in conjunction with OMB's 5-year financial management
plan. These plans are to include detailed information on costs and
milestone dates to support management decisions, project monitoring,
and budget requests. These comprehensive plans should enable top
managers to oversee and review major financial management, systems
development efforts and operations..
The OMB Circular A-11, entitled "Preparation and Submission of
Budget Estimates" (dated August 1993), requires reporting of major
information.system initiatives which will require obligations that
exceed $25 million over a system's life cycle or "$10 million in any
one fiscal year. It also requires reporting on financial management
systems, which include all core financial systems, financial and
mixed systems critical to effective agency-wide financial
management, reporting, and control. In addition, it requires
•reporting any financial and mixed systems appearing on the high risk.
list in the most recent President's budget. Agencies that obligate
more than $2 million in a year, must also prepare a report of
obligations for systems activities including telecommunications,
planning, cost-benefit, installation, operations, maintenance and
support. Further, it requires treatment of system and application
software that exceeds $25,000 as a capital investment.
The December 1985 OMB Circular A-130 requires that agencies acquire
information technology in a competitive manner that minimizes total
life cycle costs. As part of their planning, agencies need to
consider the full information system life cycle when considering the
cost of information technology. In addition, OMB's draft September
1993 update to Circular A-130 requires agencies to track life cycle
costs to aid in measuring system performance, and focus on. whether
projects remain within cost estimates; are on time; and meet the
promised objectives. Moreover, the Government Performance and
Results Act of 1993 further supports the need for performance
measurements to improve program efficiency, effectiveness,
accountability, and decision-making. FIPS Publications 38 (February
1976), 64 (August 1979), 101 (June 1983), 105 (June 1984), and 106
(June 1984) also provide details on the system life'cycle and life
cycle costing.
EPA directives and guidance support life cycle costing for major
information systems. Chapter 2 of EPA's 1987 Directive 2100
entitled "IRM Policy Manual" requires that planning for significant
investment in, and the management of, information must be supported
by analyses of the life cycle of the information requirements from
the initial stages of information system design through operational
37
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
stages of system start-up and maintenance. Chapter 4 of this manual
on software management policy requires that EPA program officials
managing the development or ongoing operations of software.
applications are responsible for the management of life cycle costs.
One of the objectives of this policy is to improve the cost
effective acquisition, development, maintenance, and ongoing
operations of software systems. In August 1994, OIRM also revised
Chapter 17 of the IRM Policy Manual which establishes the management
principles and requirements that govern the life cycle and life
cycle costs thresholds of Agency major information systems.
Information system planning policy initiatives (i.e., revisions to
Chapter 2 of the IRM Policy Manual) have also been started which
would require use of this life cycle cost information.
EPA has issued guidance, entitled "EPA System Design Development
Guidance" (dated June 1989), which requires that a life cycle
benefit-cost analysis be performed which includes, among other
categories, cost for development, maintenance, personnel, timeshare
and telecommunications. This guidance and the "Operations and
Maintenance Manual" (dated April 1990) require that the benefit-cost
analysis based on a system life cycle approach be updated as the
system progresses and for proposed system modifications. These
documents were formally issued as temporary EPA directives in April
1993.
LIFE CYCLE COSTS NOT USED THROUGHOUT IFMS
The original Arthur Young Feasibility Study for IFMS included a
cost-benefit analysis completed in 1985 comparing total costs for
each of the alternatives employing a 5-year life cycle costing
methodology. The study estimated that IFMS would cost a total of
$61 million over a 5-year useful life and recommended, as an
alternative, purchasing the AMS off-the-shelf software package—FFS.
The supporting cost figures used in this study were updated in 1986,
and resulted in no material changes.
Since 1986, the OC, OIRM, and subsequently the CFO have made
critical system decisions with limited cost information (i.e., IFMS
dedicated budgets in 2 offices over a 3-year period) rather than
based on total life cycle costs over an established system life.
For example, the Agency is reporting on costs incurred or to be
incurred over a 3-year period . (i.e., fiscal 1991-1993) which
averages $6.5 million a year. However, these annual system costs do
not reflect NDPD annual costs (approximately $5 million for
timeshare and $4 million for telecommunications) related to IFMS as
38
Report No. E1NMF3-15-0073-4100561
-------
SPA'S IFMS
well as other Headquarters, regional, and other nationwide costs17,
which we were unable to estimate. Based on the cost information
available, we estimated that the total life cycle cost of the system
to be approximately $202 million over a estimated life of 12 years
for an average cost of about $16 million a year (see Appendix VII).
IMPACT OF NOT USING LIFE CYCLE COSTING
Full Cost Information Not Available For Decision-making
Without employing life cycle costing, top management and steering
committee/management groups do not have critical information needed
to make informed, effective decisions. For instance, since 1990,
EPA's Financial Management Status Report And 5-Year Plans have not
included cost information which would aid top management in making
short- and long-term decisions on its financial systems and in
proactively managing IFMS. While we acknowledge that the Director, ,
IFMS Project Management Staff prepared an IFMS SMG spending plan,
that plan only covers one year and the funds are held at a constant
level. By using this approach, however, EPA had no assurance the
heeded resources will be available to fully implement IFMS in a cost
effective and timely manner.
Other project management decisions, such as when systems exceed
certain dollar thresholds (as prescribed in OMB Circular A-11 and
Chapter 17 of "the IRM Manual), are not triggered because EPA has not
accumulated total system costs. Consequently, top management does
not have a comprehensive way to monitor actual costs against
estimated costs during the system life cycle phases. Such a cost
comparison enables management to evaluate progress and measure
performance18, decide if the system warrants continued development,
and better plan future system development. Further, without life
cycle costing, EPA cannot perform a cost-benefit analysis and
determine whether the value of the service received is worth the
cost. Thus, EPA has no effective means to evaluate alternative
approaches for developing and implementing IFMS. For instance, for
years IFMS management has recognized the need to eliminate the
legacy systems, and the estimated cost of $568,000 a year to operate
and maintain them. However, we estimate EPA is spending about $2
million a year to support these systems, or $11.2 million over the
17 While some costs are covered by using NDPD timeshare figures, other Headquarters, regions, and
nationwide costs such as development, enhancements, and basic financial services are not available on a system
level basis.
18 The current performance measurement factors being developed by EPA's CFO relate specifically to
financial management functions and do not include life cycle costs. Also, while some IFMS performance
measurements are tracked (e.g., system response time, abends, number of transactions), IFMS life cycle costs
are not one of these factors.
39
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
period from their initially planned elimination to their revised
scheduled elimination (i;e., fiscal 1990 through fiscal 1995),
Decisions to extend the life of the legacy systems may have been re-
evaluated if life cycle costing was used. In another case,
decisions were made to implement a major modification (i.e., MARS)
without conducting an analysis of alternatives based on costs in the
context of the IFMS life cycle. In fact, the Agency is still unable
to fully determine the actual costs for MARS as a separate system.
Moreover, because of the way EPA has established its timeshare
budget, system owners and users are not held accountable for their
utilization of these resources. In fact, an FMD official indicated
that the program offices treat timeshare and telecommunications as
"free services." As a result, users are not-concerned about costs
or effective usage of computer services. System, owners and managers
are not aware of user costs of operations by application, and are
not in a position to concentrate on the high cost and demand areas
warranting attention. This cost information is fundamental to
effective planning and budgeting, and not having it has been
detrimental to EPA. An FMD official indicated that this type of
cost information could prove useful when considering distributed
processing rather than a timeshare process. Additionally, other
OIRM and OC officials have agreed that life cycle costs information
could help to effectively and efficiently manage Agency information
systems. Life cycle costing techniques would also aid management in
making decisions to continue, replace, or terminate all or part of
the system; and allow EPA to initiate actions for development of a
new system.
Some EPA Managers Have IFMS Funding Concerns
EPA could be using an inadequate basis to address system funding and
budget constraints. For instance, the September 1993 OMB report on
IFMS identified that some managers felt that funding for IFMS is
inadequate, as follows:
Contract funding for the financial systems...is approximately
$5.4 million a year... EPA management has placed IFMS funding
within the base so as to move "stable funding to parallel
stable operations. [While] most managers feel that this
funding is adequate,... several managers thought that the
planned integration efforts of financial and related systems
could not be supported by current funding. However, there
appears to be no intention to request special funding for
future efforts that may be undersupported.
We also found that EPA budget plans and submissions prescribed by
OMB Circular A-11, Exhibit 43A were understated, and thus Congress
and OMB have not been adequately informed. For instance, EPA has
reported to OMB that IFMS obligations are approximately $6.9
40
Report No. E1NMF3-15-0073-410056:
-------
EPA'8 ZFHS
million a year, whereas our life cycle approach estimates the cost
to be approximately $16 million a year (see Appendix VII).
Recently, EPA revisited efforts to create a WCF. The planned
implementation of the WCF in Fiscal 1996 is expected to enable the
Agency to account for all system costs using life cycle costing
techniques. The WCF is also expected to enable the Agency to link
IRM mission-based planning with the budget process.
\
COST ACCUMULATION PROCESS AND POLICY FOR APPLICATIONS SYSTEMS KEEP
TO BE IMPLEMENTED
Lack Of A Comprehensive Process/System To Accumulate Life Cvcle
Costs
EPA lacks a comprehensive process/system to accumulate life cycle
costs of application systems. The Acting DCFO agreed that the IFMS
costs were fragmented between OIRM, OC, and NDPD and not accumulated
on a system by system basis. This official felt the CFO manages
both the OIRM and OC portion (about $7.4 million annually) and the
Agency timeshare/telecommunications on a workload capacity level.
The Acting DCFO agreed that they need to improve controls over
computer timeshare. In fact, this official indicated that the
Agency initiated a project to create a WCF by 1996 to more cost
effectively manage and control administrative services, including
ADP and telecommunication resources. The OIG fully supports this
effort, which would, in part, provide a process to identify costs on
a system level.
Further, the IFMS system manager told us t;hat life cycle techniques
were not used because it was not a mandatory requirement (see the
next 2 paragraphs) and they were not given the resources to track
system life cycle costs. The Director, IFMS Project Management
Staff also indicated that they did track OIRM and OC costs, but did
not track all the IFMS development and operations costs largely
because they treated the other costs (i.e., telecommunications and
timeshare) as intramural costs (i.e., funds which support staff and
activities internal to EPA) rather than extramural costs (i.e.,
funds for contractors or governmental agencies other than EPA).
Consequently, they have not incorporated these NDPD costs into the
decision-making process. Computer usage and telecommunication cost
information need to be'included with other costs, which would in
turn provide a system life cycle cost perspective. This approach is
vital to proper IFMS decision-making, budgeting, planning, and
controlling.
Clearer Criteria Needed
EPA's life cycle costing policy was unclear and the supplemental
guidance, procedures, and standards had not been issued on a
41
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
.specific cost methodology. However, in August 1994, Chapter 17 of
the IRM Policy Manual was revised and clarified policy on the use of
a system life cycle approach and life cycle costing. Also, an OIRM
official indicated that additional guidance, procedures, and
standards will be forthcoming, which will cover a specific
application system life cycle cost methodology. It is also
.anticipated that the revision to Chapter 2 of the IRM Policy Manual
on mission-based planning will clarify EPA's life cycle pha'ses and
.life cycle costing policy. This policy is not scheduled to be out
in draft until December 1994. .In our opinion, incorporating life
cycle costing techniques into EPA criteria will improve the
oversight, accountability, and decision-making process relative to >
information system development and maintenance. This will be
addressed in more detail in another OIG audit of the Agency's major
systems' maintenance costs.
Additionally, in our opinion, not employing life cycle costing
techniques can be attributed, in part, to the lack of clear Federal
guidance. No consistent cost methodology is used across the Federal
agencies for systems development. While several OMB circulares
require management to track and report life cycle costs,
clarification of policies, terminology, definitions, and
responsibilities is needed on life cycle costing. Initiatives are
already underway that will update these circulares, refine cost
accounting standards, and re-emphasize life cycle costing and the
use of cost information as part of performance measurements. We
plan to address this issue in our current PCIE Software Maintenance
review. Nevertheless, in the absence of more specific Federal
criteria, EPA still should establish effective life cycle costing
policies, procedures, and standards, within the Agency in order to
ensure cost effective acquisition, development, maintenance, and
ongoing operations of software systems.
RECOMMENDATIONS
We recommend that the AA for OARM:
l. Establish an estimated useful system life for IFMS.
2. Continue to work with OIRM and NDPD to develop, an acceptable
methodology/process {including the WCF) to accumulate all
material IFMS life cycle costs.
3. Assign responsibility to track and monitor IFMS on a life cycle
cost basis and update this information annually. Utilize this
cost information for: (a) updating cost-benefit studies;
(b) considering alternatives; (c) external reporting (e.g., CFO
5-year plan); (d) mission-based planning; (e) budget planning
and execution such as the OMB Circular A-ll submissions;
42
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
(f) developing CFO performance measurement standards; and
(g) other major decisions for major enhancements and system
replacement..
AGENCY COMMENTS AND PIG EVALUATION
Based on our review of the AA for 0ARM'S June 15, 1994, response to
our draft "report and other supplementary documents, OARM agreed with
two out of the three recommendations and-partially agreed with the
third recommendation. In addition, OARM has initiated action on
three recommendations.
OARM partially agreed with Recommendation 3. At their June 15,
1994, meeting the EMG directed the Director, IFMS Project Management
Staff to establish a cost model which would include data processing
and telecommunications costs for the WCF and other purposes. These
costs were also included in the Agency's recent Cost Study dated
April 25,-1994. The study was used in a June 20, 1994, decision
paper by the CFO as a basis for continuing-the projects in the,CFO
Five year plan and using the study as a basis for selecting
reengineering projects. However, data processing and
telecommunications costs are not being reported to OMB in Exhibit
43A under the requirements of OMB Circular A-ll. In response to
Recommendation 3, Agency officials stated that they believed that
they were reporting IFMS system costs correctly to OMB, and would
not separately report IFMS-related data processing and
telecommunication costs to OMB. OMB officials told us that system
data processing and telecommunication costs should be reported as -
system costs when the Agency prepared its Exhibit 43A.
43
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
CHAPTER 4
RESULTS OF IFMS CUSTOMER SATISFACTION SURVEY
We conducted an Agency-wide IFMS customer satisfaction survey by
sampling users of five major components of EPA's core financial
management system: IFMS, FMS, ADCR, MARS, and RMIS. The survey
results for all these systems are summarized in Appendix vill. We -
received only enough responses to project statistically the response
averages for IFMS. The survey results on FMS, ADCR, MARS, and RMIS
data are presented only as information.
Overall, IFMS user satisfaction has improved since 1991. The
customer satisfaction survey statistically projected averages for
IFMS were all in or above the acceptable/satisfactory range.
Questions on IFMS availability and usefulness received the two
highest response averages of 7.8 and 7.6 out of a 10 maximum
positive rating, respectively. However, the survey indicated a need
for improvement in training and documentation. This occurred
because of the inability of the IFMS managers and coordinators to
communicate with all of the identified users about available
training and documentation. Consequently, some users were not aware
of available training and documentation that would enable them to
use the system efficiently and effectively. Subsequent to our field
work, a management official stated that the IFMS upgrade to version
5.1e was accompanied by improved documentation and refreshed
training, which should raise client satisfaction in both areas.
*
USER SATISFACTION SURVEY CRITERIA
This customer satisfaction survey is consistent with guidance
provided in the Contract Information System Audit Manual published
by the EDP Auditors Foundation (EDPAF). The manual specifically
notes that users of data or reports produced by an application
system can help the auditor identify errors in processing or other
major problem areas. The auditor should, therefore, identify and
interview a sample of the principal users of the system to determine
exactly how they use it, what they think of its accuracy,
timeliness, and completeness, and whether it meets their information
needs. The format and subject matter of the survey questions were
also consistent with the prior survey conducted by OIRM in 1991.
In response to the March 29, 1991, OIG audit report, entitled
Integrated Financial Management System: Managing Implementation Of
The New Accounting System (Report No. E1AMFO-11-0029-1100153), the
Agency agreed that they would involve all user groups in developing
and implementing the Agency's financial management system, and
44
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
document that their needs and priorities were considered in deciding
the direction and"plans for the system.
IFMS TRAINING AND DOCUMENTATION NEED IMPROVEMENT
The results of the survey reflected a need for improvement in the
areas of training and user documentation for IFMS. The survey
averages for these two areas were the lowest scores in the survey.
We asked the users of I'FMS to rate how well the training and
training materials met their needs. They responded with a overall
rating of 5.8 (+/- 10 percent) out of 10. We further analyzed these
responses and found that 20 percent of these responses were between
0 and 3.5, which fell in the "unacceptable" range. Additionally, we
asked the users to rate how current the system user documentation
was. They responded with a overall rating of 5.9 * (+/- 10 percent)
out of 10. We further analyzed these responses and found that 18
percent of these responses were between 0 and 3.5, which fell in the
"not current" range.
While users had favorable comments, other users expressed
dissatisfaction with training and documentation. Favorable user
comments included:
• I think the training is outstanding and offers a lot of
information for a new user;
• I find the IFMS HOTLINES newsletter very informative; and
• Users are being kept current on IFMS via a bi-monthly
newsletter and e-mails, which are sent out on a very timely, as
it occurs basis.
Critical comments included:
• Little or no training was ongoing for IFMS in our office;
• We have not been offered training;
• We have not received any training;
• There is very little training on enhancements and new releases;
• Coworkers share information as they stumble onto it;
• I have not received any IFMS documentation since the system was
implemented;
45
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
• The IFMS users manual needs to be updated; and
. • *
• Manuals are old and out-of-date.
Thus, some of the users are not aware of important changes to the
system as they are implemented. This will hamper the ability of
Agency personnel to use the system. In addition, users will not
utilize the full potential of the system's capabilities without
adequate IFMS training and system documentation.
BETTER COMMUNICATIONS NEEDED
The availability of training opportunities and documentation updates
is not being communicated to some of the IFMS users. All users need
to be informed of the availability of documentation updates and
training opportunities. Maintaining a list of all users and
requiring the use of electronic mail (e-mail) for IFMS communication
will enhance the ability of FSB to provide important information to
all users on documentation updates and training.
All IFMS Users Cannot Be Readily Identified In TSSMS
EPA mainframe computer financial information systems such as IFMS
depend on the Time Sharing Services Management System (TSSMS) for
establishing accounts, registering users, tracking timeshare usage,
and producing a wide variety of reports on usage activities.
However, TSSMS groups users by organizational entities rather than
specific information systems. Consequently, TSSMS cannot provide a
complete list of individual users of a particular financial system.
For example, when we requested the IFMS user universe from ASD and
the OC for our user survey, we were informed that TSSMS could only
identify accounts of organizations that access IFMS and other core
financial systems but not the individual users of these systems.
NDPD officials noted that trying to determine which computer
accounts belonged to each system was very difficult. They also
stated that they did not have any definitive information on what
accounts belonged to particular systems. During the survey, we
noted that IFMS could be accessed by 75 different accounts. OC
officials noted that the IFMS Security Officer in FMD maintains a
list of all IFMS users. This list consists of Name, Office, and 4-
character IFMS User Identifier. We were told by an official that
subsequent to our review FMD had provided this information to NDPD
This will result in TSSMS being updated with the identity of all
IFMS users and their related accounts.
All IFMS Users Need To Obtain E-Mail Access
IFMS users are not required to obtain access to any Agency e-mail
system even though the FSB uses e-mail to communicate time-critical
46
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
and important information such as system changes and training. The
current IFMS e-mail distribution list includes approximately 500
users,-but does not include all users. During discussions with FSB
staff, we were informed that not all of the financial system users
have individual e-mail accounts, especially in the regions where
IFMS e-mail is sent to a centralized office box. Furthermore, when
an e-mail message is sent to an office box, it is not always
distributed to all of the appropriate personnel. As a result, not
all users are informed of training opportunities.
Since 1989, FMD has had a strategy of using a network of IFMS
Coordinators to disseminate information to each Accounting Point.
Along this line, EPA's resource management community also
disseminates information. OC officials_noted that every member of
the financial management community is directly linked to one of 22
senior financial managers through whom copies of IFMS notices and
announcements, documentation, EMG minutes, and other information are
distributed. However, this strategy needs to be reevaluated based
on a recommendation in the EPA NPR Report Phase II, which dealt with
improving internal communications. OARM has the lead implementation
role for Recommendation R33 from the Administrator's Mandate of
Increased Use of-E-Mail. In line with this recommendation,
dissemination of IFMS information that is currently done through
layers of management would be more efficiently and effectively done
through e-mail.
OARM-RTP has undertaken an integrated e-mail project, which was
scheduled to be in place later in fiscal 1994. Its purpose is to
create electronic gateways from All-In-1 to LAN-based e-mail systems
such as WordPerfect Office (WPO) Mail or cc:Mail. A survey
conducted by NDPD showed that over 80 percent of EPA staff use the
two most popular LAN-based e-mail systems. In order to communicate
with all IFMS users (regardless of which e-mail system they use),
FMD will need to update the IFMS e-mail distribution list so that it
reflects the entire IFMS user universe, whether their address is an
All-In-1 address or a remote (LAN-based) address. Since over 80
percent of EPA employees already have e-mail addresses, communi-
cating with all IFMS users is mostly a matter of updating the IFMS
distribution list upon the completion of the e-mail integration
project.
FSB also publishes a hardcopy newsletter, HOTLINES."which contains
information on IFMS, MARS, on-going system activities and
documentation updates. HOTLINES newsletters are distributed to
about 225 users, including central locations where they are made
available to any user requesting a copy. However, the survey
comments indicated that some users may not be aware of the HOTLINES
newsletter or are not receiving a copy of it. FSB could use e-mail
to inform users on HOTLINES distribution and summaries of its
47
Report No. E1NMF3-15-0073-4100561
-------
EPA'S ZFMS
contents. In February.1994, the OC noted that they would begin this
notification process as of the next issue of HOTLINES.
Future Surveys Should Be Based On Entire User Community
We also noted that the prior 1991 OIRM survey obtained responses'
from key agency officials, but it did not use the entire user.
universe as a basis of the sample. It was sent to 22 senior
financial managers (12 Senior Budget Officers (SBO) and 10 Assistant
Regional Administrators (ARA)) who then distributed it to users
responsible for finance, funds•control, and budgeting. Samples for
future user surveys should include the entire user community to
ensure accurate representation of user satisfaction levels.
RECOMMENDATIONS
We recommend that the AA for OARM require the Comptroller to:
1. Provide IFMS user information to NDPD for updating TSSMS IFMS-
related accounts managed by Agency ADP Coordinators and Account
Managers.
2. Upon completion of the e-mail integration project, update the
IFMS e-mail distribution list so that it reflects all IFMS
users, regardless whether the address is an All-In-1 address or
a remote (LAN-based) address.
3. Use e-mail to notify users on the content and availability of
HOTLINES.
4. • Determine if a cost-effective technique can be developed to use
e-mail to distribute the entire HOTLINES. including charts and
tables.
5. Use all IFMS users as the universe for future surveys.
OARM agreed with all five recommendations. In addition, OARM has
completed action on three recommendations and initiated action on
the other two recommendations.
48
Report No. E1NMF3-15-0073-410056:
-------
EPA'a IFMS
CHAPTER 5
OARM'S FMFIA PROCESS DOES NOT SUFFICIENTLY ADDRESS THE RISKS
; ASSOCIATED WITH .CRITICAL IFMS PROCESSES
The FMFIA, Public Law 97-255, requires each executive agency to
periodically evaluate its system of internal accounting and
administrative controls and submit an annual statement of assurance
to the President and the Congress oh the status of the agency's
system of internal controls. These evaluations are made pursuant to
OMB Circular A-123. The annual reports are to state whether the
systems meet the objectives of internal control and conform to
standards developed by GAO (Standards for Internal Control in the
Federal Government, published in 1983). EPA RMD 2560 applies the
FMFIA review requirements to financial,-administrative, and program
activities at all levels of the Agency and designates the AA, OARM
as the Senior Internal Control Official responsible for coordinating
EPA's efforts to comply with the FMFIA, GAO standards, and OMB
guidance.
An agency component (i.e., assessable unit) is defined in OMB
Circular A-123 as a major program, administrative activity,
organization, or functional subdivision. Previously, EPA guidance
required:
• 5-year Management Control Plans (MCP) which summarized the
Agency's risk assessments, planned actions, and internal
control evaluations to provide reasonable assurance that
controls are in place and working.
• Assessable units including event cycles (i.e., functional areas
within specific organizational subdivisions).
• MCPs to be updated annually and used by management to-monitor
risk assessment activities ensuring that scheduled actions
actually occur.
• Management control reviews (MCRs) and alternate MCRs (AMCRs) be
conducted in accordance with the MCPs to identify internal
controls (i.e., control objectives and techniques) that need to
be strengthened or streamlined.
Subsequent to our review, EPA established a new integrity process in
June 1994. This process requires a systematic review strategy and
assessment on the effectiveness of guidance and procedures.
EPA's FMFIA 1993 annual report did not address the IFMS policy and
guidance, organizational, system development, and cost tracking
deficiencies identified in this report, nor did OARM's FMFIA
49
Report No. E1NMP3-15-0073-4100561
-------
EPA'a IFMS
process. We also examined OARM's 1993 Annual FMFIA Report and found
no material or Agency-level weaknesses related to these
deficiencies. These deficiencies meet EPA's materiality criteria
for reporting to the President and Congress in conjunction with OMB
Circular A-123 and the FMFIA.
We. reviewed OC, OIRM, and the OARM-RTP documentation for assessable
units, event cycles, 1993 5-year MCPs, and assurance letters. Based
on this review we found no on-going or planned reviews specifically
addressing the deficiencies discussed in this report. Furthermore,
these areas may not be covered in any reviews planned in the future,
because the OC's strategies and guidance expressly related to these
weaknesses were not developed. We also examined OARM's 1993 Annual •
FMFIA Report and found that no material or Agency"level weaknesses
related to these issues.
RECOMMENDATIONS
We recommend that the AA for OARM require the Comptroller to:
1. Develop strategies and guidance specific to IFMS planning and
cost tracking and include, or revise, the appropriate
management integrity principles; and
2. Incorporate into Office of Comptroller's review strategy the
evaluation of the IFMS deficiencies identified in this report.
AGENCY COMMENTS AND PIG EVALUATION
OARM agreed with the intent of both recommendations in our draft
report. However, we adjusted the recommendations to accommodate the
new integrity process.
50
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX Z
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, O.C. 20460
f 6 eat
OFFICf OF
MEMORANDUM
SUBJECT:
ts on Draft Audit Report on the
1 • - -• • - -- it Syetem (IFMS)
AND RE SOURCES
MMMCEMENt
PROM:/
TO: *"
Kenneth A. Ronz
Assistant Inspector General for Audit
Office of the Inspector General (2421)
See Appendix II
Note 1
See Appendix II
Note 2
He have reviewed this Draft Audit Report on the Integrated
Financial Management System, IFMS. Both of our offices profited
from several meetings and from a Discussion Draft that we were
able to review earlier.
OVERALL APPROACH TO RECOMMENDATIONS
Of the 17 recommendations in the Draft Report, we have
created Corrective Actlone and milestones for seven. The
remaining recommendations concern actions we have already
completed or simply say to pursue our current course.
Consequently, we offer no further Corrective Action for 10 of the
recommendationa.
Attachment 1 presents each recommendation, verbatim,
followed by our discussion and proposed corrective action.
PISAGREEMENTS WITH MATTERS OF OPINION IN THE'REPORT
We still disagree with several of the opinions in the Draft
Audit Report. He have raised all of these points earlier.
Attachment 2 sets out, in considerable detail, a point-by-point
presentation of our areas of concern. Following are the primary
issues:
Pon't Overlook Proereaa
The Draft Audit- Report has been overtaken by events in
several areas. This is not surprising, since we have been hard
at work improving the system and are now showing results. In
particular, the Final Audit Report must reflect our successful
upgrade to Version S.le of the commercial software. That effort
has proved that we can successfully convert millions of records
51
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 ZFKS
See Appendix II
Note 3
See Appendix II
Note 4
See Appendix II
Note 5
APPENDIX X
of data without impact on our user community. We are now poised to
take advantage of further Government-wide progress with the system.
The report also overlooks a recently reissued IFMS Charter,
updated for the CFOs Act; installation of a new version of the Change
Management System; completion 'of a Feasibility and Cost-Benefit Study
for the system; and imminent .steps to discharge the remaining tasks
that we still must do using the predecessor systems.
The System Maintains Data Integrity
The Draft Audit Report cites other OIG audits that have findings
on accounting or reporting. We are already responding to those
reports with appropriate action.
The implication in this Draft Report, however, is that the culprit
behind those findings is deficient application software. Nothing in
this review or in the other reports supports such a conclusion. The
data base is secure and processes transactions accurately, and we
believe the Final Audit Report should state so.
The System Isn't Pis-Integrated or Wasteful
The impression left by the language in the Draft Report is
that IFMS has many departures from the Government-wide ideal of
effortless data flow across systems. In fact, the system is
predominantly integrated. Thousands of transactions already flow
electronically into the system. Correcting the noncompliances, while
important, will add only a few more.
Similarly, we would vigorously oppose any intimation that
the expenditures discussed in the report are somehow "wasted" or
"unnecessary." We are spending money to operate an effective,
evolving financial management system to meet EPA's requirements. The
total life -cycle costs (hundreds of millions of dollars) presented in
the report are by no means wasted.
Attachments
cc: Sallyanne Harper,
DAA for Finance and Acquisition
Kathryn S. Schmoll, Comptroller
Alvin M. Pesachowitz, Director, OIRM
Carolyn Levine, OARM Audit Liaison
52
Report No. E1NMF3-15-0073-410056I
-------
EPA'8 ZFHS
APPENDIX I
Attachment 1
RESPONSE TO RECOMMENDATIONS
AND PROPOSED CORRECTIVE ACTIONS
Verbatim recommendations from the Draft Audit Report are
presented in bold face. Our discussion of the recommendation follows
in Roman type. Proposed Corrective Actions are in italics.
COSTS AND FINANCIAL MANAGEMENT RISKS f
1. Continue tot
a. Reassess and fully document the requirements, plans,
goals, alternative solutions, costs and benefits of the
remaining XFMS phased implementation and determine whether
the existing approach (i.e., system implementation and
software management activities) should continue or be
revised. The analysis of alternative solutions should
evaluate IFMS and other alternatives against prescribed
OMB requirements and BPA's financial management (user)
needs as well as involve a comparison of costs, risks, and
scheduled completion dates. Provide a decision paper
supported by this reassessment to the CFO and the
Executive Steering Committee for IRM for approval. Do not
proceed with new major changes to IPMS until the
reassessment is completed and approved. .
b. Prepare decision papers with appropriate approvals for
major IPMS projects.
c. Eliminate of the legacy systems according to scheduled
milestone dates while the reassessment is being performed.
d. Update the IFMS Strategy and Master Work Plan once the
reassessment is completed and approved. At a minimum,
this plan should also match resources with tasks, contain
priorities and realistic target dates, and be sufficiently
detailed to identify responsible action officials. Update
and revise this plan at least on annually throughout the
remaining system life cycle.
e. Update and require one generally accepted systems
development life cycle methodology to be followed by BPA
and contractors for IFMS. This methodology should require
a structured approach to system development and systematic
framework for management control. This methodology should
be referenced in all applicable BPA IRM contracts and
require minimum standard documentation (e.g., data
dictionary, cost-benefit
53
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
APPENDIX Z
analysis, requirements analysis, project plan), and Tom
prepared and updated throughout the life of the system.
£. limit the extent of customization to the off-the-ahelf
aoftware modules to the maximum extent possible and
• examine ways to change and re-engineer how SPA does
business to move towards a more standard financial system.
As DIG acknowledges, we already have significant efforts under
way. in each of these areas; for that reason, we propose no
additional corrective action.
For the Decision Paper, OIG acknowledges that we have already
completed a Task Order relating to strategies for the system and
plan to receive formal approval by the Assistant Administrator
{the CFO). We will comply with all Agency policy regarding the
Executive Steering Committee for IRM once that policy is in
place.
We propose no further corrective action for the other items: we
have already demonstrated a history of Decision Papers for major
system events (Ib); we have obligated well over $1 million to
replicate requirements still discharged by the predecessor
systems (ic) and are on schedule to eliminate them during
FY 1995; we have in place a Strategy and Master Work Plan and a
CFO Five Year Plan, with costs managed each year through a
Spending Plan approved by management (Id); standing Agency Life
Cycle Guidance is in place (le); and guidance had already been
included in the Strategy and Master Work Plan to limit
customization where possible (If).
Corrective Action:
No further corrective action required
Target Date
2. Require increased involvement by top management committees
(i.e.. Executive Steering Committee for IRM and ASC) in '
recommending and approving actions for IPMS decisions •
commensurate with their defined roles and responsibilities.
This increased involvement should be formalized and include
intermediate reviews when planned cost or time thresholds are
exceeded or performance criteria are .not met.
We will follow Agency policy toward the Executive Steering
Committee for IRM. The revised IFMS Charter omits reference to
the Administrative Systems Council; that group has ceased to
meet, with its function subsumed by the Executive Steering
Committee. Nevertheless, we will continue informing the ASC's
ex offieio membership about happenings of the system. (The
primary membership of the Administrative Systems Council was
senior financial managers
Recommendations
Printed June 8, 1994
54
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
APPENDIX I
3.
4.
in each Responsible Planning and Implementation Office--i.e.,
the 10 Assistant Regional Administrators for.Policy and
Management and the 12 Headquarters Senior Budget Officers.)
Corrective Actions.
- No further corrective action required.
Target Date
Revise the IFMS Charter to reflect the IFMS system of
accountability, including the delegation of authority under the
CPO; and the line of authority among individuals, staffs,
committees, and management control groups. Formalize the
operational role of the FHD Director under EPA Delegation 1-16.
In February, 1994, the Comptroller circulated a revised Charter
for IFMS and called for comment by the IFMS Executive Management
Group. The transmittal noted that the Charter had fallen out of
date because of the passage of the Chief Financial Officers Act
of 1990 and ensuing changes at the Agency. A revised Charter
was signed by the Assistant Administrator on May 17, 1994.
Work is currently in progress to revise EPA Delegation 1-16 and,
when completed, will reflect the FMD Director's role.
Corrective Action;
Target Date
Regarding the IFMS Charter: no further corrective action
required.
Reissue EPA Delegation 1-16 ......
11/1/94
Update the position descriptions and performance standards of
key officials to reflect the revisions in the IFMS Charter and .
to establish objective criteria on performance measurement to
correspond to current IFMS milestones.
Performance standards are reviewed on, at least an annual basis
by management officials and the managers to whom they report. .
They represent an agreement between those officials and their
supervisors about a wide range of activities for which the
individuals are responsible. Key IFMS managers are well aware
of their responsibilities for IFMS, and their performance
standards reflect those responsibilities and desired outcomes at
the level of detail deemed appropriate for their positions and
levels of responsibility. Likewise, their position descriptions
reflect duties and responsibilities at the level of detail
deemed appropriate by Agency reviewers.
Recommendations
Printed June 8, 1994
55
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
. Corrective Action.-
Target Pate
5.
6.
7.
No Corrective Action required.
Utilize project management and scheduling techniques (program
evaluation and review techniques or critical path method) in
managing IFMS on an overall basis to aid in planning and
controlling interrelated aetivitiea and establishing realistic
milestones.
Managing IFMS on an overall basis is clearly important.
Documents such as the Strategy and Master work Plan, the CFO
Five Year System Plan, and ACTION items from periodic meetings
of the SMG and EMG all point to our commitment to plan and
control interrelated activities. We will continue to apply
management techniques, at the project level as well as at the
overall level, to avoid resource contention and identify
dependencies among tasks.
Corrective Action.-
- No further corrective action required.
Target Date
Incorporate the ARTS functionality into CHS. Re-examine the
outstanding work requests and eliminate any that are no longer
necessary.
The Change Management System, CMS, is the official system to
manage change control and configuration management for the FFS
component of IFMS, for MARS, and for other related components of
the integrated financial management system. ARTS was developed
as a local correspondence log; it has been used solely within
the Financial Systems Branch of FMD and does not duplicate CMS
functionality. EPA will continue to use CMS as the sole change
management system for IFMS. .
A new version of the Change Management System has been under
development throughout the year and was implemented in May,
1994. With the new version, we are in process of reviewing all
outstanding work requests in the previous version and will not
carry over those closed or implemented Work Requests that had
•not previously been marked as such.
Corrective Action;
Complete Transfer of Open Items
to newer version of CMS
Target Pate
8/30/1994
Perform an off-the-shelf software inventory; document the
modules purchased, installed,and implemented; and capitalize
costs over $5,000. Stop paying maintenance on modules which
will not be implemented. Consult with General Counsel about
Recommendations
Printed June 8, 1994
56
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
APPENDIX I
the possibility of recouping original costs and maintenance fees
from AMS for modules purchased, but not implemented.
We note elsewhere that we ceased paying maintenance on IMAGINE,
an unused reporting tool, commencing in 1994. Previously
(through FY 1992) , maintenance was "bundled,." or aggregated
without specific cost per module. EPA remains interested in the
project cost and fixed asset modules. Through informal contact
with General Counsel and the Office of Acquisition Management,
we have discovered that there are further issues of scope
between GSA and EPA that also must be resolved.
Corrective Action;
Target Date
Request the inventory of
software modules from 0AM 7/1/34
Inventory completed 9/1/94
Determine who has authority to pursue •
remedies for delivery orders under the GSA contraqt
10/31/94
Consult with Contracts, Claims and Property Law Division
of the General Counsel's Office about recouping
costs and maintenance fees for unused modules of the
commercial software 11/30/94
LIFE CYCLE COST
1. Establish an estimated useful system life for IFMS.
We will follow Agency policy and guidance on estimated useful
life. We understand that the Agency's System Life Cycle
policies, now under Green Border review, are addressing this
issue.
Corrective Action:
Target Date
Analyze applying system life to capitalization of software
costs 12/30/94
Decision on accounting methodology to apply .
3/31/95
Apply estimated system life while capitalizing software
costs over $5,000 6/30/95
*
Continue to work with OIRM and NDPD to develop an acceptable
methodology/process (including the WCF) to accumulate all
material IFMS life cycle costs.
Recommendations
Printed June 8, 1994
57
Report No. E1NMF3-15-0073-4100561
-------
BPA's IFHS
APPENDIX I
OARM has repeatedly stated its policy interest in creating a
Working Capital Fund to encompass support services, such as the
costs of the Data Center. OARM has created a Working Capital
Fund group, reporting to the Deputy CFO. Those working with
IFMS will participate, along with the rest of the Agency, in a
Pilot WCF in FY 1995, leading to full participation in FY 1996.
Inherent in that effort is development of the "acceptable
methodology/process to accumulate IFMS costs."
Corrective Action;
Wo further corrective action required.
Target Pace
3. Assign responsibility to track and monitor IFMS on a life cycle
cost basis and update this information annually. Utilize this
coat information-for: (a) updating cost-benefit studies; (b)
considering alternativas; (c) external reporting (e.g., CFO 5-
year plan); (d) mission-based planning; (e) budget planning and
execution such as the OMB Circular A-11 submissions; (f)
developing CFO performance measurement standards; and (g) other
major decisions for major enhancements and system replacement,
We will follow Government-wide and Agency guidance on life cycle
cost analysis. The Director, IFMS Project Management Staff,
will assume primary responsibility for tracking and displaying
costs on this basis, as an extension of an already Chartered
role. That individual's reporting chain of command, i.e., the
Comptroller, Deputy CFO, and CFO, will monitor adherence to the
recommendation.
Corrective Action: Target Pate
No further corrective action required.
CUSTOMER SATISFACTION
1. Provide IPMS user information to NDPD for updating TSSMS IPMS
related Accounts managed by Agency ADP Coordinators and Account
Managers.
In June, 1994, FMD sent a listing of all IFMS to the Time
Sharing Services Management System managers at the Data Center.
FMD will continue to maintain the primary listing of all users
as part of its responsibility to maintain security over the
financial system.
Corrective Action;
Wo further corrective action required.
Recommendat ions
Target Pace
Printed June 8, 19?',
58
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX I
Upon completion of the e-mail integration project (fiscal 1994),
update the IFMS e-mail distribution list so that it reflects all
IFMS users, regardless if the address is an All-In-1 address or
a remote (LAN-based) address.
We are in favor of as much-notification as possible and will
explore ways to expand our distribution lists (after the e-mail
Pilot is completed) as well as ways to keep the list current.
Nevertheless, we believe responsibility is already assigned in
each RPIO for distribution of relevant IFMS information. We
will follow our current policy and leave primary responsibility
for notification with our existing network of IFMS Coordinators.
3.
4.
Corrective Action;
Target Date
Issue guidance to IFMS Coordinators on more extensive use
of e-mail for communications at their sites;
coordinate with findings of the Integrated E-mail
Pilot 11/1/94
Use e-mail to notify users on the content and availability of ,
HOTLINES.
We believe that further notification is worthwhile and will
begin to include this notification.
Corrective Action:
- No further corrective action required.
Target Date
Determine if a cost-effective technique can be developed to use
e-mail to distribute the entire HOTLINES. including charts and
tables.
The Agency has had notable success in providing the
Comptroller's Directives electronically to the Agency via LAN-
based software. We believe that we can extend this application
to the publication HOTLINES. For the time being, however, the
software is limited in its ability to process graphics; the
usefulness of the publication would be severely limited without
the graphical elements.
Corrective Action:
Target Date
Determine if an upgrade to LAN-based software can include
text and graphics for the publication HOTLINES11/1/94
If appropriate, include HOTLINES in LAN-based distribSi£L£*5
Recommendations Printed June 8, 1994
59
Report No. E1NMP3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX I
5. Use all IFMS users .as the universe for future surveys.
We will include all direct users in future surveys. We reserve
the ability to expand our target population to "second" and
"third" level members of the user community as well, i.e.,
Agency managers and. supervisors, some of whom rarely if ever log
on to the software. This is the approach we followed in the
1991 Satisfaction Survey.
Target Pace
Corrective Action:
Wo further corrective action required.
FMFIA PROCESS
1. Establish event cycles specific to IFMS planning and cost
tracking and include, or revise, th« appropriate control
objectives and techniques.
EPA will incorporate management integrity principles into our
• plans and procedures, such as the updated Strategy and Master,
Work Plan and the CFO Five Year System Plan.
Corrective Action;
Wo further corrective action required.
Target Date
2.
Formally schedule MCRs of IFMS planning and cost tracking issues
in the next MCP.
We will follow the Agency's current policies toward management
integrity and include. IFMS in OC's FY 1995 review strategy.
(OIG is surely aware that OMB has waived certain requirements
under OMB Circular A-123, which implements the Federal Managers'
Financial Integrity Act, so that the terminology of Management
Control Reviews and Management Control Plans has been
superseded.)
Corrective Action;
Target Date
Include IFMS in OC's FY 1995 review strategy . 12/31/94
Recommendations Printed June 8, 1994
60
Report No. E1NMF3-15-0073-4100561
-------
EFA's IFMS
APPENDIX I
Attachment 2
SPECIFIC COMMENTS ON THE DRAFT AUDIT REPORT
Verbatim extracts from the Draft Audit Report appear in
bold face, including a page and .paragraph reference to the Draft
report. Our specific responses are in Roman type.
piitl
Appendix II
Ite 6
je Appendix II
bte 2
the Agency plans to spend nearly $28.4 million to fully
implement IFHS and to maintain the legacy ays terns --more
than 3.7 times its initial estimate to implement the
system
The value should be $16.8 million, as detailed in our
response to a Position Paper. The value is consistent
with the estimates presented in the 1991 Audit Report.
This Draft Audit Report changes the basis from the
previous audit: (1) it includes Operations and
Maintenance, but only for predecessor systems; (2) it
includes Timeshare, but only for predecessor systems; (3)
it ignores offsetting costs when .processing is shifted to
IFMS . We do not concur with any of those changes .
We believe the values in the Draft Report and in Appendix
VI represent a change of basis between the 1991 audit
report and this report . The previous audit report
accepted EPA's standard accounting for costs, i.e., the
costs for development and enhancement of the system as
reported to OMB .
EPA — (4) plans to convert to the latest release of
American Management Systems' s (AMS) off-the-shelf software
in Hay 1994, which will provide increased functionality
within IFMS;
EPA has 'successfully upgraded to the latest Baseline
release of the commercial software, which we call Version
S.le. That implementation in April, 1994, culminated two
years of effective life cycle management, including
extensive testing.
Several sections of the Draft Audit Report must be revised
to reflect this accomplishment .
BPA plans to: (1) eliminate reliance on the legacy systems
We have announced elimination of reliance on predecessor
systems in several forums. We have obligated well over $1
million in FY 1993 to further
Recommendations
Printed June 8, 1994
61
Report No. E1NMF3-15-0073-4100S61
-------
EPA's IFMS
APPENDIX I
See Appendix II
Note 4
See Appendix II
Note 7
See Appendix II
Note 8
that end. We have work groups active that are identifying
the requirements that the predecessor systems still
discharge and have specific projects planned and funded to
discharge those requirements.
electronically bridge IFMS with other financial and
administrative systems
The Draft Audit would leave an impression of near-total
dis-integration to a reader unfamiliar with EPA's systems.
In fact, transactions regularly and effortlessly flow from
our Contracts Payment System; our Payroll System; our
Headquarters Grants Systems; and our Funds Management
System every night. The predecessor financial management
and budget systems are also electronically linked. The
volumes are in the thousands of transactions each night--
i.e., the vast majority of transactions flowing into IFMS.
Separately, we have identified as a material weakness our
noncompliance with OMB Circular A-127 for the Personal
Property Accountability System and for the Regional
portion of the Grants Information and Control System. We
intend to complete the final link to the Grants system and
to employ an integrated solution for Property.
Nevertheless, the likely volumes of transactions that
should be electronically bridged from those two remaining
systems will run in the hundreds, not thousands, of
transactions each night.
Overall, IPMS user satisfaction has improved since 1991.
We believe that user satisfaction has improved
dramatically since 1989, when the system first came up,
and has also improved somewhat from a Customer
Satisfaction Survey conducted for the Deputy Administrator
in 1991.
This occurred because of th« inability of the IPMS
managers to communicate with all of the identified users
about available training and documentation.
The OIG team continually overlooks the network of IFMS
Coordinators that places a local representative at each
IFMS site. Further, we believe that most users are in
fact receiving information in a timely way through their
Coordinators as well as through other means, such as IFMS
Updates distributed via electronic mail.
We built the network of Coordinators because it is nearly
impossible (and a considerable resource drain)
Recommendat ions
Printed June 8, 1994
62
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX I
P113
Appendix II
:e 9
P213
Appendix II
:e 2
Appendix n
:e 10
P713
for the central organization to keep up with the comings
and goings of hundreds of financial management personnel.
Each Coordinators is responsible for disseminating of
timely, useful information to people who work' down the
hall--not halfway across the country.
Contrary to later assertions in the Draft Report, we do
not believe that locally identified Coordinators
constitutes a "layer of management" that should be
streamlined out of existence. Rather, we view it as local
empowerment of people close to the situation in their own
locations.
m
The Agency originally planned to implement IFMS at an
estimated cost of $7.7 million in a 3-phased approach by
October 1990.
This 1987 underestimate was rightly pointed out in the
previous (1991) audit but has little relevance today. The
Agency notified OMB in September, 1990 that we were
changing direction away from the phased approach.
In response to the CFO Act of 1990— Also, in 1990 the
Agency established, through an IFMS Charter...
The CFOs Act was passed in the fall of 1990. The IFMS
Charter was issued earlier - September 14, 1990. We have
since reissued the Charter, taking the CFOs Act into
account; the revision was signed May 17, 1994.
September 1993 OXG audit report, entitled "Reconciling
Cash" ... EPA experienced conversion problems due to
different data fields between the systems. This was
reported as a material nonconformance in 1989, but has not
been fully corrected.
In FY 1993 we were able to correct the conversion errors
arising from the 1989 implementation of IFMS. We reported
this fact in the 1993 Integrity Act Report to the
President and Congress. The financial statement audit
acknowledges this fact since many of the corrections were
provided and approved by the auditors.
June 1993 OIG audit report, entitled "Report of Financial
Audit" ... statements for the Superfund Trust Fund,
Leaking Underground Storage Tajik (LUST) Trust Fund, and
the Asbestos Loan Program... disclaimed an opinion....
Many of these internal control deficiencies were due, in
part, to weaknesses in key accounting controls and
systems.
Recommendations
Printed June 8, 1994
63
Report No. E1NMF3-1S-0073-4100561
-------
EPA's IFMS
See Appendix II
Note 11
See-Appendix II
Note 2
p!0l2
See Appendix II
Note 6
See Appendix II
Note 12
APPENDIX I
Actually, the. report on the Financial Statement Audit
acknowledges that EPA had taken corrective action with
respect to the issues noted by the auditors and that they
had no new recommendations. Thus, this paragraph does not
accurately reflect the financial statement audit report
conclusions.
We believe that the issues raised are not a "system"
problem as much as they are financial management problems.
The implication of this portion of the Draft Audit Report
is that the SOFTWARE is deficient- -a conclusion not
supported by the materials referenced.
SPA has not successfully upgraded to 'more current software
versions
As of April, 1994, EPA runs the most current version of
the Federal Financial System.
EPA plans to apend nearly $28.4 million to fully implement
IFMS --more than 3.7 times ita initial estimate to
implement the system. . . These costs include up to an
estimated $11.2 million to maintain legacy systems over
the period from their originally scheduled elimination to
their revised scheduled elimination (i.e., fiscal 1990
through fiscal 1995}
Elsewhere we observe that the total cost estimate - on the
basis of the 1991 audit — has in fact declined in the last
three years. We also dispute the change of basis between
that audit report and this one and dispute the inclusion
of timeshare and O&M costs for only some of our systems.
these costs do not include an additional several million
dollars over the next couple of years to update the
account code structure and convert data from the legacy
systems to IFMS.
OIG makes these assertions as if they were fact. They are
not. We will incur some costs to convert to the expanded
account code structure . Current estimates are around
$300,000, most of which has already been obligated this
fiscal ye,ar.
ASD's Information Strategy Plan (April, 1990) predicted
that the estimated cost for National administrative
systems would be $1 million.
We doubc that we will need to "convert data" from the
predecessor systems. Rather, we expect to be able to read
historical data directly in its native format .
Recommendations . Printed June 8, 1994
64
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
Appendix II
:e 4
je Appendix II
he 3
Be Appendix II
jte 1-3
Appendix II
14
APPENDIX I
Years ago, when we began to study this issue, we
considered converting historical data, but recent work
shows that to be an unlikely solution.
complex manual reconciliations are necessary where
automated interfaces, are not in place, and lack of
interfaces causes duplicate data entry- -a material
nonconformance reported in the 1993 BPA FHFIA report
We have always acknowledged that we have some deficient
interfaces and have reported them to the Agency and to
OMB. We point out, elsewhere, that the amount of
duplicate data entry is substantially .smaller than the
amount of data that DOES pass electronically across our
systems .
Further, a number of recent audit reports identified
serious data integrity problems with IFMS.
We believe this statement is irresponsible and will be
used out of context. OIG has already acknowledged that.
the work supporting this Audit Report included no separate
tests of data integrity. We know of NO PROBLEMS with the
APPLICATION SOFTWARE that support any such statement about
data integrity. To be sure, other OIG audit reports, have
cited significant weaknesses, and we have created
appropriate corrective action. The flaws, however, lie
predominantly in policies, procedures, and processes --and
not in the workings of the commercial software.
EPA has fragmented managerial authority and has not
clearly defined lines of authority among the four
organizations primarily involved in XFMS.
We have never agreed with OIG's opinion that authority is
"fragmented." The IFMS Charter never superseded the
original chain of command, which has always been in place.
The emphasis of the Charter was in making sure that all
voices were heard on IFMS development and operations,
including the views of the user community.
the accounts receivable module was implemented with
technical shortfalls in the debt servicing area regarding
... interest, penalties, and handling charges
Every other Federal agency running FFS was affected by
this flaw in accounts receivable until a minor upgrade to
FFS 4.0.5 in about 1992. EPA Version 5.1e, implemented in
April, 1994, profits from the repaired commercial
software. EPA has recognized further discrepancies,
relating to posting compound interest in
Recommendations Printed June 8, 1994
PIOUS
65
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
See Appendix II
Note 15
See Appendix II
Note 2
See Appendix II
Note 15
See Appendix II
Note IS
P14H2
See Appendix II
Note 17
pl4l2
P1412
the Superfund program, that still remains to be corrected.
This requirement is unique to EPA and its Superfund
program.
module ... cost and project accounting .... was never
purchased.... payroll ... was purchased ... but it was
not installed. As of September 1993, the requirements to
be satisfied by these two modules were being studied
Fixed assets and reporting... have been installed ... but
they have not been implemented
EPA is investigating whether AMS's current version of the
"Project. Cost Accounting" module, recently modified for
another agency, will support our requirements. The CFO
Plan (1993) identifies a timetable for completion of
requirements, selection of software, and implementation.
EPA installed the Consolidated Payroll Redistribution
System, CPARS, to meet its payroll accounting and
distribution requirements, in lieu of the AMS module.
A Quality Action Team has recommended that EPA adopt a
version of the AMS Fixed Assets module to meet our
property accounting and accountability requirements.
EPA currently employs Reports Distribution System and
Reporter, two reporting adjuncts to Baseline FFS.
module was purchased and annual maintenance fees were
paid...•
We do not believe that the situation with annual
maintenance fees is as clear as OIG asserts. EPA paid a
fixed amount, about $47,000 per year, for yearly
maintenance from 1987 to 1992. We understood that
maintenance charges were "bundled" together; that costs
were not further attributed to specific modules; and the
costs did not change over that period. Elsewhere we agree
to explore carefully the acquisition documents supporting
the negotiated annual fee.
the Agency developed a separate reporting system (i.e.,
MARS) which was never intended in its original plan
We fail to see the current relevance of our intentions in
1985 about ad hoc reporting.
[MARS] still does not always meet the Agency reporting
needs. For instance, MARS does not provide reports
necessary to support some day-to-day accounting operations
such as transactions input.listings and
Recommendations Printed June 8, 1994
66
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
Appendix II
le 17
Appendix II
2 16
Appendix II
;e 4
pl6H3
le Appendix II
Ite 18
pl?1Jl
external financial reports submitted to Treasury and OMB.
OIG fails to recognize that EPA uses a FFS Baseline report
writer, called Reporter, to round out reporting needs.
(Reporter uses a difficult programming language that is
not suitable for all. of EPA's financial managers.) We use
that tool whenever we have requirements that were not in
the MARS design.
We also have reporting projects under way, as part of the
Eliminate FMS project to be completed in 1995, to allow
Historical Data reporting at the detail level and to
support detailed Payroll reporting. Further, we have a
continuing program of enhancements to MARS to respond to
evolving user needs .
annual maintenance for other modules not implemented.
These include five modules- -PC interface; report painter;
cost allocation; IMAGINE; and report distribution- -which
were identified on an August 1992 GSA contract.
•.
EPA currently employs PC Interface and the Reports
Distribution System. We ceased paying annual maintenance
on IMAGINE in 1994 . Elsewhere we note that determining
the basis for annual fee payments will require careful
research .
While IFMS is currently interfaced with HPAYS and CPS, we
found that PPAS and GICS have not been fully interfaced
with IFMS and are not scheduled to be completed until
September 1995
Elsewhere we note the relative numbers of transactions
that are electronically exchanged. In terms of volume,
the overwhelming number of transactions flow across
systems without rekeying. We also note that we have
declared the lack of two remaining interfaces to be a
weakness.
Internal and external studies... and internal
correspondence have shown that the 10 digit account code
structure is limited
The Agency Comptroller did not DECIDE to move to the
expanded account code until 1993 '. We are on the way to
installing the expanded 6-field, 4i-character IFMS
structure universally within the Agency.
BPA has an on-line data dictionary for ADABAS known as
PREDICT which could include a single, comprehensive
Recommendat ions
Printed June 8, 1994
67
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
See Appendix II
Note 19
See Appendix II
Note 2
See Appendix II
Note 20
APPENDIX I
IFMS data dictionary.... As of November 1993, the PREDICT
data dictionary was not comprehensive
We have since loaded information into PREDICT for IFMS.
ASD is also in the process of establishing a standards
file that will normalize data across systems.
p!8l2 We analyzed ASD's [Change Management System] CMS .,
found that out. of 237 work requests, only 25 (10.5
percent) were implemented/completed
and
The requests that are not software related cannot be
marked as Implemented. They should be marked as Closed, a
status unavailable in the prototype of CMS. There are more
work requests that are finished or completed than
identified by the old CMS status.
In May, 1994, we implemented a new version of CMS. This
version contains the Closed status indicator. It should
also be pointed out that the SMG and EMG serve as change
control boards for development activities; if not
implemented, the request is either in process or
consciously deferred by management. Smaller working
groups monitor the status of maintenance with AMS and with
the MARS contractor. The new version of CMS also tracks
work requests that are placed on-hold.
p!8l3 three other systems (iFMD's Action Request Tracking
System,] ARTS... Problem Management Detailed Record at
NDPD... IFMS Hotline run by AMS) ... track and/or manage
changes to IFMS and the core financial management systems
within different divisions.
ARTS is not a system for tracking changes to software or
configuration management, which is our definition of
"change management." OC and ASD use the Change Management
System, CMS, as the sole system for tracking and managing
those two functions.
When we identify system problems that require contractor
support, we enter that formal request into the Change
Management System, which allows us to track actual changes
to software.
The Problem Management Detail Report and .the IFMS hotline
system support activities within their respective
organizations for tracking IFMS activities outside the
scope of change management. Neither of these are change
management systems and should not be incorporated into
CMS; however, some information which they contain may be
fed into CMS at a later time.
Recommendations Printed June 8, 1994
68
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX I
Appendix II
2
Appendix II
le 21
Appendix II
:e 22
EPA updated its- 1985 requirements analysis in January 1994
and is in the process of updating its cost-benefit
analysis. Without these documents, EPA can not be assured
the system will meet its intended objectives, and user
needs.
The cost-benefit analysis was delivered to the Agency on
March 25, 1994. In general, the two deliverables
(requirements and cost-benefit analysis) confirmed the
course of action identified in the Strategy and Master
Work Plan and in the CFO's Five Year Plan. The
independent contractor's report and those two internal
studies agree in objectives and courses of action.
IPMS has remained on the OMB high risk list since 1989.
IFMS was first placed on the OMB High Risk List in 1990.
We have held discussions with the OIG staff performing the
CFO annual financial statement audit about removing IFMS
from the OMB High Risk List during the last several years.
Generally, the OIG staff has been very receptive to our
proposals but were not committal. The OIG staff believed
that until Version 5.IE was implemented that EPA should
not remove IFMS from the OMB High Risk List. We observe
that the Upgrade was installed in April, 1994.
material weaknesses identified in 1988 regarding PMS/PPAS
have not been corrected
We disagree with the implication that we are not committed
to resolve the property issue. EPA has taken timely
corrective action with respect to property. In 1988, our
Facilities Division implemented the Personal Property
Accountability System to replace the old property system.
However, we have found that this solution was inadequate.
In 1989 with the 'FMS to IFMS conversion, we had to focus
our efforts on the general ledger and the associated
performance problems and not on property. However, in
1990, we started requiring FMOs to perform reconciliation
with the regional property personnel. Further, we started
to require regional personnel to perform transaction
testing in this area. We also started to review this area
during our Quality Assurance Reviews.
In 1993, we formed a Quality Action Team to review the
property issue because adequate progress was not being
made along with other considerations. The Quality Action
Team recommendation was that the Agency
Recommendations
Printed June 8, 1994
69
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
P19l3
See Appendix II
Note 23
P21U
See Appendix II
Note 12
P2213
See Appendix II
Note 24
implement the AMS Fixed Assets module that would come with
an interface to the AMS Core System.
four of eight nonconformances identified between 1983 and
1989 are atill not corrected.
EPA reported in FY "1993 only three accounting system non-
conformances, not four. The three nonconformances are:
(1) Regional/General Ledger Accounts Receivables, (2)
Property Accounting Process, and (3) Accounting System
Interfaces. This paragraph leads the reader to believe a
nonconformance, a departure from an "ideal" system, is the
same as a material weakness, which is_ a more serious
determination. Accounting system nonconformance is
defined in OMB Circular A-127: Financial Management
Systems.
[earlier study] estimated that it would cost $310,000 to
modify .. .• systems [for the changed account code] . PHD
officials estimated in 1992 that modification of the
account code structure and conversions of data ... could
cost up to several million dollars.... FMD had initiated
another study to update the estimated costs which still
has not been completed.
The study is complete. It essentially reconfirmed the
earlier estimate of about $300,000. We do not expect to
convert data between the old systems, but simply to read
historical data in its original format.
the average life of a financial system is 12 years
We read with interest the 1992 IEEE article that OIG used
to develop the assertion about system life. The following
observations are relevant about the article and about
OIG's assertion.
1.
2.
The survey supporting the numerical conclusion was
conducted among colleagues of the Systems Division
of Nippon Oil Information Systems Corporation. We
are not sure that the respondents reflect the
Federal Government's situation.
Dispersion about the mean. The "Accounting Appli-
cation Area" sample consisted of 16. samples, with an
average life of 12.1 years and a standard deviation
of B,8 years. With that large a standard deviation,
a useful life of over 20 years is not that unlikely.
3.
Recommendations
"Useful Life" is ambiguous. We note that it is
term of art in accounting, to reflect the
Printed June 8,
1994
70
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX I
Appendix II
le 3
denominator for capitalizing and amortizing costs.
It also represents a span of usefulness, which need
not be synonymous with the previous meaning of the
term. Usefulness can be greatly extended with
prudent upgrades.
4. The concept of Version-Up is mentioned, primarily in
connection with system software. In that light, EPA
already "retired" its original implementation
(Version 3.0) when we successfully upgraded to the
current Version S.le. „ Similarly, AMS has predicted
a Version 6.0 of the commercial software, which
would re-commence a useful life when installed later
in this decade.
p23H4 IFMS has serious actual data integrity problems ,
because of system-related, as well as data integrity,
weaknesses 'in the accounts receivable, property, and
accounts payable areas....[S]ystem weaknesses and the lack
of controls have contributed to inaccurate financial
information and material adjustments to the Agency's
financial statements.
These comments about data integrity are not supported by
this audit team's work nor are the inferences from other
OIG reports accurate. Based on our conversations with the
financial statement auditors, there exist certain system •
deficiencies but they do not impact on data integrity.
The issues raised in the financial statement audit center
on financial management, not software problems. The
financial statement audit indicated that compliance with
policies and procedures by regional personnel and self-
imposed OIG scope limitation led to the disclaimer, not
that the system was inadequate. Further, the EDP review
report by the OIG contractor did not reveal any serious
data integrity problems.
The OIG Team did not perform any testing of EDP controls
and the wrong inferences are made from our FMFIA letter
and other OIG reports, when we questioned the audit
manager of the financial statement audit, we were informed
that the OIG staff will do the testing of IFMS 5.le during
1994. Thus, our view is that until the auditors develop
" sufficient evidential matter to support their statements,
this paragraph should be deleted from the report.
p25H2 EPA has not kept up with the current JFMIP requirements...
accounts receivable function include provisions to
calculate interest...not currently
Recommendations
Printed June 8, 1994
71
Report No. E1NMF3-15-0073-4100561
-------
EPA's ZFHS
See Appendix II
Note 25
See Appendix II
Note 2
See Appendix II
Note 26
See Appendix II
Note 26
APPENDIX I
performed under IFMS aoftware version 3.0... installment
receivables..', compound interest.
JFMIP has just issued interim final CORE Requirements that
are more performance based and less directive; the
statement about "current JFMIP requirements" is
misleading.
Further, with the implementation of Version S.le, EPA has
a much improved Accounts Receivable function. He now have
the capability of installment payments; however, it does
not handle some types of nonstandard installment
agreements that are costly and difficult to automate.
Other Federal users have the same problem.
JFMIP does not require compounding interest as a
requirement, so that EPA would need to replicate that
functionality outside IFMS (or further customize the
system) .
CMS work orders for software modifications/enhancements, do
not provide a crosswalk (audit trail) to the contract's
SOW task orders.
The Prototype CMS did not; we relied on the expertise of
the Delivery Order Project Officer. The version of CMS
installed May, 1994, categorizes Work Requests by task of
the Del ivery • Order .
We estimate that IFMS system life will end in the year
2001
The principal strategy underlying the Decision Paper on
the system includes a major Upgrade to FFS Version 6.0 in
FY 1999 and FY 2000. On some definitions, this approach
of "Version-Up" constitutes the end of useful life of the
preceding Version. Another view would be that it reflects
evolutionary development rather than the "end" of system
life as traditionally conceived. We find "system life"
for evolving commercial software to be a difficult
concept. Still, we expect to be using an upgraded, right -
sized version of the Federal Financial System well into
the next century.
the Agency will achieve limited benefits before the end of
the system life.
The benefits are not limited if software evolves through
Version-Up. Part of the CFO's strategy as articulated in
the Five Year Plan is to participate aggressively on the
Federal Financial System Users Group, which acts as the
Recommendations Printed June 8, 1994
P2712
72
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX I
Appendix II
:e 27
Appendix II
Ite 28
le Appendix II
Ite 29
Change Control•Board for the AMS product. Successful
participation will mean that the next versions of Baseline
software will contain not only improvements developed at
other agencies but will also include, as Baseline, some
enhancements developed first at EPA.
p28H4 little evidence that other top management officials were
involved in approving major system development decisions
Nowhere in the entire Draft Audit Report is "top
, management" defined or the assertion adequately supported.
Pursuant to the IFMS Charter, we bring issues to the
Executive Management Group, which provides a forum for
OARM and user representatives to discuss all aspects.
(Membership of the EMG is Division Director-level.) All
communications from that group have been signed by an OARM
Office Director. All decisions a.e made within the CFO's
chain of command and always have been. Installation of
the Upgrade to Version S.le, a major system event, and the
revised IFMS charter were approved by the Assistant
Administrator.
p28^4 DAA for Administration and Resources Management was not
involved in major decisions regarding IFMS until in August
1991 when he approved the creation of the IFKS project
management functions
In point of fact, that official was involved in setting up
the IFMS project team in February, 1990. Earlier events
are surely out of scope of this Audit Report.
p29l3 ASC, established in 1990, developed a draft charter which
includes responsibilities to advise the AA...
The Administrative Systems Council served a useful purpose
as a conduit for communications about IFMS. The Council
consisted of all senior financial managers in the Agency
(all Senior Budget Officers and all Assistant Regional
Administrators for Management), as well as other senior
OARM officials.
Recently the Executive Steering Committee for IRM has been
chartered; its duties subsume those of the ASC, which is
being disbanded.
p30H3 Our March 1991 audit report recommended ... independent
organization with authority, responsibilities, and
resources to successfully complete the objectives of the
IFMS implementation and that this organization
Recommendations
Printed June 3, 1994
73
Report No., E1NMF3-15-Q073-4100561
-------
EPA'a IFMS
APPENDIX Z
directly report to senior management.
established an IFMS Charter
In response, OARM
See Appendix II
Note 30
p30H3
See Appendix II
Note 31
p30^3
See Appendix II
Note 13
See Appendix II
Note 32
IFMS Charter was signed September 14, 1990, not "in
response" to the 1991 audit report. The OARM response to
the audit report referred to the Charter and to an
organization defined by it, the Project Management Staff.
The authority for decisions always rested, as it had in
the past, with the Comptroller. Later delegations after
the CFOs Act was passed reiterated that chain of command.
Major activities of this IFMS staff include ... approving
software sub-releases
As we clarified working relationships under the CFO, we
settled on the procedure in place now: the Project Manager
recommends approval of releases, after seeking the advice
of the Executive Management Group, which consists of four
Division Directors reporting to the CFO as well as user
representatives outside OARM. Releases are approved at,
the appropriate level within the CFO's chain of command.
OMB [review team] concluded that there is no clearly
established office or individual accountable for ensuring
that decisions from the planning process are effectively
carried out
The Agency Comptroller is the "clearly established"
individual accountable for ensuring that decisions are
effectively carried out.
absence of a comprehensive plan has resulted in IFMS being
implemented on a piecemeal basis through the budget
process (i.e. primarily the SM6 IFMS spending plan). This
process is short term and only provides detailed costs for
the current year, rather than a structured, well thought-
out, SDLC approach over an estimated system life
The SMG Spending Plan --prepared by the Project Manager,
managed day-to-day by the SMG, and overseen by the EMG--
has been the vehicle for planning and tracking costs since
1990. The OIG team continues to understate its value,
time scope, or comprehensiveness in managing the system.
Recommendations
Printed June 8, 1994
74
Report No. E1NMP3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
p33Hl
|e Appendix II
-e 33
P3414
fee Appendix II
Ite 34
The Spending Plan follows the structure of the IFMS
Strategy and Master Work Plan, which defines projects
within broad budget formulation categories.
The Plan for the Operating Year is the result of
progressive refinement of a Spending Plan that was created
previously. Contrary to OIG's opinions on how to
formulate a budget, this approach to the plan follows
EPA's established approach toward budget formulation and
execution.
Furthermore, the Agency made a conscious decision to embed
IFMS funding within its Base program for formulation
purposes. The intent was to stabilize funding for the
now-stabilized financial system. The approach has been
recently confirmed in the Cost-Benefit analysis, which
shows that the CFO's Five Year Plan can be achieved with
stable funding over the next several years.
Finally, the President's Budget for the system has been
identical for the last four fiscal years. In every year
but one, the Agency was forced to reduce that budget at
the time of the Congressionally enacted appropriations.
We believe that this point in the cycle of budget
formulation presents the most risk to funding for the
system and have created the most detailed and useful means
of dealing with reductions to the Agency's Operating Plan.
Further, the Acting Director of FMD indicated that
although IFMS was not cost-effective or efficient, it was
"running.-"
The Director does not recall making such a statement on
cost-effectiveness and disavows it. He stands by the
assertion that the system is running: making payments,
monitoring budget execution, supporting external and
internal reporting, and otherwise performing the core
functions expected of a financial management system. The
Director did state that EPA's experience in installing a
new system, despite our difficulties, compares very
favorably with a number of other agencies' experiences.
we identified 11 different system development
methodologies... in contracts' SOWs
Of the 11 methodologies identified, 4 were internal
procedures of the contractor. These contractor
methodologies are standard operating procedures and are
necessary for the contractor to demonstrate that they
Recommendations
Printed June 8, 1994
75
Report No. E1NMF3-15-0073-4100561
-------
EPA'e IFMS
APPENDIX I
p36ll
See Appendix II
Note 35
See Appendix XI
Note 36
use a life cycle methodology, and it is their standard
procedure in providing system support internal to their
organization.
EPA need not be care what their internal method is as long
as they design and develop code that .follows the EPA
methodology, and that ALL EPA life cycle methodology
requirements are satisfied.
Different EPA methodologies are also identified in IFMS
delivery orders. Historically, they are indicative of the
EPA organization managing the IFMS contractors as well as
the task being performed. By now, ASD has established its
methodology which is based upon the Agency life cycle
methodology, and is referenced in delivery orders managed
by ASD.
BPA'a planned implemented version of S.le will be 29
percent greater in terma of lines of code...
(customization) then the off-the-ahelf package 5.1
version. This percentage of change is significantly leas
than the changes to the version of 3.0, but is nonetheless
still excessive...
Without a view of the type of customizationoorthae
requirements it meets, OIG's numbers are meaningless.
A significant amount of the customized code represents a
separate module (Budget Preparation) , tightly integrated
with the rest of FFS, prepared to meet EPA-specific
requirements. Another significant portion of the
customized code represents reports.
Further, as is the case at -other major agencies, EPA has
significant, valid, and unique requirements that must be
discharged. We believe that it is now cost-effective to
continue already- created customization than it is to
replicate the valid functionality outside FFS, with
attendant O&M costs.
Based on the cost information available, we estimated that
the total life cycle cost of the system to be
approximately $202 million over a estimated life of 12
years for an average cost of about $16 million a year (see
Appendix VIII) .
Elsewhere we point out specific problems with the precise
value set out here; in particular, the amounts are
expressed neither in total dollars nor in present-value
1994 dollars. We do agree that running a large, effective
financial system costs a substantial amount of money.
Recommendations Printed June 8, 1994
76
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
p43t2
Appendix II
32
p44Hl
Appendix II
:e 37
se Appendix II
ite 38
APPENDIX I
While we acknowledge tfiat the IFMS project manager
prepared IFMS, spending plans based on OIRM and OC budgets,
these plans cover only one year
Elsewhere .we show that the Spending Plans cover several
years in a structure, consistent with the Strategy and
Master Work Plan. They also support the most difficult
practical exercise, namely responding to reductions from
the President's -Budget as the Agency creates its Enacted
Operating Plan.
we estimate EPA is spending unnecessarily about $2 million
a year to support these [legacy] systems, or $11.2 million
over the period from their initially planned elimination
to their revised scheduled elimination (i.e., fiscal 1990
through fiscal 1995)
We reject the statement ("unnecessarily") and the amounts.
1. The costs are not "unnecessary." The predecessor
systems are still discharging valid requirements, ,
such as access to historical data and access to
detailed payroll data, that must be replicated in
any solution.
2. The value is a "gross," not a "net" value. Any
eventual solution meeting EPA' s requirements will
itself incur processing costs. OIG has not
estimated the likely offsetting costs.
3 . OIG performed no tests of the accuracy of the
processing costs identified for the legacy systems.
BPA budget plans and submissions prescribed by OMB
Circular A-ll, Exhibit 43A are understated, and thus
Congress and OMB have not been adequately informed. For
instance, SPA has reported to OMB that IFMS obligations
are approximately $7.4 million a year, whereas our life
cycle approach estimates the cost to be at least $16
million a year
We do not believe that we are understating our costs and
specifically object: to the implication that we are being
purposely misleading. Rather, we believe we have
correctly interpreted the guidance on Circular A-ll as
stated on pages 155 and 156 of the Circular. Although
information system costs are non allocated to specific
systems, EPA in fact reports all of its information
systems costs to OMB.
Recommendations
Printed June 8, 1994
77
Report No. E1NMP3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX I
P5112
See Appendix II
Note 8
Appendix
III
See Appendix II
Note 2
Appendix
VI, VII
See Appendix II
Note 37
Appendix
VIII
See Appendix II
Note 36
In addition, after checking with other agencies (IRS and
Department of State) we have found out that they track
their financial systems costs similar to the way EPA does,
i.e., timeshare is reported as an aggregate and not
tracked separately for specific systems.
dissemination of IFMS information that is currently done
through layers of management would be more efficiently and
effectively done through e-mail.
DIG continually overlooks the purpose of IFMS coordinators
in disseminating information. The Coordinators' job is to
meet local information needs at the local level. They are
hardly "layers of management."
IFMS MANAGEMENT STRUCTURE
Show a single Branch-level box on the Organization Chart
for the Administrative Systems Division, as has been done
for every other OARM Division. The IFMS Charter uses a.
single point of contact in each division; within ASD, it's
th« Client Support Branch, CSB.
ESTIMATED ANNUAL AND TOTAL COST OF MAINTAINING LEGACY
SYSTEMS...
He do not concur with the computation or usage of the
purported "costs of legacy systems."
Elsewhere the OIG team acknowledges that BPA's current
billing systems are not adequate to allocate processing or
telecommunications costs among systems.
The implication is that the costs, however computed, are
entirely redundant. That is not the case. The
predecessor systems discharge valid EPA requirements in
the main. Further, to replicate those requirements
outside their current systems will incur costs equal to,
and perhaps larger, than the current costs.
SUMMARY OF IFMS LIFE CYCLE COSTS
We have never concurred with the values presented in this
exhibit or in earlier draft Position Papers.
Recommendat ions
Printed June 8, 1994
78
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX I
Appendix
VIII
[Appendix II
36
Recommendat ions
We AGREE that the costs to operate, maintain, and enhance
any financial system, such as IFMS, are substantial and
span many years.
We DO NOT AGREE that those values represent activity that
is somehow excessive -or wasteful.
Projected coats do nob reflect present value methodology
or inflation. We did not perform a detailed review of the
sources or systems which generated these figures for
accuracy and reliability.
-------
EPA's IFMS
APPENDIX II
PIG EVALUATION OF AGENCY COMMENTS
The following notes present the OIG's response to, certain portions
of the June 15, 1994 response from the AA for OARM.
Note;
We have made tremendous efforts and progress to
accommodate the Agency concerns and come to agreement on
the findings and recommendations. We issued three
position papers in August 1993, November 1993, and
January 1994 which formed the basis of the formal draft
report. We received written responses from your staff on
each of those position papers and conducted over a doz'en
meetings in order to discuss proposed changes to the
findings and recommendations in the position papers. As
a result of those meetings, we made many adjustments to
our findings and recommendations, where justified, in
order to accommodate. EPA management concerns. In
addition, we met with the Comptroller and the Director,
OIRM in January 1994, and at their request we issued a
discussion draft report in March 1994, thereby delaying
our formal draft report. .We received a 3-page response
to our discussion draft report in April 1994. As a
result of EPA management concerns expressed in that
response, we made further adjustments to our report, and
issued the formal draft report in May 1994. We
subsequently received your 29-page response and have made
further adjustments in this report.
We recognize that the Agency has achieved numerous IFMS
successes during the period of our review, and we have
updated this report to reflect those successes. The
following chart lists the major system accomplishments
since our field work ended in December 1993. We are very
pleased that significant actions have been taken to
address concerns raised during our audit. We did not
attempt to overlook the many recently completed actions,
many of which were completed after we issued the 'draft
report. Although many major system projects are started,
they still need to be completed over the next five years
to fully implement needed- Agency functions in IFMS.
80
Report No. E1NMF3-15-0073-4100SS1
-------
EPA's IFMS
APPENDIX II
These projects include elimination of legacy systems,
implementing a new account code structure, installing
fixed asset accounting, and project and cost allocation
accounting, electronically bridging IFMS with other
systems, and enhancing MARS.
ACTIVITY COMPLETED
Updated IFMS Requirements Analysis
Updated System Cost Study
FFS Upgrade to Version 5 . le
Cost -Benefit Analysis For Fixed Assets
Module in IFMS and Concurrence for
Implement at ion
Revision of IFMS Charter Setting out Roles
and Responsibilities
Implementation of new CMS system
Decision Memorandum IFMS Directions for
Next Seven years
Decision Memorandum for Elimination of
ADCR
Memorandum on Status and Implementation
Plan for Implementing a Enhanced Fixed
Asset Subsystem in IFMS and Reconciling
Data in PPAS
DATE-
January 10, 1994
April 25, 1994
April /May 1994
May 2, 1994
May 17, 1994
May 24, 1994
June 20, 1994
July 21, 1994
July 26, 1994
The report states that ineffective implementation of IFMS
has contributed to data integrity problems. We did
eliminate in our report references to other OIG reports
which cited data integrity problems. Although it was not
our objective to perform a data integrity review, we
still address data integrity problems identified that
were system related.
Automated interfaces have been reported by the Agency as
a material nonconformance under FMFIA since 1985.
81
Report No. E1NMP3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
Further, the Agency's CFO Financial Management Status
Report and Five Year Plan dated August 1993 which stated:
In the next five years...The Agency will eliminate
reliance on predecessor systems...[and]...
electronically bridge the Integrated Financial
Management System (IFMS) with other Agency financial
and administrative systems to eliminate duplicative
data entry and inconsistent application of terms.
We are simply restating known IFMS problems reported
under FMFIA by the CFO and the CFO's plans to correct
them. We are. not trying to give the impression that
theour report to dispel that impression and to give
examples of automated interfaces with IFMS (e.g., EPAYS
and CPS). However, we still have serious concerns about
fully interfacing IFMS with other systems such as GIGS'",
PPAS, CERCLIS, and CIS/APDS. This cannot happen until
the financial systems consistently define terms with a
common data dictionary across the systems. Without an
adequately implemented data dictionary, EPA can not
effectively address redundancy or share data among
different systems. Additionally, FFS still needs to be
directly interfaced with EPAYS, RMIS, and Superfund Cost
Recovery Image Processing System (SCORE$). Currently,
one legacy system, FMS, functions as the system interface
for these Agency subsystems.
We have made adjustments in Chapter 3 of our report to
eliminate the inference that the funds are "wasted" or
unnecessary. - Our point in that chapter is that EPA
management does not have cost information needed to make
informed system decisions. Tracking and reporting of
IFMS costs would provide management critical information
for prioritization, planning, evaluation of progress,
accountability, and performance measurement.
We did not change the basis of the estimates presented in
the 1991 Audit Report. Our estimate for IFMS development
and enhancement costs of $17.2 million (see Appendix XI)
were derived from the 1991 IFMS audit report and
subsequently updated by the Agency.
We also reported the operations and maintenance costs of
$11.2 million {see Appendix XII) for the predecessor
systems because these were other costs not originally
planned in the initial estimate. The predecessor systems
82
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
10
11
12
were originally planned to be eliminated in September
1989.
We adjusted the report to clarify that the total. $17.2
million estimate presented in our report covers costs to
fully implement IFMS .{i.e./ IFMS development and
enhancement costs) and the $11.2 million (less the some
offsetting costs) to maintain and operate the predecessor
systems (i.e., from 1990 until 1995). We clarified that
some offsetting costs would be incurred "when processing
is shifted to IFMS.
The objective of our review was to assess the current
level of user satisfaction. Although we have no evidence
to support a "dramatic" improvement in customer
satisfaction since 1989, the Director IFMS Project
Management Staff said that in his opinion there has been
dramatic improvement.
We did not intentionally leave out coordinators. We
adjusted our report to include coordinators. However,
the point we are making is that IFMS managers and
coordinators are not able to communicate with all users
primarily because TSSMS cannot provide a complete list of
individual users- of a particular financial system. This
was evident based on comments from our customer survey.
We are recommending that IFMS user list be provided to
NDPD to update TSSMS accounts, so that IFMS information
can be provided to all users. We are not advocating
elimination of coordinators.
The comment is provided as background to enable the
reader to have a historical perspective of IFMS. OIG
reporting standards (Chapter 110, OIG Manual) requires
that our reports present a background section which
includes historical information about the activity and
operations of the subject being reported upon.
We agree with the Agency comments and we adjusted our
report accordingly.
We agree with the Agency comments and have eliminated in
our report the references to the June 1993 report.
.We adjusted the report to reflect that the "additional
several million dollars" is a potential cost instead of
an expected actual cost. However, conversion of data
from the legacy systems may be required and the costs for
83
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
13
14
15
this process, which have not been estimated, could be
extensive. For example, 22 of the 47 systems have
requirements for historical data, which- would normally
require the conversion of data. Yet, a final decision
has not been made on.how the Agency will handle this data
when FMS is eliminated.
We still contend that IFMS management authority is
fragmented for the reasons stated in our report. This is
also the opinion of the OMB review team, which concluded
in their September 21, 1993 report on IFMS that there was
no clearly established office or individual accountable
for ensuring that(decisions from the planning process are
effectively carried out. The report further stated that
EPA needed to: (a) establish clearer accountability for
IFMS development and implementation under the
Comptroller's Office; (b) empower the Director, IFMS
Project Management Staff to exercise line authority over
major projects in the different working groups; (c)
ensure that the Comptroller's Office, as a major
representative of the CFO and Deputy CFO, exercise
greater leadership in leading financial systems
improvement projects.
We agree with the comments made, which supports and
strengthens our point. We recognize that the Agency may
have some unique needs, but it still needs to address
those needs.
We agree with the actions the Agency is currently taking
regarding the Project and Cost Allocation Accounting
module. Still, a requirements study has not been done
and no decision has been made about how the Agency plans
to address the need for project and cost allocation
accounting. Although the 1993 CFO plan calls for a cost
accounting requirements study to be completed by October
1994, the Director, IFMS Project Management Staff was net
sure when the issue would be ready for a decision
memorandum. The Director also stated that the Agency
does use the software "Reporter" and "RDM" which replaced
"RDS", but this software is not considered part of FFS.
Regarding OARM's response about the payroll module, we
were told by a management official that CPARS was
installed in lieu of the AMS payroll module, but PARS
(CPARS predecessor system) was in place prior to the
development of IFMS. CPARS replaced PARS three months
84
Report NO. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
16
17
18
19
20
after the other IFMS modules were implemented. Thus, EPA
procured the AMS payroll module, but did not implement
it. • •
We agree with the Agency's proposed actions regarding
recouping costs and maintenance fees for unused modules
of commercial software and the termination of maintenance
payments on IMAGINE. In addition, we are aware of the
fact that the contract maintenance costs are not listed
separately for each item of software procured. We
obtained an inventory of software from AMS of FFS
software EPA has purchased and installed and is
maintaining. We also have the best and final offer for
AMS's contract, which provides detailed cost information
on maintenance costs listed in the contract. We informed
OARM staff in the Spring 1994 that this information was
available in our workpapers for your review at any time.
Our point was that EPA still had not implemented a
reporting system which satisfied the Agency financial
reporting requirements. Neither MARS or Reporter had the
capability for detailed payroll reporting or historical
data reporting. Subsequent to our audit, we were told
that MARS had been modified to view payroll data.
Actually, the Agency planned to replace the account code
structure was made prior to 1989 as a result of two
studies completed in 1988. Furthermore, one of the
twelve key requirements used in 1985 study justifying the
replacement of the legacy systems was the ability to
replace the 10 digit account code structure with a more
comprehensive structure. Nevertheless, we agree the more
current decision to proceed with expanding the account
code structure. In addition, we made an adjustment in
our report to reflect that decision and our agreement.
We agree with the actions that the Agency, has taken
regarding IFMS data standards. However, we still did not
see any plans for a single, comprehensive data dictionary
for IFMS and other financial systems, which is critical
to the management of an integrated system.
The three other systems identify, track, and correct many
operational IFMS problems. Some of these problems result
in change requests that eventually are under CMS. Once a
proposed change has been approved by the change control
85
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
APPENDIX II
21
22
23
24
board, it would clearly fall under the configuration
management process in CMS. We are not advocating
elimination of these three systems, but it should be
clear when the Change Control Board•approves a change, it
becomes a system change request controlled under CMS.
The-OIG staff has not taken a formal position on removing
IFMS from the OMB High Risk List. We still consider IFMS
a high risk, at least at an Agency level, primarily until
the legacy systems have been eliminated. Even the
Director, FMD also stated in October 13, 1992 memo that
"...support for FMS is becoming increasingly tenuous...if
there were a catastrophic failure to FMS, EPA would be
unable to access critical data for an indefinite
period... restoration would be very difficult... major
functions could not be performed... Risk of a
catastrophic failure increases the longer we rely on the
[legacy] system."
We agree with the actions that the Agency has taken and
deleted the reference to FMS/PPAS.
We agree that only three accounting system noncon-
formances were reported in the Fiscal 1993 FMFIA.
However, a fourth nonconformance for EPAYS was reported
in fiscal 1985 with a planned corrective action of fiscal
1989. Yet, the EPAYS nonconformance was never reported
as corrected. Three other nonconformances were reported
as corrected during the period 1983 - 1989. We adjusted
the report to clarify this point.
System life cycle is a common system development term
used in the Federal Government as well as in private
industry. We recognize that variances of life cycle
exist between different systems. In the case of IFMS,
the Agency had not documented or estimated a life cycle
period. Therefore, we used a 12-year period for IFMS,
which was cited in the IEEE article as an average system
life cycle period. This estimate is also supported by
the April 1994 IFMS cost study which projected costs to
2001 (12 years since original implementation). In
addition, the August 2, 1994 advance copy of EPA's new
System Life Cycle Management policy requires the
establishment of defined life cycle period to calculate
total life cycle costs. The policy cites 12 years as an
average system life cycle period.
86
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
25
26
27
We recognize (see Note 2} that several significant IFMS
improvements and actions have occurred since January 1994
and we reflect the implementation of Version 5.1e several
places in the report.
The concept of continuously upgrading to newer
generations of software "Versioning-Up" may be possible.
However, unless the system remains "vanilla" {i.e.,
little or no customization), we believe It will be at an
unacceptable cost and use excessive resources. EPA does
not have a history of limiting customization (e.g., an
average of over 3,000 changes to IFMS per year, a portion
of which were customizations). In addition, as
•technology rapidly changes software and hardware, newer
versions will make existing FFS software obsolete. For
example, when AMS completes developing'"System 2000," it
may become impractical and cost-prohibitive to continue
supporting EPA's non standard FFS version. Furthermore,
System 2000 is a client-server based system (i.e.,
distributed processing), which is the type of environment
in which most of EPA currently operates in or is evolving
to. If EPA transitions to System 2000, that would be a
new generation (the end of the system life for FFS).
We define the term, "top management," as the DAA level
and above. We have footnoted that in this final report.
We recognize that top management leadership and direction
over IFMS has been provided, especially over the past
year. Our point in the report is that it was not enough
in the past and the current level of top management
attention needs to be continued to successfully implement
and integrate IFMS with other Agency financial-related
systems. We disagree that our statement is not
supported. In our report, we give the following
examples:
• The ASC meeting minutes dating back to 1990 showed
that the DCFO never attended a Council meeting, and
that the former DAA for OARM (who was the designated
chair-person) has not attended a meeting since
November 1990. The ASC was charged with providing
oversight for IFMS implementation.
87
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX II
28
29
30
31
32
• Based on a review of the IRM Steering Committee
minutes and documents of meetings .prior to May 1994,
we identified that IFMS has not been a topic of
discussion since 1987. The principal members of the
current IRM Steering Committee include the Assistant
Administrators and the Regional Administrators.
' We were told by the Director, IFMS Project Management
Staff that the DAA for OARM was involved with the IFMS
decision-making as early as May 1990. However, we were
unable to find any official documents to supporting the
DAA's involvement in any decisions as early as February
1990.
The current Executive Steering Committee for IRM does not
specifically maintain oversight over IFMS. The ASC
oversight duties were removed from the IFMS Charter with
the new charter dated May 17, 1994. The current IFMS
Charter establishes oversight responsibilities in the
EMG, which is a lower level,group {e.g., Division
Directors and below) chaired by the Director, IFMS
Project Management Staff. This group makes
recommendations through the chain-of-command to the CFO.
We adjusted the report to reflect the changes which
occurred after December 1993.
Our point was that the IFMS Charter dated September 14,
1990 was established during our prior audit which was
issued in 1991. Some of major elements of that charter
responded to the concerns we had raised during the course
of our prior audit.
We agree with the Agency's comments and we adjusted the
final report accordingly.
The SMG annual spending plan has only been used for
short-term planning. Traditionally,•it has only dealt
with spending for current operational needs and
enhancement projects for Office of the Comptroller and
Office Information Resources Management.. In fiscal 1994,
some National Data Processing Division costs were also
included, but the annual spending plan has never
attempted to identify all system resources.
Neither the annual spending plan nor the budget use a
SDLC based cost model. The cost analysis could be a
88
Report No. E1NMF3-15-0073-4100561
-------
EPA'e IFMS
APPENDIX II
33
34
35
basis for long term planning, but it should be integrated
into the Strategy and Master Work Plan. OMB, in the
September 1993 report on IFMS, recommended using the
strategic plan to drive the budget process and that EPA
request special funding where the strategic plan suggests
resources were not available to meet the long term goals.
The Agency's current Strategic Plan does1 not include an
analysis of resource needs for the goals identified over
the next five years. Unless the IFMS Strategy and Master
Work Plan includes estimated resources, management has no
assurance that the current stable funding approach will
provide resources to accomplish planned projects as
scheduled and to correct FMFIA weaknesses.
We dropped this comment from the final report.
Our point is that although EPA contracts do include a
general clause requiring contractors to comply with
Agency policies, specific work requests are written to
allow utilization of multiple methodologies. Without a
standard method to control changes and document
enhancements, the Agency will not be in position to
adequately manage IFMS's baseline software. In our
opinion, using multiple methodologies is unacceptable
because it raises the risk that the Agency could become
hostage to that contractor's process or lose control over
changes to mission critical baseline software. We
interpret the Agency comments to be that ASD intends to
utilize the methodology delineated in the new Agency
System Development Life Cycle Policy for IFMS
development. We further interpret the Agency comments to
be that ASD also intends to specify requirements to
follow the Agency's SDLC methodology in the contracts'
statements of work.
The numbers provided are based on our discussions with
the software vendor AMS. Obviously, we agree that
different parts of the EPA baseline are customized more
than the 29 percent and other parts are significantly
less. Although the FFS baseline is most of IFMS,
additional modules for such as project, cost allocation
and fixed asset accounting functions are scheduled to- be
installed which could involve further customization. In
addition, functions performed by the legacy systems need
to be incorporated into FFS.
89
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
36
37
Nevertheless, our overall point is that. EPA has been
over-customizing FFS -and customization of modules in the
FFS baseline software is a high risk activity. While
Agency officials do not agree with percentage of
customization, they agree that customization needs to be
limited.
Our analysis was pointing to the lack of cost information
for management to use in its planning process. In the
absence of an Agency estimate for IFMS cycle costs, we
chose to estimate those costs in a conservative manner.
While the OARM challenges our estimate, we need to note
that the Agency's April 1994 Cost Study yielded much
higher IFMS life cycle cost estimate. The Agency agrees
that it will cost a substantial amount to maintain and
operate IFMS.
However, our point is that management needs to be aware
of these costs in order to make informed decisions
regarding future directions regarding financial systems
for the Agency. We addressed total estimated cash
outlays which would show cost trends throughout the
remainder of the system's estimated life. We were not
performing a cost-benefit analysis per OMB Circular A-94
which defines net present value as the discounted
monetarist value of expected benefits (i.e., benefits
minus costs). The April 1994 Cost Study did perform an
estimate of the future costs, with adjustments, based on
several assumptions which subsequently adjusted future
cost estimates to fiscal 1994 dollars. Both our analysis
and the Cost Study used comparable cost figures for
fiscal 1994. However, we did not use the same method for
projecting future costs because we were not performing a
cost-benefit study. The .Cost Study costs "for "stay the
course" for fiscal 1994-2001 was $164.6 million, which
was significantly higher than our $123.5 million estimate
based on our projections using cost trends. Our
projections were only intended to be conservative
estimates of IFMS's SDLC costs.
We eliminated the term, "unnecessarily," from the
referenced section in our final report. Nevertheless,
the legacy systems duplicate much of what the new system
90
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX II
38
performs. Additionally,-it would have been cheaper to
incorporate the few functions needed in 1989 and
eliminate the legacy .systems at that point, because it
costs significantly more to retrofit by customizing the
system.
We reported that management did not report to OMB and
Congress all IFMS life cycle costs as prescribed by OMB
Circular A-11, because EPA has not been accumulating
total life cycle system costs. In addition, we verified
with OMB that system timeshare and telecommunication
costs should be separately reported, which is contrary to
the OARM's interpretation. Also, since 1989, the
Agency's system development requirements require that
these costs be accumulated. Further, the Agency's recent
revision to Chapter 17 of Directive 2100-defines system
life cycle costs as including timeshare and telecommuni-
cations costs. Chapter 17 also establishes approval
thresholds for system plans based on scope and cost
criteria. In fiscal 1996, a WCF for the timeshare and
telecommunications costs will bill these costs to each
program office's budget.
91
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX III
ACTIONS TAKEN ON 1991 PIG IFMS REPORT RECOMMENDATIONS
REPORT ON INTEGRATED FINANCIAL MANAGEMENT SYSTEM: MANAGING
IMPLEMENTATION OF THE NEW ACCOUNTING SYSTEM (Audit Report. No.
E1AMF011-0029-1100153] March 29. 1991
OARM agreed to all the recommendations in the March 1991 audit
report. Based on our review, we concluded that four were
partially implemented and nine were implemented. The
recommendations from the audit report appear below, and a status
of each follows:
Recommendations and Status: ;
1. Recommendation: Formally establish the proposed IFMS
Project Management Staff.
Status: Implemented. The IFMS Project Management staff
(one position) and the IFMS Project Manager was
created informally in 1990 and reported to OIRM. As of
October 1992, the Director, IFMS Project Management
Staff (formerly the IFMS Project Manager) and his staff
report to the Office of the Comptroller.
2. Recommendation: Fill the positions of the IFMS Project
Management Staff.
Status: Partially Implemented. Two of the five full-
time positions on the project were filled--the
Director, IFMS Project Management Staff and Senior
Management Analyst. The remaining positions were never
filled.
3. Recommendation: Require the updating of the needs
statement, feasibility study, and the cost-benefit
analysis (including an analysis and comparison of the
current direction of IFMS with other alternatives).
Status; Implemented. Inventory of Additional Needs
documents were prepared based on user input (1990 and
1991). Draft and Final Strategy and Master Work plans
were prepared which outlined broad needs. A
requirements analysis was completed in January 1994. A
cost analysis was completed in April 1994.
92
Report No. E1NMF3-15-0073-41005S1
-------
EPA'8 IFMS
APPENDIX III
8,
Recommendation: Revise the project plan to include the
updated information from the needs statement,
feasibility study, risk analysis, and cost-benefit
analysis.
Status; Partially Implemented. The Strategy and Master
Work Plan was approved in-July 1993; a risk analysis
was completed; and some user needs were solicited.
January 1994 the IFMS requirements ana-lysis was
revised. In April 1994, the IFMS cost analysis was
revised. However, the IFMS Master plan dated July 93
has not been updated to reflect the revised .
requirements and cost analyses done in 1994.
Recommendation: Require the completion of a formal
system decision paper based on the above documents and"
other available information, and approve or disapprove
the system decision paper.
Status: Implemented. A decision memorandum was
approved by the CFO in June 1994 that was based on the
April 1994 cost analysis and January 1994 requirement
analysis.
Recommendation: Promulgate formal EPA guidance
regarding the system decision process during the
development and implementation of large information
systems.
Status: implemented. In August 1994, Chapter 17 of the
IRM Policy Manual was revised and clarified policy on
the use of the system life cycle approach and the
system decision paper process.
Recommendation: Develop comprehensive test plans for
all future IFMS modifications and modules.
Status: Implemented. The Agency has required test
plans for IFMS modifications and modules.
Recommendation; Fully document test results and
findings including the identification of actions and
milestones for the correction of deficiencies.
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFHS
APPENDIX III
9.
10
11
12
Status; Implemented.' The Agency has documented test
results and deficiencies found in the commercial
software.
Recommendation; Require formal acceptance of software
developed or provided by the contractors before
approving final payment.
Status: Implemented. OARM officials monitor testing
within the CMS and require system decision papers to
activate the subreleases. EPA has not received or
accepted, any additional commercial software modules
from 1991 to December 1993.
Recommendation: Perform a review to ensure that IFMS
contains an adequate level of security.
Status: Implemented. Reviews of IFMS have been
performed including a IFMS risk analysis and several
reviews at NDPD reviews. Implementation of security
measures and safeguards are ongoing.
Recommendation: Emphasize the need for preparing and
updating system documentation, preparing options
analyses, and providing users with adequate training
and documentation.
Status; Partially Implemented. MARS user manual has
been updated, and revisions have been made to IFMS user
documentation regarding 5.le. The upgrade to version
5.le included updating the IFMS users manual and
training users. Training and documentation for users
appears adequate, however, communication of the
availability of training and documentation could be
improved.
Recommendation; Ensure that IFMS documentation is
centrally maintained and notify users of the
availability and location of documentation.
Status: Partially implemented. While user
documentation is centrally maintained, notification of
user documentation availability could be improved.
94
Report No. E1NMF3-15-0073-410056:
-------
EPA'S IFMS
APPENDIX III
13.
Recommendation: Involve all user groups in developing
and implementing the Agency's financial management
system, and document that their needs and priorities
were considered in deciding the direction and plans for
the system.
Status; Implemented. Users have been involved in
developing and implementing IFMS (through strategic
planning sessions), and needs and priorities have been
documented. However, by identifying additional users,
greater user input could be achieved.
95
Report NO. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX IV
UJ
w *
£z
LU
O UJ
z5
^
H
U.
o
CO
GEN. LEDGER
BUDGET EXEC.
• BUDGET FORM.
• A/P
' A/R
» TRAVEL
• PURCHASING
"Z.
O
CO
a:
UJ
>
CO
U. O
U. rn
•
• FIXED ASSETS
• PROJ. ACCT.
• PAYROLL DIST.
• STD REPORTS
a.
o
CO
O
a.
o
CO
o
5
CO
<
Q.
a.
96
Report No. E1NMP3-15-0073-4100561
-------
EPA's IFMS
APPENDIX.V
97
Report No. E1NMF3-15-0073-4100561
-------
EPA'e IFMS
APPENDIX VI
111
oc
u
DC
2 3
5fc
U) t
I
tti
v>
a
o
U)
m
m
&
aig
w
<
<
<
(0
LL
i
j?
u.
O
A
<
S u.
Q ""u
SM
^5 ^j y* O
^r **• O *C
Q Z
i
a
2
u
C
u
L
u
C
2
a
«
C
i
i
i
j
•
)
i
)
•
•
)
•
£
•
>
__^
^^
«
a
u.
o
CO
O
03
u-
^^^^^^^
cn
m
*
i
a
S
4
e.
t
*
CQ
tn
U
1
a
V)
a.
H
2
as
O
a.
Q
z
I
1
I
Ifl
o
e
u
O
3
K
e
I
2
1
98
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 1FMS
APPENDIX VII
8 '-3
rt -«;.•;»
o :••-;••- '• .'.;«•>;.:
-
o« at
0 in
0 •«
«n.
o
£'
£
S3
sa
2-
(A
^ eo
q 0 eo
oi d s
o> >- o
r^ in 0
O? IN »-"
«>«>«>
o 10 «-
«N «o o>
(0 «- CD
r> ta co
» - »
'
q
0
9
o
q
d
q
d
q
d
q
d
-«o
q os
§ o
in
^' »
;'»
to
q
u>
.V
•0
0
°
0*
0
•^
m
in
0
VO
0
0
PJ
o>
m
O)
«> «>
oT
q
d
in
q
d
q
d
in
i
in
0
P)
O>
in
01
oi
0
0
q
0
0
m
0
oT
q
o
0
a
99
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
-
-J|' in m «
bwj 00 CM ' *••
j^| ••»•«•> en
5 S"
CM if) ro co
SI CO CM d
Mjl C") ^^ **"
S •". «8 ^.
£ . s s s
M
f-CM O)| O
O35 *°j £ *" O 10
^_ jlll ^ CO O fli
DC ^t ^>t ^ ^ ^ ^
5^iu 5 *> ** *>•
Q 1/1 f"*| Q *^ O) O
L *^ ml rv oo o
^ t/) T O co cp
g5 "' » « r.
5= 2 Z S
£1 co iv 06
Jill *r> f- oo
1 in o> q,
c«j «^ rJ
«> *> «»
SI r";;^ ^
21 *- •** • *4
f> . ;'*^ O*..
<•} -:^l •: ."^
4» ... rf»'- -^4fr ..'
"; 1 I ^
1 ! . 1 ••
H 5
§ 1 -2 gj
' ^ "S C 5
S 1 ^ i
':'5:. ^^ g* 86
V O O Ch
T» in ^
P1* O) C*5
(O ^^ C*Q
to ft d
co to in
en q »-
* 06 in
5 . i 8*
CO C) "i""*
•*• co ui
r- rs. CM
". « "X
in ^t m
0 q -;
^f co in
m r«. CN
w" V m
": P r
«f> co in
o r» CM
00 V P>
q q «-.
in eo in
>
r»~ V m
CM q *-.
^ co in
APPENDIX VII
'
%
»
1
'&
~f*
:«
— :E
> S
x ^
^ #
s. &
^ a
'o
° Ss
« ;E^
•g il
" -ss
*! i«
ill
f 15
> if
'O .p:^
£ ''**'
I i
CB i'C'
A} :*^
* u.
100
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IPMS
APPENDIX VII
METHODOLOGY
Life Cycle Costs Basis
Our analysis concentrated on gathering system life cycle cost
figures for IFMS over an estimated 12 year system life (1989-
2001). There was approximately a two year (1987-8) development
phase prior to implementation in 1989. The cost figures also
include costs of the other Core Financial Systems (i.e., FMS, •
ADCR, RMIS, CPARS, and MARS) incurred within the IFMS estimated
system life. Our life cycle cost analysis was based on the
system life cycle phases, and is consistent with OMB circular A-
11, A-109, A-130 and applicable FIPS Publications. We estimated
the 12 year system life for IFMS based on an Institute of
Electrical and Electronics Engineers (IEEE) publication entitled
"Software Lifetime and its Evolution Process over Generations"
dated 1992. We note that the IFMS life cycle cost figures do not
include all costs (i.e., headquarter, regional, lab costs) over
the estimated system life as outline in the finding. This
approach has resulted in a conservative estimate of total life
cycle costs. Projected costs do not reflect present value
methodology or inflation. We did not perform a detailed review
of the sources or systems which generated these figures for
accuracy and reliability.
Specific Cost Elements Methodology
We used official EPA cost figures from OMB Circular A-ll, 43A
Exhibits, Financial Management Status Reports and Five Year
Plans, and INFOPAC billing reports. The shaded numbers on the
Exhibit reflect these official costs. We did not include cost
estimates of FMS, RMIS, ADCR, MARS, and CPARS prior to 1989. We
included 1987-1988 costs for IFMS as reported for the initial
development of the system. The projected 1989 figures for IFMS
was prorated 7 months—March 1989 to September 1989. We assumed
that IFMS would not be eliminated until the end of fiscal 2001.
We did hot factor in increased IFMS costs, once the legacy
systems are eliminated. This would effect all cost elements.
Planned elimination of ADCR in 1994, and FMS and RMIS in 1995
were taken from the 1993 Integrity Act Report To The President
and Congress. The projections for these systems were prorated
accordingly.
Cost Elements Descriptions
Development/Enhancements: The 1987 figure represents the
original AMS contract for the purchase of the off-the-shelf
101
Report NO. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX VII
software. The 1988-1990 figures were taken from the Financial
Management Plans dated September 1988, October 1989, and
September 1990. The 1991-1995 figures were taken from the OMB
Circular A-ll, Exhibit 43A submissions to OMB. The remaining
years were projected using Lotus 1-2-3 linear regression analysis
based on the Exhibit 43A submissions, with the last 3 years (1999
- 2001) reflecting the end of the life cycle. The projected
costs for the last 3 years were prorated 50 percent, 25 percent,
and 0 percent respectively.
Operations and Maintenance: We did not estimate the cost for
1987-1988 since IFMS was not put into production "until 1989. The
1991-1995 figures were taken from the OMB Circular A-ll, Exhibit
43A submissions to OMB. The preceding and remaining years were
projected using Lotus 1-2-3 linear regression analysis based on
the Exhibit 43A submissions.
Personnel; We did not estimate the cost for 1987-1988 since IFMS ,
was not put into production until 1989. The 1991-1995 figures
were taken from the OMB circular A-ll, Exhibit 43A submissions to
OMB. The preceding and remaining years were projected using
Lotus 1-2-3 linear regression analysis based on the Exhibit 43A
submissions.
Subtotal; The subtotals for 1991 - 1995 agree with the OMB
Circular A-ll, Exhibit 43A submissions. They reflect costs for:
one Responsible Planning and Implementation Office (RPIO) — OARM
(16); 2 allowance holders (OIRM (55) and OC (42)); and 4 program
elements within each allowance holder. The other years were
projected as explained above. These costs reflect actual and
estimated total development, maintenance, operations, and
personnel within OIRM and OC.
Timeshare; We did not estimate the timeshare cost for 1987-1988
since IFMS was not put into production until 1989. The 1991-1993
costs were taken from the INFOPAC Fiscal Year To Date statements.
INFOPAC is an NDPp billing report system for timeshare charges.
Timeshare (which is an NDPD allocation process for their
information system expenditures) reflects usage of mainframe
Central Processing Unit (CPU), public and private disks, courier
services, and standard, foreign and archive tapes. These cost
were not reflected on the OMB Circular A-ll, Exhibit 43A
submissions for IFMS. The preceding and remaining years were
projected using an average of the actual timeshare costs (1991 -
1993) . Since we only had actual timeshare costs for three years
and a dramatic decrease in costs/rates for the third year (1993)
of 34 percent, we did not use linear regression analysis.
102
Renort No. E1NMF3-15-0073—4100561
-------
EPA'S IFMS
APPENDIX VII
Telecommunications/Administration: We did not estimate
telecommunication/ administration costs for 1987-1988 since IFMS
was not put into production until 1989. We projected the 1989 -
2001 costs using the methodology currently used by NDPD to
allocate these costs to the Interagency Agreement (IAG) and
Superfund accounts. IAG and Superfund accounts .have
telecommunication charges applied against them via the
"Telecomm/Administration" category on the INFOPAC report.
Telecommunication charges are allocated based on a percentage of
the total monthly timeshare usage. This charge includes
telecommunications as well as planning and acquisition, e-mail,
information centers, and other items. The percentages applied
against IAG and Superfund for FY 1991, 1992, and 1993 were 69.1
percent, 69.1 percent and 100.7 percent respectively. We applied
the same percentages for these years. For the preceding and
remaining years we used an average (79.63) of these percentages
to project the "Telecomm/Administration" costs.
Annual/Total Life Cvcle Costs: These costs reflect totals by
fiscal year and a total life cycle costs based on available cost
data.
103
Report MO. E1NMF3-15-0073-4100561
-------
EPA's ZFMS
APPENDIX VIII
Responses
Percent
POSITION:
Responses
Percent
ANALYSIS OF INTEGRATED FINANCIAL
MANAGEMENT SYSTEM (IFMS)
CUSTOMER SATISFACTION SURVEY
( 90% CONFIDENCE LEVEL )
Headguarters Region' Field
44 74 28
30.1% 50.7% 19.2%
Manager Supervisor Operations
10 12 99
8,3% 9.9% 81.8%
~ •- - — -J
Total
146
100.0%
Total
121
100.0%
How long have you used 1FMS?
Responses
Percent
QUESTION 1.
Number of
Responses
148
QUESTION Z.
Number of
Responses
146
less than 2 years More than a years
30 110
21.4% 78.6%
Indicate each system used and the frequency of use.
Never Rarely Often
0....1....2....3....4....5....6....7.1..8...I9....10
Job Function. Identify primary job/mission functions that
IFMS performs for you.
Number Percent
Planning 23 5.3%
Budgeting 41 9.4%
Monitoring 84 19.3%
Branca 82 18.8%
Data Entry 64 14.7%
Funds Cor.trol 70 16.1%
Reporting 72 16.5%
Total 436 100.0%
Tote]
140
100.0% .
Average
Response
<+/- 10%)
8.2
— — — — • •-• •>-- • - " — — • • — ..- — .
104
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX VIII
ANALYSIS OF INTEGRATED RNANCIAL
MANAGEMENT SYSTEM (IFMS)
CUSTOMER SATISFACTION SURVEY
(80% CONFIDENCE LEVEL)
QUESTION 2. (Continued) . How well does the system support you job function?
Number of
Responses
147
Never Satisfactory Outstanding
0....1....2....3....4....5....6.I7J..8....9....10
Average
Response
M-10%)
7.1
QUESTION a Accuracy. What describes your experience with Ihe accuracy
of system information?
Number of Average
Responses Never Usually Highly Response
Accurate Accurate Accurate {+/-10%)
147 0....1....2....3....4....5....6..|77].8....9....10
72
QUESTION 4. Response Time. Wrat describes your experince with the
on-line response time? Average
Number of . Response
Responses Not Acceptable Acceptable Outstanding M-10%)
147 7.0
QUESTION 5. Availability. What describes your experience of how the
system is available? Average
Number of Response
Responses Never Frequently Always (+/-10%)
148 7.8
QUESTION & Training. What describes how welt the training and
training materials met your needs? Average
Number of Response
Responses Unacceptable Acceptable Outstanding ' (+/-10%)
137 5.8
QUESTION 7. Documentation. Is the system user Aacumentation current?
Average
Number of' Response
Responses Not Current Mostly Current Completely Current (+/-10%)
129 5.9
105
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX VIII
ANALYSIS OF INTEGRATED FINANCIAL
MANAGEMENT SYSTEM (IFMS)
CUSTOMER SATISFACTION SURVEY
( 90% CONFIDENCE LEVEL )
QUESTION 8.
Number of
Responses
145
Usefulness. How useful is information from the following
system (IFMS) 7
Not Useful
Acceptable
Very Useful
Avenge
Response
7.6
QUESTIONS.
Number of
Responses
143
Ease ol Use. For the systems you use, how easy is it to use
the following?
Not Easy Easy Very Easy
0....1....2....3....4....5....O7....6....9....10
Average
Response
(+/- 1OTM
6.4
QUESTION 10.
Number of
Responses
78
Interlaces. What other financial systems/interfaces do you use?
Percent
CPS
ORDOPS
EPAYS
G1CS
CPARS
PPAS
APDS/CIS
Other
Total
13
5
36
14
26
6
9
22
131
9.9%
3.8%
27.5%
10.7%
19.8%
4.6%
6.9%
16.8%
100.0%
QUESTION 11. Operations. Whenlhe system is down, what alternative sources
of tie data are available?
•NOTE* The response rate did not meet an acceptable minimum
threshold, therefore tvs question has been dropped from
the survey.
106
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX VIII
ANALYSIS OF INTEGRATED FINANCIAL
MANAGEMENT SYSTEM (IFMS)
CUSTOMER SATISFACTION SURVEY
( 90% CONFIDENCE LEVEL)
QUESTION 12.
Number of
Responses
143
Customer Service. If you have operational problems, how do you
resolve them? (Indicate one or more.)
Hotline
Request a change to the system
Wait and try again later
Other
Total
Number
105
21
86
24
238
Percent
100.0%
QUESTION 12. (Continued)
Number of
Responses
134
How satisfied are you with the responsiveness of
customer service?
Dissatisfied Satisfied Always Satisfied
0....1....2....3....4....5....C3....8....9....10
Average
Response
M- 10%)
6.4
QUESTION 13.
Number of
Responses
135
Overall Assessment Overall, has IFMS improved during the
past year?
Worse This Year Satisfactory Major Improvement
Average
Response
M-10%)
6.1
107
Report No. E1NMP3-15-0073-4100561
-------
EPA'B IPMS
APPENDIX VIII
LOCATION:
Responses
Percent
POSITION:
Responses
Percent
QUESTION 1.
Number of
Responses
94
QUESTION 2.
Number of
Responses
88
ANALYSIS OF MANAGEMENT AND
ACCOUNTING REPORTING SYSTEM (MARS)
CUSTOMER SATISFACTION SURVEY
Headquarters
35
37.6%
Region
40
43.0%
Field
18
19.4%
Supervisor
12
14.8%
Operations
61
75.3%
Indicate each system used and the frequency of use.
Never Rarely Often
0....1....2....3....4....5....OI..8....9....10
Job Function. Identify primary job/mission functions that
MARS performs for you.
Planning
Budgeting
Monitoring
Finance'
Data Entry
Funds Control
Reporting
Total
Number
10
18
40
34
4
32
76
214
Percent
4.7%
8.4%
" 18.7%
15.9%
1.9%
15.0%
35.5%
100.0%
Total
93
100.0%
Total
81
100.0%
Average
Response
6.7
108
Report No. E1NMF3-15-0073-4100561
-------
EPA'e IFMS
APPENDIX VIII
ANALYSIS OF MANAGEMENT AND
ACCOUNTING REPORTING SYSTEM (MARS)
CUSTOMER SATISFACTION SURVEY
QUESTION 2. (Continued) How well does the system iupport your Job function?
Number d
Responses
63
Never Satisfactory Outstanding
0....1....2....3....4....5..CE1.7....8....9....10
Average
Response
e.o
QUESTION 3.
Number of
Responses
91
Accuracy. What describes your experience with the accuracy
of system information?
Never Usually Highly
Accurate Accurate Accurate
0....1....2....3....4....5...IO7....8....9....10
Av«r«g«
R>»POni«
8.2
QUESTION 4.
Number of
Response Time. Whet describes your experinee with the
on-line response time?
Not Acceptable Acceptable Outstanding
$5
QUESTION 5.
Number of
R«tpont«»
89
Availability. What describes your experience at how the
system is available?
Never
Frequently ' Always
Average
Response
6.1
Average
Response
7.7
QUESTION «.
Number of
Re»pon»e»
«7
Training. What describes how well the training end
training materiel* met your needs?
Unacceptable Acceptable Outstanding
Average
Response
8.1
QUESTION 7.
Number of
Responses
62
Documentation. Is the system user documentation current?
Not Current Mostly Current Completely Current
0....1 ....2....3....4....5....0--8--9--10
Average
Response
6.S '
109
Report No. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
APPENDIX VIII
ANALYSIS OF MANAGEMENT AND
ACCOUNTING REPORTING SYSTEM (MARS)
CUSTOMER SATISFACTION SURVEY ,
QUESTION 6.
Number of
Responses
90
Usefulness. How useful is information from the following
system (MARS) ?
Not Useful
Acceptable
Very Useful
Average
Response
7.4
QUESTION 9.
Number of
Responses
90
Ease of Use. For the systems you use, how easy is it to use
the following?
Not Easy
Easy
Very Easy
Average
Response
5.7
QUESTION 10.
Number of
Responses
46
Interlaces. What other financial systems/interfaces do you use?
Number Percent
CPS
ORDOPS
EPAYS
GICS
CPARS
PPAS
APDS/CIS
Other
Total
10
4
23
7
20
6
2
16
88
11.4%
4.5%
26.1%
8.0%
22.7%
6.8%
2.3%
18.2%
100.0%
QUESTION 11. Operations. When the system is down, what alternative sources
of the data are available?
•NOTE* The response rate did not meet an acceptable minimum
threshold, therefore this question has been dropped from
the survey.
110
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX VZXZ
ANALYSIS OF MANAGEMENT AND
ACCOUNTING REPORTING SYSTEM (MARS)
CUSTOMER SATISFACTION SURVEY
QUESTION 12.
Number of
Customer Service. If you have operational problems, how do you
resolve them? (Indicate one or more.)
93
Hotline
Request a change to the system
Wait and try again later
Other
Total
Number
76
20
54
17
167
Percent
45.5%
12.0%
32.3%
10.2%
100.0%
QUESTION 12. {Continued) How satisfied are you with the responsiveness of
.customer service?
Number of
Responses Dissatisfied Satisfied Always Satisfied
89
Average
6.1
111
Report No. E1NMF3-15-0073-4100561
-------
EPA'8 IFMS
APPENDIX VXZZ
.
LOCATION:
Responses
Percent
POSITION:
Responses
Percent
QUESTION 1.
Number of
Responses
66
QUESTION 2.
Number ot
Responses
63
f
ANALYSIS OP
FINANCIAL MANAGEMENT SYSTEM (FMS)
CUSTOMER SATISFACTION SURVEY
Headquarters Region Reid ' Total
19 36 10 65
29.2% 55.4% 15.4% 100.0%
Manager Supervisor Operations Total
5 6 42 53
9.4% 11.3% 79.2% 100.0%
Indicate each system used and the frequency of use.
Average
Never Rarely Often Response
0....1....2....3....4....5....OL.8....9....10
6.6
Job Function. Identify primary job/mission functions that
FMS performs tor you.
Number Percent
Planning . 5 3.5%
Budgeting 10 6.9%
Monitoring 27 -18.8%
Finance 35 24.3%
Data Entry ' 16 11.1%
Funds Control 10 6.9%
Reporting 41 28.5%
Total 144 100.0%
112
Report No. E1NMF3-15-0073-4100561
-------
EPA's XFHS
APPENDIX VIII
QUESTION 2.
Number of
Responses
65
QUESTION a
Number of
Responses
65
QUESTION 4.
Nurrberof
Resporees
61
QUESTION 5.
Number of
Responses
66
QUESTION 6.
Number of
Responses
53
QUESTION?.
Number of
Responses
53
ANALYSIS OF
FINANCIAL MANAGEMENT SYSTEM (FMS)
CUSTOMER SATISFACTION SURVEY
(Continued) How well does the system support your job function?
Never Satisfactory Outstanding
Accuracy. Whal describes you experience wrth the accuracy
of system information?
Never Usually Highly
Accurate Accurate Accurate
Response Time. What describes your expertnce with the
on-fine response time?
Not Acceptable Acceptable Outstanding
N
Availability. What describes you experience of how the
system is available?
Never Frequently Always
Training. What describes how well the training and
training materials met your needs?
Unacceptable Acceptable Outstanding
Documentation. Is the system user documentation current?
Not Current Mostly Current Completely Current
Average
Resessi
6.0
Average
Resconse
5.7
Average
Response
6.7
Average
Response
6.0
Average
Response
5.3
Average
Response
4.6
113
Report No. E1NMF3-15-0073-4100561
-------
SPA'S IFMS
APPENDIX VIII
ANALYSIS OF
FINANCIAL MANAGEMENT SYSTEM (FMS)
CUSTOMER SATISFACTION SURVEY
QUESTION 8.
Number of
Responses
63
Usefulness. How useful is information from the following
system (FMS) ?
Not Useful
Acceptable
Very Useful
Average
Response
6.7
QUESTION 9.
Number of
Responses
61
EaseofUse. For the systems you use, how easy is it to use
the following?
Not Easy
Easy
Very Easy
L..8....9....10
Average
Response
6.6
QUESTION 10.
Number of
Responses
38
Interfaces. What other financial systems/interfaces do you use?
Number Percent
CPS
ORDOPS
EPAYS
GICS
CPARS
PPAS
APDS/CIS
Other
Total
9
3
18
5
16
5
2
13
74
100.0%
QUESTION 11. Operations. When the system is down, what alternative sources
of the data are available?
"NOTE* The response rate did not meet an acceptable minimum
threshold, therefore this question has been dropped from
the survey.
114
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX VIII
ANALYSIS OF
FINANCIAL MANAGEMENT SYSTEM (FMS)
CUSTOMER SATISFACTION SURVEY
QUESTION 12.
Number of
Responses
63
Customer Service. Hyou have operational problems, how do you
resolve them? (Indicate one or more.)
Hotline
Request a change to tf* system
Wait and try again later
Other
Total
Number
51
15
38
10
114
44.7%
13.2%
33.3%
8.6%
100.0%
QUESTION 12. (Continued)
Number of
Responses
60
How satisfied are you with the responsiveness of
customer service?
Dissatisfied Satisfied Always Satisfied
0....1....2....3....4....SC3....7....6....9....10
Average
Response
115
Report No. E1NMF3-15-0073-4100561
-------
EFA'S IFMS
APPENDIX VIZI
LOCATION:
Responses
Percent
POSITION:
Responses
Percent
QUESTION 1.
Number of
Responses
52
QUESTION 2.
Number of
Responses
49
ANALYSIS OF AUTOMATED DOCUMENT
CONTROL REGISTER (ADCR)
CUSTOMER SATISFACTION SURVEY
Headouarters Region " Field Total
20- "25 3 48
41.7% 52.1% 6.3% 100.0%
Manager Supervisor Operations TotaJ
1 4 38 43
2.3% 9.3% 88.4% 100.0%
l
Indicate each system used and the frequency of use.
Average
N«fi»r Rarely Often Response
0....1 ...,2....3....4....5....6....7iZal..9....1 0
7.8
Job Function. Identify primary job/mission functions that
ADCR performs tor you.
Number Percent
Planning 5 3.4%
Budgeting • 20 13.4%
Monitoring 20 13.4%
Finance 12 8,1%
Data Entry 43 28.9%
Funds Control 27 18.1%
Reporting 22 14.8%
Total 149 100.0%
116
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX VIII
ANALYSIS OF AUTOMATED DOCUMENT
CONTROL REGISTER (ADCR)
CUSTOMER SATISFACTION SURVEY
QUESTION 2. (Continued) How well does the system support your job function?
Number of - Average
Responses Never , Satisfactory Outstanding Response
51 6.4
QUESTION 3. Accuracy. What describes your experience with Ihe accuracy
of system information?
Number of
Responses Never Usually Highly Average
Accurate Accurate Accurate Response
49 0....1 ....2....3....4....5....6.U71...8....9....10
6.9
QUESTION 4. Response Time. What descrtoes your experince with the
on-line response time?
Number of Average
Responses Not Acceptable Acceptable Outstanding Response
49 7.0
QUESTION 5. Availability. What descrtoes your eiperience of how the
system is available?
Number of Average
Responses Never Frequently Always Response
51 8.4
QUESTION 6. Training. What describes how well the training and .
raining materials met your needs?
Number of Average
Responses Unacceptable Acceptable Outstanding * Response
47 6.7
QUESTION?.
Number of
Responses
46
Documentation. Is the system user documentation current?
Not Current Mostly Current Completely Current
0....1....2....3....4....5....9....7....8....9....10
Average
Response
6.6
117
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IFMS
APPEKDIX VIZZ
ANALYSIS OF AUTOMATED DOCUMENT
CONTROL REGISTER (ADCR)
CUSTOMER SATISFACTION SURVEY
QUESTION 8.
Number of
Responses
50
Usefulness. How useful is information from the following
system (ADCR)?
Not Useful Acceptable Very Useful
0....1 ....2....3....4....5....6..J771.8....9....10'
Average
7.2
QUESTION 9.
Number of
Responses
SO
Ease of Use. For the systems you use. how easy is it to use
the following?
Not Easy
Easy
Very Easy
Average
Response
8.2
QUESTION 10.
Number of
Responses
24
interfaces. What other financial systems/interfaces do you use?
Number Percent
CPS
ORDOPS
EPAYS
GICS
CPARS
PPAS
APDS/CtS
Other
Total
2
1
17
5
11
3
1
9
49
4.1%
2.0%
34.7%
10.2%
22.4%
6.1%
2.0%
18.4%
100.0%
QUESTION 11. Operations. When the system is down, what alternative sources
of the data are available?
•NOTE* The response rate did not meet an acceptable minimum
threshold, therefore this question has been dropped irom
the survey.
118
Report No. E1NMF3-15-.0073-4100561
-------
EPA.' a XVKS
APPENDIX VIII
ANALYSIS OF AUTOMATED DOCUMENT
CONTROL REGISTER (ADCR)
CUSTOMER SATISFACTION SURVEY
QUESTION 12.
Number of
Responses
51
Customer Service. If you have operational problems, how do you
resolve them? (Indicate one or more.)
Hotline
Request a change to the system
Wait and try again later
Other
Total
Number
44
7
32
4
87
Percent
50.6%
8.0%
36.8%
4.6%
100.0%
QUESTION 12. (Continued)
Number of
How satisfied are you with the responsiveness of
customer service?
Responses
51
Dissatisfied
Satisfied
Always Satisfied
Average
Response
6.6
119
Report No. E1NMF3-15-0073-4100561
-------
EPA'e ZFNS
APPENDIX VIZI
LOCATION:
Responses
Percent
POSITION:
Responses
Percent
•
QUESTION 1.
Number ol
Responses
17
QUESTION 2.
Number of
Responses
12
ANALYSIS OF RESOURCE MANAGEMENT
INFORMATION SYSTEM (RMIS)
CUSTOMER SATISFACTION SURVEY
Headquarters . Region Rejd Total.
11 6 0 17
64.7% 35.3% - 0.0% 100.0%
Manager Supervisor Operations Total
1 1 11 13
7.7% 7.7% 84.6% 100.0%
Indicate each system used and the frequency of use.
Average
Never Rarely Often Response
0....1....2....3...A...5Q--7....8....9....10
5.8
Job Function. Identify primary job/mission functions that
RMIS performs for you.
Number Percent
Planning 2 7.4%
Budgeting 6 22.2%
Monitoring 5 ' 18.5%
Finance 1 3.7%
Data Entry 3 11.1%
Funds Control 3 11.1%
Reporting 7 25.9%
Total 27 100.0%
120
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX V1I1
ANALYSIS OF RESOURCE MANAGEMENT
INFORMATION SYSTEM (RMIS)
CUSTOMER SATISFACTION SURVEY
QUESTION 2. {Continued) How well does the system support your job function?
Number of
14
QUESTION 3.
Number of
Responses
16
QUESTION 4.
Number of
Responses
12
QUESTION 5.
Number of
Responses
14
QUESTIONS.
Number of
Responses
11
QUESTION?.
Number of
Responses
12
Never
Satisfactory Outstanding
Accuracy. What describes you experience with Ihe accuracy
of system information?
Never
Accurate
Usually
Accurate
Highly
Accurate
Response Time. What describes your experince with the
on-line response time?
Not Acceptable Acceptable Outstanding
Availability. What describes you experience of how the
system is available?
Never
Frequently
Always
Training. What describes how well the training and
framing materials met you needs? •
Unacceptable Acceptable Outstanding
Documentation. Is the system user documentation current?
Not Current Mostly Current Completely Current
Average
5.3
Average
6.9
Average
Response
6.8
Average
Response
8.4
Average
Response
3.7
Average
Response
4.9
121
Report No. E1NMF3-15-0073 -4100561
-------
EPA'e IPHS
APPENDIX VIZI
ANALYSIS OF RESOURCE MANAGEMENT
INFORMATION SYSTEM (RMIS)
CUSTOMER SATISFACTION SURVEY
QUESTION 8.
Number of
Responses
16
Usefulness. How useful is information from the following
system (RMIS) ?
Not Useful
Acceptable
Very Useful
Average
7.5
QUESTION 9.
Number of
Responses
13
EaseofUse. For the systems you use, now easy is it to use
the following?
Not Easy
Easy
Very Easy
Average
Response
5.6
QUESTION 10.
Number of
Responses
11
Interfaces. What other financial systems/interfaces do you use?
Number Percent
CPS
ORDOPS
EPAYS
GICS
CPARS
PPAS
APOS/CIS
Other
Total
0
1
8
1
5
1
1
2
19
0.0%
5.3%
42.1%
5.3%
26.3%
5.3%
5.3%
10.5%
100.0%
QUESTION 11. Operations. When the system is down, what alternative sources
of the data are aval table?
•NOTE* The response rate did not meet an acceptable minimum
threshold, therefore this question has been dropped from
the survey.
122
Report No. E1NMF3-15-0073-4100561
-------
EPA'a IPMS
APPENDIX VIII
ANALYSIS OF RESOURCE MANAGEMENT
INFORMATION SYSTEM (RMIS)
CUSTOMER SATISFACTION SURVEY
QUESTION 12.
Number of
Responses
16
Customer Service. If you have operational problems, how do you
resolve them? (Indicate one or more.)
Hotline
Request a change to the system
Wait and try again later
Other
Total
Number
12
4
11
2
29
Percent
41.4%
13.8%
37.9%
6.9%
100.0%
QUESTION 12. (Continued)
Number of
Responses
15
How satisfied are you with the responsiveness of
customer service?
Dissatisfied
Satisfied
Always Satisfied
Average
Response
5.3
123
Report No. E1NMF3-15-0073-4100561
-------
•I
EPA'8 IFMS
APPENDIX VIII
SURVEY ANALYSIS METHODOLOGY
Customer Survey of EPA's Integrated Financial Management System
As part of the audit of EPA's Integrated Financial Management
System, we conducted a customer satisfaction survey. , The survey
was an Agency-wide attribute sample of users of five major
components of EPA's Core Integrated Financial Management System:
IFMS, FMS, ADCR, MARS, and RMIS. If a sufficient number of
responses were received, this approach would allow projecting the
results of the completed responses for IFMS, FMS, ADCR, MARS, and
RMIS to the entire universe of users.
We sent the survey questionnaire directly to individual users in
EPA's program offices, regions and labs. The universe of users
that we used for our survey was derived from the Time Sharing
Services Management System (TSSMS), as of March 1993. The total
universe was made up of 2535 users. We excluded 508 contractor
personnel and 121 inactive EPA employees. Therefore, we arrived
at a sampling universe of 1906 users. This universe reflects all
active user ID's (except contractor personnel) in 75 accounts
which permit access to IFMS, FMS, MARS, ADCR, and RMIS. Some of
these users had multiple user ID'S and therefore if the sample
selected an individual more than once, another user ID was
selected at random,
In order to perform a statistically valid survey, first we
identified the accounts that users had that accessed these
systems, then we created a sampling plan. This sampling plan was
generated with E-Z-Quant Quantitative Methods For Auditors
Version 2.1 issued by the Defense Contract Audit Agency (DCAA).
The plan noted:
Universe Size: 1906
Anticipated Error Rate in Universe (%): 50
Desired Precision of Estimated Universe Error Rate (%): 10
Desired Confidence Level (%): 90
Sample Size: 256
124
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX VIII
We then generated random numbers for sample selection. Using
these numbers we generated the list of users (256) to be sent the
survey.
The users were asked to fill out the questionnaire and return it
directly to the OIG. Individual users were told that their
replies would be considered confidential working papers, however
we would summarize the questionnaire responses and distribute
that summary. When the completed surveys were received, their
data was entered into 5 electronic files, one file for each
system under review. In reviewing the survey responses, it was
determined whether or not the user accessed one, .some, all, or
none of the systems being reviewed (IFMS, FMS, MARS, ADCR, or
RMIS).
In order to project the findings with a 90 percent confidence
level and an estimated error rate of +/- 10 percent, we need a
response rate of 50 percent (128) to the 256 sample size. We
received 52 responses for ADCR, 66 responses for FMS, 148
responses for IFMS, 94 responses for MARS, and 17 responses for
RMIS. Therefore, we can only project for IFMS, for all other
systems we are only reporting (not projecting) the summary data.
Due to the extremely low response rate for RMIS, it should not be
considered representative.
This user satisfaction survey is consistent with guidance
provided in the Computerized Information Systems Audit Manual
published by the EDP Auditors Foundation . specifically, it
notes that users of data or reports produced by an application
system can help the auditor identify errors in processing or
other major problem areas. The auditor should, therefore,
identify and interview a sample of the* principal users of the
system to determine exactly how they use it, what they think of
its accuracy, timeliness, and completeness, and whether as
currently structured it meets their information needs.
Recommendation 13 from the March 29, 1991 Audit Report No.
E1AMFO-11-0029-1100153 Integrated Financial Management System;
Managing Implementation Of The New Accounting System, page 32,
stated "Involve all user groups in developing and implementing
the Agency's financial management system, and document that their
needs and priorities were considered in deciding the direction
and plans for the system." In preparing the survey, we were
consistent with the prior user survey questions used by OIRM in
1991. We also utilized the prior survey for format and subject
matter.
125
Report NO. E1NMF3-15-0073-4100561
-------
EPA'S IPMS
APPENDIX VIII
This user survey also follows the spirit of Executive Order 12862
of September 11, 1993 Setting Customer Service Standards, which
requires departments and agencies to "identify the customers" and
to "survey customers in order to determine the kind and quality
of services they want and their level of satisfaction with
existing services.11
Customer Survey Questions
Each user was asked for their location, position, how long they
had used IFMS, and 14 other (numbered) questions. The questions
are self-explanatory. The users were asked to answer the
questions only for the systems that they used by checking the
appropriate box. We analyzed the data by question by system and
summarized the results. The analysis shows how many responses
were received for each question within each system. This is
because not all questionnaires were completely filled out.
126
Report No. E1NMF3-15-0073-4100561]
-------
EPA's IFMS
APPENDIX IX
'JT^AfunciW mw nw A t/*v»-» j*-*-** MUA-AU** .*<-»* .•„•«»•* — — ... -— .__-„_ _.
Phase 1 Components: Planned Implemented Not19 Target
Date Date Implemented Date
(Fiscal)
o Modules
Budget Execution 1/89 3/89
o
Accounts Payable
General Ledger
Budget Formulation
Travel
Accounts Receivable
Reporting
Eliminate Systems '
FMS
RMIS
ADCR
1/89 3/89
1/89 3/89
1/89
1/89 3/89
1/89
1/89
9/89
9/89
9/89
x( custom)
x(partial)
X(MARS)
x
X
X
Phase 2 Components:
o
Modules
Purchases /Proc.
Fixed Assets
Cost Project Acct.
Payroll Distribution
10/89 3/89
10/89
10/89
10/89
X
X
X
KTD20
TBD
NTD
1995
1995
1994
•,
TBD21
1995
NTD
o Replace Account Code
Phase 3 Components;
o Interfaces
EPAYS
GICS
CPS
PPAS
CIS/APDS
10/89
10/90
10/90
10/90
10/90
10/90
1989
1989
1994
x(partial)1994
X 1995
x(replace)1998
19
Not implemented denotes thai the module wai either partially TnrHi-"^ (u indicated); never purchaaed; purchaicd but not implemented; or
entirely fimnmiTrd. Alio. not implemented denote* (hit (be kef icy «yitemi were not "J""1"'"*- the account code itructuri wit not reviled; and interface* not
NTD-No Tarfrt Dale. EPA no longer ba* plan* to implement the off-the-shelf module*.
TBD-To be determined. Requirement* and option* are being ttudied in 1994; no target date available for implementation
127
Report NO. E1NMF3-15-0073-4100561
-------
EPA'S IFMS <•]
APPENDIX X
OKCORRECTED MID CORRECTED LONGSTANDING FINANCIAL MANAGEMENT
yEAKNESSES REPORTED UNDER FMFIA (FISCAL 1989 - 1993)
YEAR
AREA 83 84 85 86 87 88 89 90 91 92 93 94 95
X—— X X- X X X X X X' X X X X
A. OMB HIGH RISK:
1. Financial
Systems
B. MATERIAL
WEAKNESSES:
1. IFMS
2. FMS/PPAS
x_—p p
3. Accounts x p
Receivable
C. NONCONFORMANCES:
1. Financial x p c
Reporting
2. IFMS X p p p
Accounts Receivable
3. Reconciliation x— p c
Treasury Reports
4. Property x p p- p P tbd
Management
5. General Ledger x p p c
FMS
6. Interfaces x p p p p
7. Letter of x p c
Credit
8. EPAYS x • p
x—weakness identified
p—planned corrected action
c—corrected
tbd—to be determined
128
Report NO. E1NMF3-1S-0073-4100561'
-------
EPA'a IFMS
APPENDIX XX
YEAR
1987
1988
1989
1990
1991
1992
1993
1994
1995
TOTAL COSTS
IFMS AND LEGACY
1
IFMS
SYSTEM
$510,000
2,694,000
3,362,000
2,192,000
3,107,400
862,400
3,322,100
1,150,000
$17,199,900
SYSTEMS COSTS
, 2 •
LEGACY
SYSTEMS
-
.
1,990,600
1 ,990,600
1,990,600
1,990,600
1 ,990,600
1,266,000
$11,219,000
Footnote 1: These costs reflect actuals and estimates for development and enhancements only.
These costs do not represent all system life cycle costs.
Footnote 2: These systems include FMS, ADCR, and RMIS maintenance, operation, timeshare,
and telecommunications costs (see Appendix VII).
129
Report No. E1NMF3-15-0073-4^00561
-------
EPA'S IFMS
APPENDIX XII
ESTIMATED ANNUM. AND TOTAL COST OF MAINTAINING LEGACY SYSTEMS
(1990 THROUGH 199S1
Est. Ann. Estimated
Cost
Total
Costs
Estimated Annual Costto Maintain
The Legacy Financial Systems
—Financial Management System (FMS)
—Resources Management Information
System (RMIS)
—Automatic Document Control Register
(ADCR)
Totals,
$1,285,500 $7,605,800
95,600 565', 400
609,500 3,047,600
$1,990,600 $11,218,800
Notes:
The legacy financial system figures include operational and
maintenance, timeshare, and telecommunications costs using
available fiscal 1992 and 1993 cost information. The Director,
IFMS Project Management Staff provided documentation on
operational and maintenance costs of the legacy systems for 1993
and indicated that these costs are representative of prior and
future years expenditures under the specific systems. NDPD
provided 1992 cost figures under the specific system for
timeshare; and we applied a 69.1 NDPD-derived percentage for
telecommunications. We used the NDPD timeshare and
telecommunication figures to estimate prior and future costs.
The legacy financial system figures do not include costs for:
hardware; personnel; maintenance of the interfaces for these
systems; and the manual reconciliation processes. Other costs
not included are the operations and maintenance at the 14
Servicing Finance Offices for these systems, as we did not
analyze these operations.
130
Report NO. E1NMF3-1S-0073-410056:
-------
EPA's IFMS
in
LJJ
D
O
LU
g
LU -•
O 5
X
O
u.
O
D
t-
<
Li.
O ±
APPENDIX XIII
LO
131
Report No. E1NMF3-15-0073-4100561
-------
EPA's IFMS
APPENDIX
ACRONYMS
AA
ADABAS
ADCR
ADP
AMCR
AHS
APDS
ARA
ARTS
ASB
ASC
ASD
BARS
BO
BITSB
CERCLIS
CFO
CICS
CIS
CMS
CPARS
CPS
Assistant Administrator
Adaptable Data Base System
Automated Document Control Register
Automated Data Processing '
Alternate Management Control Review
American Management Systems
Automated Procurement Documentation System
Assistant Regional Administrators
Action Request Tracking system
Application Software Branch
Administrative System Council
Administrative Systems Division
Budget and Accounting Reporting System
Budget Division
Budget Information and Technical systems Branch
Comprehensive Environmental Response,
Compensation, and Liability Information System
Chief Financial Officer
Customer Information Control System
Contract Information System
Change Management System
Combined Payroll Redistribution and Reporting
System
Contract Payment System
132
Report NO. E1NMF3-15-0073-410Q561
-------
EPA'S IFMS
CPU
CSB
DAA
DCFO
DOT
DSO
EMG
EPA
EPAYS
FAD
FFS
PIPS
FIRMR
FMD
FMFIA
FMS
FSB
GAO
GICS
GSA
H.R.
HAOB
IAG
IBM
APPENDIX XIV
Central Processing Unit
Client support Branch
Deputy Assistant Administrator
Deputy Chief Financial Officer
Department of Transportation
Designated Senior Official
Executive Management Group
Environmental Protection Agency
EPA Payroll system
Financial Audit Division
Federal Financial Systems
Federal Information Processing Standards
Federal Information Resource Management Regulation
Financial Management Division
Federal Managers' Financial Integrity Act
Financial Management System
Financial Systems Branch
General Accounting Office
Grants Information and Control System
General Services Administration
House of Representatives
Headquarters Accounting Operations Branch
Interagency Agreement
International Business Machines
133
Report NO. E1NMF3-15-0073-4100561
-------
EPA'S IFMS
IFMS
INFOPAC
IRM
IRS
JFMIP
LUST
MARS
HAS
MCP
MCR
MVS/ESA
NC
NDPD
NPR
OARH
OC
OIG
OIRM
OMB
ORD
OTUS
PMB
PCIE
APPENDIX XIV
Integrated Financial Management System
Billing Report Distribution System
Information Resources Management
Internal Revenue Service
Joint Financial Management Improvement Program
Leaking Underground Storage Tank
•t
Management and Accounting Reporting system
Multiple Award Schedule
Management control Plans
Management Control Review
Multiple Virtual storage/Enterprise Systems
Architecture
North Carolina
National Data Processing Division
National Performance Review
Office of Administration and Resources Management
Office of the Comptroller
Office of Inspector General
Office of Information and Resources Management
Office of Management and Budget
Office of Research and Development
Online Timeshare Utilization System
Planning and Management Branch
President's Council on Integrity and Efficiency
134
Report NO. E1NMF3-15-0073-410056:
-------
EPA'S IFMS
PPAS
PRA
RMD
RMIS
RPIO
RTF
SBO
SCORES
SDLC
SEC
SFO
SMG
SOW
SPUR
TB
TSSMS
VA
WCF
APPENDIX XIV
Personal Property and Accounting System
Paperwork Reduction Act
Resources Management Directive
Resources Management Information System
Responsible Planning and Implementation Office
Research Triangle Park
«
Senior Budget Officers
Superfund Cost Organization and Recovery System
System Development Life Cycle
Security and Exchange Commission
Servicing Finance office
System Management Group
Statement of Work
Special Package for Unique Reports
Telecommunications Branch
Time Sharing Services Management System
Department of Veteran Affairs
Working Capital Fund
135
Report Mo. E1NMF3-15-0073-4100561
-------
SPA'S IFMS
APPENDIX XV
REPORT DISTRIBUTION
Office of Inspector General
Inspector General (2410)
Deputy Inspector General (2410)
EPA Headquarters
Assistant Administrator for Administration and
Resources Management (3101)
Assistant Administrator for Policy, Planning, and
Evaluation (2111)
Assistant Administrator for Enforcement (2211)
Office of General Counsel (2310)
Assistant Administrator for Water (4101)
Assistant Administrator for Solid Waste and Emergency
Response (5101)
Assistant Administrator for Air and Radiation (6101)
Assistant Administrator for Prevention, Pesticides and
Toxic Substances (7101)
Assistant Administrator for Research and Development
(8101)
Associate Administrator for Regional Operations &
State/Local Relations (1501)
Associate Administrator for Congressional and
Legislative Affairs (1301)
Associate Administrator for Communications, Education
and Public Affairs (1701)
Comptroller, Office of the comptroller (3301)
Director, Office of Information Resources Management
(3401)
136
Report No. E1NHF3-15-0073-4100561J
-------
EPA'S IFMS
APPENDIX XV
Director, Financial.Management Division (3303F)
Director, IFMS Project Management Staff (3301)
Agency Followup Official (3101)
Attn: Assistant Administrator for Administration
and Resources Management
Agency Followup Coordinator (3304)
Attn: Director, Resource Management Division
Audit Followup Coordinator (3102)
Attn: Program & Policy Coordination Office
EPA Headquarters Library
Regional Offices
%
Regional Administrator, Region 1
Regional Administrator, Region 2
Regional Administrator, Region 3
Regional Administrator, Region 4
Regional Administrator, Region 5
Regional Administrator, Region 6
Regional Administrator, Region 7
Regional Administrator, Region 8
Regional Administrator, Region 9
Regional Administrator, Region 10
Research Triangle Park. North Carolina
Director, Office of Administration and Resources
Management (MD-20)
Director, National Data Processing Division/OARM
(MD-34)
137
Report NO. E1NMF3-15-0073-4100561
-------
External
Office of Management and Budget
General Accounting Office
General Services Administration
Information Resources Cento-
US cFW (3404)
401 M Street, SW /
Washington, DC 20460
EPA'8 IFMS
APPENDIX XV
138
Report Mo. E1NMF3-15-0073-410056:
-------
-------
------- |