Office of Inspector General
Audit Report
Information Technology
EPA Management of
Information Technology Resources
Under The Clinger-Cohen Act
Report No. 2002-P-00017
September 30, 2002
-------
Inspector General Division
Conducting the Audit:
Information Technology Audits Division
Program Offices Involved:
Audit Team Members:
Office of Environmental Information
Office of the Chief Financial Officer
Office of Enforcement and Compliance Assurance
Office of Air and Radiation
Office of Solid Waste and Emergency Response
Office of Water
Jim Rothwell, Project Manager
Jim Haller, Technical Support
Ernest Ragland, Auditor
Michael Young, Auditor
Robert Shields, Auditor
Robert Smith, Auditor
Abbreviations
CIO
CPIC
CTO
DCIOT
EPA
GAO
ICIS
IRM
IIS
rr
I-TIPS
OEI
OIG
OMB
RCRAInfo
SDWIS/STATE
SMP
VPN
Chief Information Officer
Capital Planning and Investment Control
Chief Technology Officer
Deputy CIO for Technology
U.S. Environmental Protection Agency
General Accounting Office
Integrated Compliance Information System
Information Resources Management
Information Investment Subcommittee
Information Technology
Information Technology Investment Portfolio System
Office of Environmental Information
Office of Inspector General
Office of Management and Budget
Resource Conservation and Recovery Act Information Management System
Safe Drinking Water Information System/State Version Modernization Effort
System Management Plan
Virtual Private Network
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
ern « n onf« OFFICE OF
Otr 3 U dJUt THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT: Final Report: EPA's Management of Information Technology Resources
under the Ginger-Cohen Act
Audit No. 2001-0591
Report No. 2002-P-00017
FROM: Patricia H.
Director, Business Systems (2421T
TO: Kim Nelson
Assistant Administrator and Chief Information Officer (2810A)
Attached is our report titled "EPA 's Management of Information Technology Resources under
the Clinger-Cohen Act." Our objective was to evaluate whether EPA has established a Chief
Information Officer (CIO) position with sufficient authority and administrative controls to effectively
manage Information Technology (IT) resources agency-wide, and to assess whether the CIO has
adequately implemented the Act's requirements. The audit also evaluated whether the CIO coordinated
with the Chief Financial Officer to help provide sufficient direction and guidance to Agency managers to
ensure IT investments are acquired in a cost-effective manner.
This audit report contains findings thai describe pioblems the Office of Inspector General (OIG)
has identified and corrective actions the OIG recommends. This audit report represents the opinion of
the OIG, and the findings contained in this audit report do not necessarily represent the final EPA
position. Final determinations on the matters in the audit report will be made by EPA managers in
accordance with established EPA audit resolution procedures.
Action Required
In accordance with EPA Order 2750, you, as the primary action official, are required to provide
us with a written response to the audit report within 90 days of the final report date. If corrective actions
will not be complete by the response date, we ask that you describe the actions that are ongoing and
reference specific milestone dates which will assist us in deciding whether to close this report. In
addition, please track all action plans and milestone dates in EPA's Management Audit Tracking
System.
Recycled/Recyclable • Printed with Vegetable Oil Based Inks on 100% Recycled Paper (40% Postconsumer)
-------
We appreciate the cooperation afforded us during the course of this audit by the Office of
Environmental Information, Office of Chief Financial Officer, Office of Air Quality Planning and
Standards, Office of Water, and Office of Solid Waste and Emergency Response. We have no
objections to the further release of this report to the public. Should you or your staff have any questions
regarding this report, please contact James Rothwell, Project Manager for Information Technology
Audits Division, at (202) 566-2570.
Attachment
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Executive Summary
Introduction
In 1996, the U.S. Congress enacted the Clinger-Cohen Act (Act), initially known as the
Information Technology Management Reform Act, to improve the management of
federal agencies' information technology (IT) resources. The Act requires each agency
head to develop and implement a process for maximizing the value of and assessing and
managing the risks of IT acquisitions. This process is known as the IT Capital Planning
and Investment Control (CPIC) process. The CPIC process relates to an agency's
selection of information technology investments, the management of such investments,
and the on-going evaluation of funded investments. The Act requires the Chief
Information Officer (CIO) to establish an Enterprise Architecture and to use it as part of
the CPIC process. The Enterprise Architecture establishes the entity-wide road map to
achieve an agency's mission. An agency's capital planning and control process must
build from its current Enterprise Architecture, and support the transition from its current
to target architecture.
Objectives
We audited to determine whether:
• EPA has established a CIO position with sufficient authority and administrative
controls to effectively manage IT resources Agency-wide.
• EPA's CIO has adequately:
• managed and controlled investments using a comprehensive IT CPIC
process;
• developed and maintained an Enterprise Architecture;
• monitored IT investment projects and provided standard tools and practices
for managing system development projects; and
• coordinated with the Chief Financial Officer to help provide sufficient
direction and guidance to Agency management regarding cost effective
acquisitions.
Results in Brief
EPA's CIO has sufficient authority to shape and direct Information Resource
Management (IRM) activities. Nevertheless, past CIOs have not provided the
leadership needed to fully implement the changes required by the Act. Since
established in 1998, EPA's CIOs have taken some actions to implement and
institutionalize the Agency-wide authority and responsibilities for IT capital
investments. Yet many strategic planning and development activities only started in
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
fiscal 2001. A first step in addressing EPA's planning needs was the CIO approval of
an updated EPA Strategic Information Plan on July 29, 2002.
EPA's new CIO recognizes the importance of the issues raised in this report and is
taking aggressive steps to address the Act's fundamental components. For example,
in May 2002, the CIO established a Chief Technology Officer position to coordinate,
implement, and advise on the Strategic Technology Plan, Agency Architecture,
E-government activities and IT investments. Also, in June 2002, the Deputy CIO for
Technology (DCIOT) was assigned responsibility for establishing and publishing
standards and procedures based on the Act. However, institutionalizing structured,
centralized controls and oversight processes will take additional effort and resources.
Some program managers have not taken the Act seriously and have viewed its
requirements as another step to satisfy the annual OMB budget call.
Several key factors continued to limit the realization of a successful program:
• Senior program managers continued to use outdated and unauthorized IT
acquisition practices, because Agency IT policies conflicted with the Act's
requirements and the CIO's authority.
• The Agency was still developing its Enterprise Architecture Plan, and had not
established a formal chain of command, either through policies or formal
delegation, from the CIO to the Chief Technology Officer, DCIOT, and Chief
Architect. In particular, formalization of the Chief Technology Officer and Chief
Architect positions will help ensure sufficient management authority and resources
to implement the Act. Also, position descriptions for all three roles should be
updated to address respective responsibilities for the development of an Enterprise
Architecture and execution of related IT activities.
• EPA had not implemented a CPIC performance-based measurement system for
assessing and managing risks of FT acquisition, and implementing, monitoring and
evaluating IT projects.
EPA is in the process of implementing an FT cost accounting system to support such
areas as IT budget reporting, project management, and system life cycle management.
Project cost accounting is a critical management tool for EPA to achieve acceptable,
efficient and effective accounting, budgeting, and procurement of IT investment
projects.
With regard to the fiscal 2002 budget, we believe the CIO had minimal assurance that
IT investments reported to OMB would maximize their value. Moreover, the CIO had
little assurance that these investments were adequately assessed for risk factors, that
risks were being managed, or that products were procured consistent with the Act's
requirements. EPA reported investments that totaled more than $449 million for the
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
fiscal 2002 budget. Our review showed that EPA continued to spend millions on IT
investments that appeared to be making minimal or insignificant progress. During the
period under review, EPA's IT investments were not maximizing the efficiency of IT
operations nor resolving long-standing problems, such as integration of environmental
data. Existing IT contracts, with a maximum value totaling approximately $1.6 billion,
can be awarded new work without proper delegated authorization from the CIO.
Furthermore, EPA continued to award new IT contracts without required CIO
approval.
Recommendations
Improving the fundamental issues addressed in this report will require a series of inter-
related corrective actions. To help EPA management plan for and channel its resources
in a methodical manner, we prioritized the recommendations listed in Chapters 2
through 6 of this report. The most prominent recommendations are summarized below.
The CIO will need to complete and implement these actions in order to improve the
way EPA's IT investments are assessed, managed, and evaluated.
• Revise outdated policies to remove unauthorized IT business practices and add new
requirements.
• Formally re-delegate authority and responsibilities for implementing the Clinger-
Cohen Act to the Chief Technology Officer and, in turn, further re-delegate to the
Chief Architect the management authority and responsibilities for maintaining an
Enterprise Architecture.
• Establish and update policies for the Enterprise Architecture and execution of
related FT investment activities under the Act.
• Implement an automated project management system.
• Implement individual project monitoring and evaluation processes for IT
investments.
The CIO also will need to work with other Agency officials to establish delegations,
policies, and procedures for IT procurements.
Agency Comments and OIG Evaluation
We received comments from EPA's CIO, Comptroller, Assistant Administrator for
Solid Waste and Emergency Response, and the Director, Information Transfer and
Program Integration Division of the Office of Air and Radiation. We amended the
report based on these responses, as well as additional discussions with appropriate
management officials.
111
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
The CIO agreed with our emphasis on the importance of an effective IT investment
management program and agreed to continue to aggressively address issues identified
by the report. The CIO noted substantive accomplishments toward that goal, such as
establishing new policies, promulgating a new information strategic plan, hiring a Chief
Technology Officer, employing a risk-based process for IT investments, and
establishing a cost tracking system.
While we agree that EPA has taken significant initial steps to address the report's
findings and recommendations, there are still significant recommendations that need to
be addressed, such as implementing an automated system to manage the CPIC
process. Also, authorities and responsibilities for the Chief Technology Officer and
Chief Architect need to be incorporated into Agency policy, and resources need to be
dedicated to complete and maintain EPA's Enterprise Architecture. The CIO has
established an ambitious schedule to address this report's recommendations, and it will
require EPA to continue dedicating significant resources.
The Comptroller responded that his office was working with an Office of Environmental
Information workgroup to ensure consistent treatment of IT costs with common system
life cycle stages. The Comptroller did not agree to amend existing IT contracts and
stated that the interim policy announcement provided adequate controls. We still have
concerns about the adequacy of the new cost accounting process for categorizing
project costs by life cycle phases. However, we will defer making formal
recommendations until a more detailed assessment of the new process can be
competed as part of the Fiscal 2002 financial statements audit.
The Assistant Administrator for Solid Waste and Emergency Response, and the
Director for the Office of Air and Radiation's Information Transfer and Program
Integration Division, both disagreed with our conclusion that project management
controls were inadequate. We did not review all project management controls, but we
did document inaccurate and/or unsupported information being reported as part of the
budget for the IT system projects. We also found that the projects did not comply with
existing Agency systems development life cycle policy documentation requirements.
We consider these to be significant project management weaknesses.
IV
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Table of Contents
Executive Summary i
Chapters
1 Introduction 1
2 CIO Needs to Fully Implement Clinger-Cohen Act Requirements 5
3 Weaknesses in CPIC Process Place EPA's IT Investments at Risk 11
4 EPA Needs to Organize and Integrate Planning for IT Investments 19
5 EPA Needs to Strengthen IT Project Management Controls 29
6 Project Cost Accounting System Vital for
Planning and Managing IT Investments 37
Appendices
1 Details on Scope and Methodology 41
2 Office of Environmental Information's Response to Draft Report 45
3 Office of the Chief Financial Officer's Response to Draft Report 67
4 Office of Air Quality Planning and Standards'
Response to Draft Report 71
5 Office of Solid Waste and Emergency Response's
Response to Draft Report 73
6 Report Distribution 75
v
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
VI
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 1
Introduction
Purpose
The audit's objectives were to determine whether:
• EPA had established a Chief Information Officer (CIO) position with sufficient
authority and administrative controls to effectively manage Information
Technology (IT) resources Agency-wide;
EPA's CIO had adequately:
Managed and controlled investments using an IT Capital Planning and
Investment Control (CPIC) process, including a determination of whether
investment decisions minimize the risk to the Agency, provide a positive
return on investment, and satisfy the Clinger-Cohen Act requirements;
Adopted the Federal Enterprise Architecture Framework components
necessary for developing and maintaining an Agency Enterprise Architecture,
as prescribed by the Office of Management and Budget (OMB) and the
Federal Chief Information Officers Council;
Monitored IT investment projects and provided standard tools and practices
for managing system development projects; and
• Coordinated with the Chief Financial Officer to help provide sufficient
direction and guidance to Agency management to ensure IT investments
were acquired in a cost-effective manner.
Background and Criteria
Act Established CIO Role and CPIC Process
The Clinger-Cohen Act of 1996 (Public Law 104-106) intended for a central process,
led by a CIO, to manage IT investments across an agency. Since 1996, EPA has
taken two significant actions to implement the Clinger-Cohen Act. In 1998, EPA's
Administrator established the CIO position through Delegation 1-84. The Delegation
assigned responsibility to exercise all responsibilities of the CIO pursuant to the Clinger-
Cohen Act, such as establishing an IT Architecture and an IT CPIC process. Then, in
1999, EPA reorganized its Agency IT management, and established an Office of
Environmental Information (OEI) and a Quality Information Council.
1
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
The Act requires the CIO to implement a CPIC process for maximizing the value and
assessing and managing the risks of an agency's IT acquisitions. The CPIC process is
to provide for the selection of investments using minimum criteria, both quantitative and
qualitative, for comparing and prioritizing alternative information systems projects. In
addition, the CPIC process must provide a means for senior managers to obtain timely
information regarding progress (at established milestones).
The Act identifies numerous requirements and responsibilities for the agency head,
CIO, and other key officials. Specific responsibilities for the CIO include:
• Developing and implementing a sound and integrated Enterprise Architecture;
• Monitoring and evaluating the performance of IT programs based on defined
measurements, and determining whether to continue, modify, or terminate a
program or project;
• Implementing and enforcing applicable government-wide and Agency IT
management policies, principles, standards, and guidelines;
• Acquiring and managing information resources in a manner consistent with Federal
laws and internal policies and procedures.
• Integrating Information Resources Management (IRM) operations and decisions
with organizational planning, budget, financial management, and program decisions;
• Developing a full and accurate accounting of IT expenditures, related expenses, and
results; and
• Establishing a process to select, control, and evaluate the results of major
information system initiatives.
Law and OMB Circulars Further Define Requirements
Under Title 44, U.S. Code, Section 3506, agencies are responsible for developing and
maintaining an IRM strategic plan, as well as a current and complete inventory of its
information resources.
OMB Circular A-130, Management of Federal Information Resources, requires the
CIO to: (1) prepare and update a cost-benefit analysis for each information system, as
necessary throughout its life cycle; (2) conduct cost-benefit analyses to support ongoing
management oversight processes; (3) conduct post-implementation reviews of
information systems to validate estimated benefits and document effective management
practices; and (4) establish information system management oversight mechanisms.
This Circular also emphasizes that IRM planning should help the Agency link IT to
mission needs. Furthermore, IRM planning should coordinate with other agency
2
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
planning processes, including strategic, human, and financial resources. The agency
should employ mechanisms to ensure that major information systems proceed in a
timely fashion towards agreed-upon milestones, meet user requirements, and deliver
intended benefits to the agency and the public.
OMB Circular A-11, Preparing and Submitting Budget Estimates, lists
requirements for preparing and submitting IT budget estimates, including requirements
to evaluate full life cycle costs, benefits, and Return on Investment.
CIO Council Addresses Best Practices and Provides Guidance
Federal CIO Council, Capital Planning and IT Investment Committee,
Implementing Best Practices, dated June 1998: The 24 major Federal agencies
participated in a Best Practices Workshop highlighting their approaches for selecting,
controlling, and evaluating critical FT investments.
A Practical Guide to Federal Enterprise Architecture, Version 1.0, February
2001: This guide states that an Enterprise Architecture establishes the agency-wide
road map to achieve an agency's mission through optimal performance of its core
business processes within an efficient IT environment. The Chief Architect, in
conjunction with the CIO and select Agency business managers, defines the
architectural principles that map to the organization's IT vision and strategic plans. As
shown in Figure 1, architectural principles should represent fundamental requirements
and practices believed to be good for the organization.
Strategic Plans
IT Visiu.ii,
Rm| uiiuxtuutls,
Hiiil
EA
Policies and Guidelines
• EA Development
• EA Use
• EA Maintenance
• EA C omplianee
Principles
• F.A
I
business JNeeds
Systems Life Cycle
1 Systems Migration.
1 Technology Insertion
• Dual Opwaliuiii:
l. PlHTIS
Investment Control
• Piujeul Sfieuliuii
• Pi i iji-:i :l. H iiTil.Tiil
• Prnjrr.t. F.-irnliifltinn
• Return on Investment
Figure 1. Role of Architecture Principles
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
EPA Delegation for CIO
EPA Delegations Manual 1200, 1-84, Information Resources Management, dated
December 18, 2001, specifically requires the CIO to:
(1) Approve the Agency's IRM Strategic Plan, Five-Year IRM Implementation
Plan, IRM investment portfolio, and IRM contracting strategy;
(2) Establish policies and procedures for the management and security of records,
files, data, and information systems and technology;
(3) Approve the acquisition of information technology resources; and
(4) Establish and maintain a continuing program for the management and security of
records, files, data, and information systems and technology.
Authorities (3) and (4) above were re-delegated on June 13, 2002, to OEI's Director
for Technology Operations and Planning. These authorities may be re-delegated
further to Assistant Administrators, Regional Administrators, the Chief Financial Officer,
and other senior Agency officials. Moreover, these officials may further re-delegate
authorities within their respective organizations.
EPA Requirements for Software Development
EPA Directive 2100, IRM Policy Manual, establishes a policy framework for IRM
programs at EPA. In particular, Chapter 17, System Life Cycle Management,
identifies life cycle requirements for information systems projects. These requirements
include the System Management Plan, cost-benefit analysis, and a risk analysis at each
stage of the system development life cycle. Chapter 17 also prescribes that a system
charter be developed during project initiation, including an estimate of life cycle costs,
and identifying the appropriate management levels for approving decision papers. A
System Management Plan decision paper should be produced at the conclusion of the
analysis stage and should be updated as the project progresses.
Scope and Methodology
We conducted this audit at EPA Headquarters in Washington, DC, starting in January
2001 and issued a draft report in April 2002. Subsequent to the draft report, we
updated portions of the findings to reflect recent Agency accomplishments. We
performed our audit in accordance with the Government Auditing Standards, as
issued by the Comptroller General of the United States, and included such tests as
necessary to complete our objectives. Exhibit 1 details our scope and methodology, as
well as prior audit coverage.
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 2
CIO Needs to Fully Implement
Clinger-Cohen Act Requirements
EPA's CIO needs to demonstrate strong leadership by providing IT technical expertise
and a workable investment management structure to ensure the Agency's many
program offices implement the IT capital investment process envisioned by the Clinger-
Cohen Act. While EPA has taken steps to implement Clinger-Cohen functions, many
aspects continue to evolve, with plans, policies, and guidance still in development. EPA
did not effectively manage its IT investments from an Agency-wide perspective;
however, it recently established a Chief Technology Officer to provide leadership and
implement a comprehensive IT investment program. For the period under review, we
found that program officials were still operating under invalidated IT acquisition policies
and procedures that allowed them to individually make investment decisions. EPA
appeared to be using a slowly evolving, volunteer-based, and decentralized approach
to developing, supporting, and managing IT capital investments. In addition, the lack of
a monitoring process allowed projects to be executed without a minimum level of
management controls. Finally, some program managers did not take the Act seriously
and viewed the Agency requirements as another step to satisfy the annual OMB budget
call.
CIO Relies on IT Budget Instead of Investment Portfolio Process
The CIO used the Fiscal 2002 annual budget call to plan IT investments. The Act
intended that the CIO establish a performance-based system for implementing, monitoring
and evaluating IT projects. The Agency's IT investment process was primarily a budget
reporting process. It was used to meet OMB IT program annual reporting requirements
and to recommend an annual budget for major systems investment projects. Financial
management, procurement, and project management controls were not adequately
integrated into the Agency's CPIC process. Moreover, project management practices
were inconsistent throughout the Agency. Numerous examples demonstrated that the
peer review used objective, yet constantly evolving, criteria for evaluating investment risk.
While the peer review process adequately quantified and documented risk
determinations, we could not substantiate the basis for Information Investment
Subcommittee's (IIS) decisions to (1) lower the risk determinations assigned to some
investment proposals, and (2) make recommendations for funding them to the Quality
Information Council and CIO.
Investment Portfolio Structure Missing Fundamental Elements
In 1998, EPA established a CIO position. In 1999, EPA created the OEI and
reorganized its IRM structure. However, more than 5 years after implementation of the
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Act, EPA still had not sufficiently implemented some fundamental elements of a
centralized investment portfolio structure (strategic IRM plan, CPIC process, Enterprise
Architecture, and cost accounting process). Specifically:
• Senior Agency program managers continued to use outdated and unauthorized IT
investment practices. Policies and procedures, such as EPA Directive 2100, need to
be revised to incorporate new CIO responsibilities relating to IT procurement,
systems development life cycle, project management, cost accounting, and budget.
• EPA's IRM Strategic Plan dated back to 1994, and did not reflect Clinger-Cohen
Act requirements. However, on July 29, 2002, the Agency updated the plan and
issued the EPA Strategic Information Plan: A Framework For The Future.
• Leadership and organization for developing the Enterprise Architecture changed
significantly over the past two fiscal years.
• Until the fall of 2001, the Agency budget submission included the architecture
project as a component of infrastructure proposals and, as such, was under that
leadership. In its fiscal 2003 budget submission, EPA identified it as a separate
architecture project and intensified efforts to complete the baseline and target
architectures.
• In February 2002, the CIO announced a Chief Architect position to manage the
development of an Enterprise Architecture. Then, in May 2002, the CIO
established a Chief Technology Officer position to coordinate, implement, and
advise on numerous IT investment management activities, including the Agency's
architecture. Also, through EPA's CPIC policy, the Deputy CIO for
Technology (DCIOT) was assigned responsibility for establishing and publishing
standards and procedures for the Agency Architecture, E-government activities,
and IT planning. These are positive actions, but the Agency has not yet
established a formal chain of command from the CIO to the Chief Technology
Officer, DCIOT, and Chief Architect. Formalization of the Chief Technology
Officer and Chief Architect positions would help ensure sufficient management
authority and resources to implement the Act.
• EPA believes it will be able to complete the Enterprise Architecture baseline,
target, and sequencing approach by October 2002. However, we have not
reviewed the recently-completed draft baseline, and have not evaluated whether
available resources will enable the Agency to achieve this milestone.
• Senior managers could not obtain timely and accurate cost, benefit, and performance
information on IT projects. In 2001, EPA purchased a service level agreement to
use off-the-shelf software called Information Technology Investment Portfolio
System (I-TIPS), a federally-sponsored software product, for monitoring and
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
evaluating IT projects in the CPIC process. EPA indicated it has assigned resources
for implementation, developed milestones for production, and will use the software to
generate automated reports to OMB for the 2004 budget submission. Furthermore,
management states that I-TIPS will be expanded agency-wide in 2003.
• Actions are needed to strengthen IT project management controls. Program
managers used inconsistent management tools, and EPA had no standard project
cost accounting system for providing useful data to project managers. Managers
used outdated cost-benefit assessments or chose to omit the assessment as part of
the system development process. Moreover, the CIO had not established
monitoring or evaluation processes to ensure major information systems proceeded
in a timely and cost-effective fashion, met user requirements, and delivered intended
benefits to the Agency and affected public.
These issues are covered in greater detail in Chapters 3 through 6.
EPA's Process Creates Unacceptable Risk for IT Investments
The absence of a fully-developed, centralized investment portfolio structure resulted in
management's:
• inconsistent and undocumented evaluations - IIS approval of IT investment
proposal projects which were documented as high risk by a peer review process,
• inability to effectively monitor IT system development or enhancement projects'
schedules and costs,
• omission of investment benefit evaluations for completed IT proj ects, and
• inability to document and account for IT project investment costs.
The slowly evolving and decentralized approach that was being used to develop an IT
investment control structure was not successful. EPA's approach allowed IT projects to
be funded without proper justification, and in the absence of adequate management
controls. EPA invested resources on outdated systems that did not maximize the
efficiency or resolve long-standing problems, such as integration of environmental data.
For example, the Air Quality System spent over $8 million from fiscal 1996 through
2001 for the project's Phase 1. The fiscal 2001 budget submission for the project
included a statement of intent to make modifications in Phase 2 to adapt the system to
function with EPA's Central Data Exchange portal and incorporate Agency data
standards. However, these critical functional modifications were not addressed until
fiscal 2002, about 6 years into the project.
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Conflicts between the EPA Delegation 1-84 and prior procurement policies caused
program and regional managers to award new IT contracts without proper CIO
approval. Also, existing IT contracts, with a maximum value totaling approximately $1.6
billion, can be awarded new work without proper authorization. Under EPA Delegation
1-84, the CIO is the only manager authorized to approve acquisitions of IT resources.
In June 2002, this authority was re-delegated to OEFs Director for Technology
Operations and Planning. This authority can be re-delegated further. However, this
delegation conflicts with and invalidates prior EPA procurement policies and practices in
EPA Directive 2100.
Overall, there is a high risk that EPA's technology investments will not result in significant
improvements in organizational efficiency and productivity, or enable EPA to work
better with states, tribes, local governments, private industry, and the general public.
EPA planned to spend approximately $449 million for IT investments in fiscal 2002, so
poor investment choices could have significant monetary ramifications. To avoid risk,
EPA must ensure that its target enterprise architecture is fully integrated with its
Government Performance and Results Act goals and objectives, IRM Strategic Planning,
and IT acquisition processes. Until this integration is achieved, EPA will continue to
struggle with its ability to reinvent organizational processes, integrate and manage data,
and build a scalable and reliable network architecture.
In its fiscal 2003 budget submission, EPA took the first step in consolidating duplicate
systems when it combined four modernization efforts into two investment proposals.
EPA's process for evaluating investment proposals appears to consider data standards
requirements and system duplications; however, management must continue to
strengthen procedural controls to minimize effects of a weakly integrated process, such
as:
• IT investments that are not driven by business priorities and mission goals,
• investing in stovepipe and duplicate systems,
• IT investments that do not take advantage of technology advances and reduced
costs,
• inefficient reporting processes for states and private industry users,
• application systems that do not comply with environmental data and interoperability
standards, and
• not meeting increased public access and security requirements.
Until EPA fully implements the Act's requirements, management will be unable to make
fully-informed decisions regarding IT investments.
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Strong CIO Leadership Needed to Implement and Enforce Act
Although it has been more than 5 years since the Clinger-Cohen Act was implemented,
EPA has yet to comply fully with its statutory requirements. We believe this was due, in
part, to the fact that EPA did not have a presidentially-appointed and Senate-approved
CIO prior to December 2001. Although EPA reorganized its IRM office and
established a CIO position, there was little change in the Agency's IT operations or
investment practices until recently.
The lack of strong CIO leadership and a comprehensive investment portfolio structure
perpetuated the Agency's unsuccessful, decentralized IT investment process. The CIO
should target key agency-wide problems through the CPIC process (e.g., integration of
environmental data, electronic reporting, duplicate systems, Geospatial Information, data
standards, and data management). The new CIO's actions show that she agrees. For
example, EPA used the CPIC process findings to stop operating funds for the
Geographical Information Systems' investment.
Recommendations
We recommend the Chief Information Officer:
2-1. Assign sufficient resources and expertise to ensure timely and effective
implementation of report recommendations.
2-2. Continue with strategy to develop and execute a comprehensive, prioritized,
multi-year plan to address gaps and bring EPA's IT policy collection to the
"should be" state. In particular, the plan should include appropriate practices for
the Enterprise Architecture, CPIC process, and IT acquisitions addressed in the
Clinger-Cohen Act, OMB guidance, and EPA Delegation 1-84.
2-3. Continue to work with the Director for Acquisition Management to (a) direct
contracting officers and other procurement personnel to only accept procurement
requests with a formal CIO approval or officially re-delegated procurement
authority; and (b) establish interim delegations, policies and procedures for IT
procurement, until formal re-delegations are revised and implemented.
Agency Response
The CIO agreed overall with the emphasis placed on establishing an effective IT
resource investment program. However, the CIO identified specific findings and
recommendations that the CIO did not believe reflected recent Agency
accomplishments.
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
OIG Evaluation
We made changes to the report findings and recommendations based on the CIO's
response, acknowledging that accomplishments not previously noted were due to
(1) recently-completed actions, and (2) EPA's evolving IT investment process,
procedures, and selection criteria. While we updated the report's information based on
management's comments, we believe significant issues still need to be addressed to
institutionalize the Act's requirements. Establishing Agency policies and procedures is
only the first step. Monitoring and evaluating IT investments against a set of minimum,
critical criteria can ensure the institution is operating as desired for IT capital investments.
Furthermore, formalizing the Chief Technology Officer and Chief Architect authorities
and responsibilities should help ensure adequate resources are dedicated to the
completion and maintenance of the Enterprise Architecture. Then, monitoring and
evaluation of IT investments can provide a basis to recommend modifications to the
Agency's Enterprise Architecture. The CIO has established an ambitious schedule to
address this report's recommendations and, to succeed, EPA will need to continue
dedicating significant resources for planning, procuring, monitoring, and evaluating FT
investments.
10
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 3
Weaknesses in CPIC Process
Place EPA's IT Investments at Risk
The Agency's CPIC process was inadequate to properly manage EPA's IT investments.
Most of EPA's major fiscal 2002 IT investment proposal projects are high risk and
operating with little oversight. Moreover, projects are not evaluated upon completion.
In total, the fiscal 2002 budget submission indicated EPA was planning to spend $449.4
million for IT investments, including $203.2 million for major projects. EPA's fiscal
2002 CPIC investment portfolio process was primarily a peer review risk assessment
process that: used constantly evolving Agency-wide priorities for selection, provided
little oversight of individual projects' execution during the Control phase, and did not
evaluate the adequacy of completed projects in an Evaluation phase. EPA's fiscal 2003
CPIC process was basically the same. As a result, as discussed in Chapter 2, the
Agency may have invested resources on outdated systems that did not maximize the
efficiency or resolve long-standmg problems, such as integration of environmental data.
Numerous Documents Provide Federal Guidance
OMB provides the primary Federal guidance in Circular A-130, Management of
Federal Information Resources; Circular A-l 1, Preparing and Submitting Budget
Estimates; and Circular
A-94, Guidelines and
Discount Rates for Cost-
Benefit Analysis of
Federal Programs. The
CIO Council and General
Accounting Office (GAO)
have both published
additional Federal
guidance that describes
the process. GAO
provides an illustration of
this process (see figure 2)
in Information
Technology Investment
Management: An
Overview of GAO's
Assessment Framework
(Exposure Draft),
GAO/AIMD-00-155,
May 2000.
Figure 2. IT Capital Planning and Investment Control
Process
11
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Existing CPIC Process Inadequate to Manage
EPA's IT Investments
EPA's IT CPIC process did not adequately select, control, and evaluate the appropriate
mix of IT capital investments using objective, risk-based criteria consistent with the
Agency's Enterprise Architecture and IRM Strategic Plan. Under the current process,
EPA's Chief Financial Officer prepares three exhibits (52, 53, and 300b), at varying
times of the fiscal year, for EPA's annual IT Budget submission. OEI's Information
Investment Subcommittee (US) considers the results of an annual risk assessment review
of the major investment proposals listed in Exhibit 300b and, during the Select phase,
makes funding recommendations to the Quality Information Council and CIO.
However, EPA's CPIC process provides little oversight of individual projects' execution
during the Control phase and does not evaluate the adequacy of completed projects in
an Evaluation phase, as recommended in Figure 2. The peer review risk assessment
was the most substantive and documented process that EPA used to objectively manage
annual IT investments. However, at the nS review level, we found a decision process
that lacked adequate evidence to (1) substantiate subjective executive decisions that
differed from peer review recommendations, and (2) describe how discrepancies
identified by the peer review were resolved.
As such, Agency management planning and budgeting recommendations for fiscal 2002
appeared to be based on nS opinion, rather than the objective peer review risk
evaluations. While the peer review process objectively quantified and documented risk
determinations, we could not adequately substantiate the basis for the nS votes which
lowered the risk assigned to investment proposals by the peer review process.
Nevertheless, the IIS recommended funding the proposals to the Quality Information
Council and CIO. The same basic CPIC process was used for EPA's fiscal 2003 IT
Budget submission, although specific criteria for the peer review process changed. Our
review of the three specific phases disclosed the following:
Select Phase
Recommendations Not Supportable or Justified. Many IIS recommendations were
not supportable based on objective criteria. We evaluated information from EPA's
Exhibit 300b IT budget submission, the major IT project document; OMB's risk
analyses of that submission; and EPA's internal CPIC Peer Review risk assessment.
From those sources, we summarized the investment proposal responses, focusing on
4 key risk factors for the 48 major IT proposals listed in EPA's fiscal 2002 budget
submission to OMB. We compared the 48 investment proposals to the results of EPA's
CPIC Peer Review risk assessment, OMB's risk assessment report card, and our
assessment for the 4 key control areas. OMB clarified that they considered projects to
be high risk if they did not demonstrate compliance with key requirements, or the
information provided was not sufficient to determine the risks. OMB's risk assessment
report card reflected that, overall, 89 percent of EPA's major projects were high risk,
12
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
while EPA's Peer Review assessed that only 8 percent were high risk. Our assessment
concluded that all 48 proposals were high risk, based primarily on the fact that the
Agency had not provided an Enterprise Architecture for IT managers to use in preparing
IT investment proposals. In spite of not having an Enterprise Architecture, all the
proposals nonetheless indicated they were aligned with an Architecture. Details on our
comparison are in the following table.
EPA Major Investment Proposals
Key Project Risk Factors
(Fiscal 2002)
Key Risk Factors
Percentage of IT projects not aligned
with Enterprise Architecture
Percentage of IT projects not including
adequate security planning or when not
clear
Percentage of IT projects not including
a completed current cost-benefit
analysis or when not clear
Percentage of IT projects not having
approved system management plan or
when not clear
Percentage of High-Risk
IT Investment Proposals
OMB
Assessment '
100%
4%
100%
N/A
89%
OIG
Assessment
100%
33%
56%
48%
100%
Peer Risk
Assessment
N/A
56%
40%
42%
8%
The nS reviewed the internal risk assessments and agreed with the conclusions that
some of these projects were high risk. Nevertheless, the IIS recommended to the
Quality Information Council and the CIO that all 48 projects be recommended for
funding in the fiscal 2002 budget submission. OEI told us that these projects were
recommended for funding only after substantial corrective actions were taken to make
the business case, and a fourth review of the project proposal was conducted.
1 We calculated percentages based on raw data (# of projects) and footnote information
associated with the "major" projects (Steady State, Mixed and Development/Modernization/
Enhancement), as taken from documents provided to EPA by OMB regarding the Agency's
fiscal 2002 IT budget submission (dated July 26, 2001).
13
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Significantly Deficient Projects Recommended for Funding. In spite of the risk
assessment process, all the projects with significant weaknesses were recommended for
funding in the fiscal 2002 and 2003 IT budget submissions. For example, major projects
were found to have significant weaknesses by the peer review process. The nS
downgraded these projects from "red light" to "yellow light" in fiscal 2002, but we found
no evidence of how the significant deficiencies were resolved. In fiscal 2003, the peer
review process once again stated these projects contained significant weaknesses. Once
again, the IIS recommended them for funding. The documentation provided did not
contain clear, objective evidence from which we could conclude whether the cited
deficiencies had evolved during the 2-year span or simply remained unchanged. Our
analysis was confined by the fact that the risk assessments used different documentation
and evaluation requirements each year. The CPIC process should rely on one minimum
set of consistent objective criteria applied throughout all levels of the selection review
hierarchy.
Inconsistencies Noted. The narrative for the CPIC IT budget submissions were
unclear about the Enterprise Architecture and conflicted with the Agency's fiscal 2002
Annual Performance Plan goals. For example, EPA's key architectural project, the
Information Integration Program, refers to the Integrated Compliance Information System
(ICIS). The Enterprise Architect document states that ICIS is "being designed to
interface with only a few ... legacy systems, but the technology is scalable .. .."
However, the fiscal 2002 Annual Performance Goals discusses ICIS in terms of 14
existing systems. From these conflicting perspectives, it was not clear how the existing
legacy systems were to be integrated with the Enterprise Architecture strategic
framework.
Control Phase
EPA was not monitoring the execution of IT capital investment projects during the year,
thereby preventing the CIO from adequately managing ongoing IT investment projects.
In fiscal 2002, OMB established baselines to measure progress and performance for
projects' scheduled milestones and costs estimates. OMB required that agencies explain
schedule slippages and increased costs greater than 10 percent. EPA reports this
information in annual Exhibit 300b reports. However, common industry practice is to
use a 4 week time frame for monitoring and measuring variances from the project plan.
In our opinion, the Agency should monitor the execution of its projects through periodic
reports (at least quarterly) that managers can use to identify emerging cost or schedule
problems and initiate compensating actions.
Evaluate Phase
The Agency did not perform any post-implementation reviews or evaluations of
completed IT projects. EPA's OEI has taken steps to implement a Post-Implementation
Review Phase. In addition, management prepared a list of completed or terminated
14
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
projects that would require review for the first time during the fiscal year 2003 CPIC
process.
CPIC Management Problems Stem from Several Causes
Many factors have contributed to the ineffectiveness of EPA's current CPIC process, as
discussed below.
CIO Needs to Institutionalize a CPIC Process
In June 2002, EPA issued EPA Order # 2100. A. 1 to formally recognize CPIC policies in
the Agency Directives. As a next step, the CIO needs to establish Agency-related CPIC
procedures and guidance.
Insufficient Staff Dedicated to CPIC Process
In our opinion, the CIO had not dedicated sufficient resources to administering a fully
functional CPIC process. The lack of administrative and financial resources restricted
EPA's capability to implement a comprehensive system for managing its IT investment
portfolio. The Agency's IT program for fiscal 2002 totaled $449 million. Yet, the CIO
only established two full time positions (team leader and one staff) as the primary
resources to implement and execute an EPA CPIC process. The permanent positions
were supplemented by an ad hoc team for the peer risk assessment and the review of
proposals by the US. EPA should assign sufficient resources and expertise to address IT
acquisition and development.
Implementing I-TIPS Would Structure CPIC Process
Implementing the Federally-sponsored I-TIPS software, an automated investment control
and reporting system, would provide EPA with a valuable tool for monitoring and
managing its IT investment portfolio. This tool already is being used by more than half of
major Federal agencies. Implementing I-TIPS would help EPA select IT proposals,
monitor the execution of funded IT projects, and electronically report IT investment
submissions to OMB.
Although EPA's OEI appeared to seriously consider using I-TIPS, during the review
cycle, management could not provide evidence to support that they planned to implement
the software product in the near future. In March 1999, OEI conducted a study, Report
on the Results of I-TIPS Process Analysis and Feasibility. Then, in 2001, EPA
purchased a Service Level Agreement for I-TIPS. In response to the draft report, OEI
indicated that it would use I-TIPS during the current budget cycle for generating reports
to OMB. Agency-wide implementation of the product is tentatively scheduled for the
fiscal 2005 budget cycle.
15
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Recommendations
We recommend the Chief Information Officer:
3-1. Assign sufficient staff to develop a formal manual for the CPIC process in the
EPA Directives system, and cross reference it to updated IT policies in Directive
2100 on budget, management, procurement, and the System Development Life
Cycle. At a minimum, the manual should include:
(a) a description of how IT investments are linked to the Enterprise
Architecture and IRM Strategic Plan,
(b) a minimum set of mandatory obj ective, risk-based criteria for use by both
the technical peer review and the nS review for the Agency's IT
investment portfolio.
(c) performance measures for monitoring and evaluating progress on IT
investments, and
(d) provisions for post-implementation review and evaluation of IT
investments.
3-2. Direct the IIS to not recommend funding IT projects identified by the Peer
Review process as having significant weaknesses (i.e., do not meet the minimum
established requirements) or duplicating existing projects, until critical deficiencies
are resolved and the resolution steps adequately documented. In addition, nS
should clearly document how all risk weaknesses identified by the peer review
are addressed and/or resolved prior to the Subcommittee making
recommendations to fund projects to the Quality Information Council and CIO.
3 -3. Direct the Information Investment Subcommittee to monitor the execution of IT
projects during the fiscal year (at least quarterly) to identify emerging cost or
schedule problems and initiate corrective actions.
3-4. Initiate a formal process with written evaluations of ongoing, completed, and
terminated information technology projects to evaluate whether the projects or
systems are successfully delivering promised benefits at an acceptable cost.
3 -5. Complete implementation of an automated portfolio management system (e.g., I-
TTPS) to provide timely, reliable information for investment decisions.
Agency Response
The CIO's response noted that OEI has issued formal criteria for the CPIC process each
year since the requirement began. The CIO also stated that EPA used a highly
structured approach for its annual data call, although that process continued to evolve
16
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
from year to year. Lastly, the CIO indicated that EPA expects to integrate updated
OMB Circular A-l 1 requirements and the Agency's Enterprise Architecture into the next
IT investment review cycle.
OIG Evaluation
Based on the CIO's response and additional discussions with management, we amended
the report and its recommendations. The primary area of confusion relates to our use of
the terms 'formally establish' and 'structured process.' We agree that EPA annually
issued formal guidance and criteria for the annual budget data call for the years under
review. The use of an annual data call may be structured for that one year, but evolving
criteria from year to year does not provide an adequate baseline for evaluating progress
from year to year. Also, this was the first time the CPIC process used a risk-based
process, and it was for the purpose of producing risk-ranked budget data. However, the
Act intended a portfolio management process, not simply a risk-ranking of projects in the
annual budget data call.
We modified the report to clarify our intent for the phrases 'formally establish' and
'structured process.' Generally, our concern was the need for formal policies and
procedures to establish a consistent management structure. Without this management
structure for capital investments, EPA cannot establish a consistent baseline to evaluate
and prioritize IT projects over several years. This minimum baseline information is critical
for the CIO, IIS, Quality Information Council, and program managers when comparing
IT investments, preparing IT investment proposals, accumulating project costs, monitoring
the execution of IT investment projects, and evaluating completed projects.
17
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
18
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4
EPA Needs to Organize and Integrate
Planning for IT Investments
EPA's ability to organize and integrate planning for IT investments depends on the
quality and timing of several important factors. EPA must ensure that the Enterprise
Architecture is fully integrated with the Agency's Government Performance and Results
Act goals and objectives, IRM Strategic Plan, and IT acquisition processes.
Otherwise, EPA will continue to struggle with its ability to reinvent organization
processes, integrate and manage data, and build a scalable and reliable network
architecture. Although EPA has made some progress in developing an entity-wide
Enterprise Architecture, the Agency needs to do more to organize and integrate
planning for IT investments. For example, numerous essential components of the
Enterprise Architecture have not been fully addressed or integrated. EPA's fiscal 2003
and prior IT investments were not driven by business priorities to result in organizational
improvements. However, for the fiscal 2004 budget cycle, EPA's Enterprise
Architecture Team has provided guidance and worked closely with proposal preparers.
Background
During 2001, EPA completed many actions towards establishing a baseline enterprise
architecture for IT planning purposes. In April, EPA provided OMB with
documentation of EPA's first Enterprise Architecture, dated March 29, 2001. The
document was not provided to EPA program offices until an Agency-wide conference
in July 2001, about 2 months after the IT investment proposals for the fiscal 2003
budget submission were submitted for the Agency CPIC review process. Furthermore,
when the OIG met with EPA's Office of Acquisition Management in October 2001,
neither the IT Contracting Officer nor the Procurement Office were aware of the
document.
OMB reviewed the Agency's fiscal 2001 IT Investment Portfolio and noted that they
could not match the projects in the proposed Enterprise Architecture to the portfolio.
In August 2001, OEI established a workgroup to identify and verify EPA's business
processes for the Enterprise Architecture baseline. The work group's efforts occurred
after completion of our field work, although we were informed that the group is
updating the business processes and aligning them with OMB's Business Reference
Model.
19
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Executive Buy-in and Management Controls Required
The Chief Information Officer Council recognizes the importance of executive buy-in and
support to the IT investment process. The Council also states that an organization should
create an architectural team to define and integrate the components. The enterprise
architecture is an expansion of the IRM strategic plan that provides an enterprise view of
information technology in the context of EPA's business environment. The enterprise
architecture defines the current and target (future) components. A transition plan
sequences the evolution from current to target. As such, the enterprise architecture
should be a document that is continuously modified and maintained to reflect the
Agency's current baseline and target business practices, organizational goals, visions,
technology, and infrastructure. Figure 3 below depicts the major components of the
Enterprise Architecture that must be addressed to accomplish EPA's strategic goals and
perform its business.
bnwranmenflal
Business
flrthlecture
x
/ Frccasces m
Techrdogy
Figure 3. Enterprise Architecture Framework
Various Components Essential to Quality of IT Planning
EPA's ability to organize and integrate planning for IT investments depends on the quality
and timing of several important factors. Clearly defining the Enterprise Architecture is
particularly important because it provides the conceptual framework for integrating the
Agency's information technology environment and core business processes to accomplish
strategic goals. In the following subsections, we present issues that EPA management
must address to ensure the integrity and effectiveness of its IT investment planning system.
20
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
IRM Strategic Plan Goals Need to be Incorporated
into the Enterprise Architecture
EPA needs to incorporate the updated IRM Strategic Plan goals into a target enterprise
architecture. During our review, EPA was severely criticized by Congress, National
Academy for Public Administration, GAO, and environmental and industry groups for not
having such a plan. On July 29, 2002, the Agency completed its revised plan: EPA
Strategic Information Plan: A Framework For The Future.
EPA Has Yet to Fully Baseline its Business Processes
As of the end of field work, EPA had yet to fully baseline and validate the Agency's
business processes essential for establishing a portfolio for future IT investments. EPA's
draft Enterprise Architecture document included very high-level business processes;
however, these processes had yet to be validated by the responsible program offices.
We were informed that some of these business processes have been revised, but were
unable to substantiate whether the applicable program offices formally endorsed the work
group's conclusions. EPA understands the importance of this activity, and plans to
perform a validation process this year.
Draft Enterprise Architecture Baseline Security Architecture
Needs to be Expanded
Although OEI's draft baseline Security Architecture addresses many pertinent risks in
EPA's Security program, it does not adequately address two important components:
facility physical security and personnel security requirements. The Enterprise Architecture
document states the Agency maintains a security infrastructure of approximately 1,600
servers for network support, application hosting, scientific computing, and graphics. OEI
centrally supports these servers. The document also indicates that the Agency owns an
additional 900 servers not supported by OEI personnel, but it does not adequately
address who supports these servers. OEI confirmed that these servers store sensitive
data. Therefore, the physical and personnel security requirements of these servers need
to be added into the baseline security architecture.
Key Data Needs to be Developed, Analyzed, and Controlled
As shown in Figure 3, the Enterprise Architecture conceptual framework should consist
of five components. As such, the Enterprise Architecture should define mission-critical
data needs to properly support the IT investment process. However, the draft Enterprise
Architecture plan we reviewed did not (1) specifically recognize (i.e., require)
21
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
individual Agency data standards and related metadata2 baseline information, and
(2) adequately address other critical data used by stakeholders and programs business
processes. EPA states it will address program-specific data needs across several
dimensions.
As of the end of field work, EPA had approved six Agency data standards, and recently
it adopted a seventh standard. In addition, the Office of Water had implemented some
program data standards. Although these efforts were underway, EPA's intended
infrastructure for managing and sharing environmental data did not adequately address
how EPA's program users and stakeholders were to use existing and future data
registries to manage data. In fact, this issue has been a long-standing OIG concern, as
noted in a prior report, Information Resources Management: Office of Water Data
Integration Efforts (No. 8100177), dated June 22, 1998. We had recommended that
EPA support its data standards program by using the Environmental Data Registry as a
central repository for publishing and recording Agency data standards. The Enterprise
Architecture Plan we reviewed did not incorporate this recommendation. However, EPA
states that its current draft version of the Enterprise Architecture clearly describes the
registry as a critical component of its target architecture.
In its draft Enterprise Architecture, EPA recognizes that more detailed descriptions of
critical data are necessary. Among other things, EPA will need to validate the information
flow and relationships, as well as data descriptions and relationships, described in the
initial Enterprise Architecture. Without this step, EPA cannot begin to establish a target
architecture and define the required sequencing plan for migrating from the baseline to the
target architecture.
Complete Inventory of Systems Needed for Enterprise Architecture
EPA needs to complete an update of its inventory of general and application information
systems. This baseline of systems should identify current critical business processes,
related systems (major and significant), and mission-critical data in those systems. At
that point, the baseline can be used to identify IT investment projects that will meet the
Agency's current needs, eliminate redundant systems, and build an IT structure to
accomplish EPA's goals. However, we noted a number of inconsistent inventories.
EPA's March 2001 submission to OMB included a Year 2000 Systems Inventory that
listed 70 major and significant application systems. However, the Enterprise
Architecture, dated March 2001, only listed 46 major systems. In September 2001, the
CIO reported to OMB in its On Implementation of the Government Information
Security Reform Act report that it had 189 systems. In its response to the draft
report, OEI stated the Enterprise Architecture will incorporate all systems into an
Information Resources Registry System, which is scheduled to be operational by the
2 Explanation of specific data fields, including information regarding its source, collection
method(s), and in what context the data can be used.
22
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
end of fiscal 2002. OEI did not indicate how long it would take to fully populate the
Registry System. OEI also plans to link the Registry System and the Enterprise
Architecture.
In addition, the Enterprise Architecture document states that sufficient information on
Agency application interfaces is not available. The document states the CIO plans to
gather and document this information as part of the Agency's ongoing application
inventory initiative, including documentation regarding major interfaces with applications
outside of the Agency. For example, this year, EPA intends to gather more information
on internal system interfaces and partner interfaces within the framework of its National
Environmental Information Exchange Network.
Enterprise Architecture Needs to Address Scalability
of Virtual Private Network
The draft Enterprise Architecture does not adequately address EPA's existing and future
technology components for its next-generation wide area network. The Agency needs
to address "scalability" and Virtual Private Network (VPN) concepts to grow with the
Agency's evolving needs. Scalability refers to the ability to expand a network to
accommodate future needs; a VPN is an electronic network, without physical limitations,
specifically designed to secure transmissions. With regard to scalability, the Enterprise
Architecture document did not explicitly identify minimum response times for key
transaction-based systems and for business application systems on the Agency's wide
area network. Moreover, EPA's July 2001 Network Requirements Study indicated that
bandwidth utilization for some circuits experienced bottlenecks for certain portions of the
network and responsiveness for newer systems ranged from very poor to good. Also,
whereas management has recognized the need for virtual private networks, they only
reference it in light of long-term needs. We believe the VPN concept is needed today to
help the Agency comply with existing Federal telecommuting statutory requirements and
to satisfy current business needs.
We agree with Agency officials that technical issues, such as transaction response
requirements and scalability, normally are addressed in a Technical Architecture. OEI's
response to the draft report mentioned a "Technical Reference Model" and, we agree,
that may be a suitable planning document in which to address these issues. OEI agrees
with the importance of secure external communications and states they will take critical
steps to start implementing VPNs next year and, pending available resources, will make
full operations available on an enterprise basis in 2004.
Enterprise Architecture Should Address Middleware
EPA's Enterprise Architecture should identify the middleware architecture needed to
address those client-server systems already implemented, as well as those envisioned
and planned to strengthen the overall usability of the distributed architecture.
23
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Middleware architecture includes such things as message brokers, extensible Markup
Language, and directory structures used to facilitate interconnection of systems and
applications. EPA's draft Enterprise Architecture overlooked this aspect of IT planning,
but management may want to address these topics as part of the "Technical Reference
Model" mentioned in OEFs response to the draft report. To minimize the risk of
incompatible communications, a standard middleware architecture could greatly benefit
application developers with a single consistent interface for both inter- and intra-
application communications.
Various Causes Contributed to Lack of Planning
No Central Planning Organization or Appointed Authority
EPA's IT planning activities suffered from a lack of a central organization and authority.
EPA's IT planning is currently managed using a decentralized and fragmented structure
involving numerous individuals and offices. Agency-level coordination was generally
accomplished through project briefings to the Quality Information Council and its four
subcommittees. With regard to the fiscal 2002 budget process, informal meeting minutes
would support that the Council deferred formal management planning decisions in lieu of
receiving briefings by numerous project managers and the Council's subcommittees.
Also, EPA needs to define the role and authority of its Chief Architect for IRM. The
role of this Chief Architect is to oversee development and coordination of the Enterprise
Architecture with other planning elements that should materially shape and drive the IT
planning structure. The CIO named an individual to this role in February 2002 (via
electronic mail), but there has been no formal definition of the position's scope and
responsibilities in policy, nor any official delegation of authority.
Further, we identified several IT planning-related, Agency-wide documents, projects,
and work groups that should be coordinated to ensure their individual visions and plans
are aligned. Together they will enable EPA to optimally execute its program goals and
deliver environmental and human health improvements.
To EPA's credit, management established a central Enterprise Architecture workgroup
in August 2001. While EPA has planned activities to coordinate and develop the
Enterprise Architecture, management must also establish a permanent central
organization with dedicated resources and assigned responsibility to maintain this living
document. Agency-wide Enterprise Architecture components need to be addressed and
maintained for the following functional areas: the identification of EPA's major and
significant systems; defining the security architecture; validating the business processes
with program offices; developing the Middleware architecture and defining baseline
telecommunication requirements; defining Working Capital Fund capital investments; and
approving individual IT project management plans for major projects or systems.
24
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Finalizing Information Integrated Program Plan Needed
In its fiscal 2003 budget submission, EPA identified the Information Integration Program
as its only major architectural project for deriving and completing an enterprise
architecture. As critical as the project is to EPA's Enterprise Architecture development
efforts, no final management work plan has been implemented for this project since the
draft was issued in December 2000. Management is required to issue a final, approved
work plan in accordance with Agency Directive 2100, and should do so to ensure the
timely success of the individual program, as well as the overall quality of the Enterprise
Architecture Plan and the Agency's future technology investments.
The Chief Architect provided information that indicates EPA's program and regional
offices will be asked to co-develop the Agency's baseline and target elements for the
Enterprise Architecture. With OEI's leadership and facilitation, the program and
regional offices will conduct their own architectural needs analysis, and realign their
respective systems with EPA's evolving target. During our fieldwork, we were unable
to substantiate how this will be accomplished. In OEI's response to the draft report,
management assured us that participants have been informed of their roles and
responsibilities. In addition, they stated the Chief Architect is developing explicit
guidance to formalize roles and responsibilities for regional and program offices.
Management also stated that the Enterprise Architecture was scheduled for completion
by October 2002.
Recommendations
As the number one priority, we recommend that the Chief Information Officer direct the
Chief Technology Officer to:
4-1. Formally institutionalize:
(a) in policy the Enterprise Architecture program to plan, manage, monitor,
and control the development and maintenance of the Enterprise
Architecture plan.
(b) the Chief Architect position by clearly defining and documenting the
roles, responsibilities, and authority of the job in policy or through a
delegation.
Next, we recommend the CIO target the following key actions to complete the
Agency's baseline and future plans for the Enterprise Architecture:
4-2. Establish a permanent organization under the leadership of the Deputy Chief
Information Officer for Technology to update and maintain the Enterprise
Architecture in accordance with the Agency IRM Strategic Plan and its
Government Performance and Results Act requirements.
25
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
4-3. Identify current major and significant general and application systems to establish
an accurate inventory of such systems and integrate this information with both the
Agency's Enterprise Architecture application component and the IT CPIC
Portfolio.
4-4. Complete the project to publish an updated Enterprise Architecture and
document the project as required by Agency policy.
4-5. Finish implementing a robust Agency information repository and
(o) require the use of the data registry for Agency maintained data,
(p) map EPA's data and information resources, and
(q) complete on-going efforts to adopt life-cycle data management
principles for the Enterprise Architecture data and systems components.
The CIO should implement the following recommendations as the Enterprise
Architecture is developed and updated:
4-6. Use a top management verification, validation, and approval process to ensure
program business processes and goals are accurately reflected and incorporated
into the Enterprise Architecture. Subsequently, formalize the process as a
discipline for updating the Enterprise Architecture document.
4-7. In coordination with the Office of Acquisition Management, jointly develop an
approval process that ensures the Enterprise Architecture concept is
incorporated in future IT contract activities for large and significant IT projects.
4-8. As part of a Technical Reference Model or Technology Architecture, address
technology components, such as interfaces, transaction response times, and
baseline telecommunications requirements to support a scalable, reliable, and
secure network infrastructure for the Enterprise Architecture.
Agency Response
The CIO generally agreed with our recommendations, but believed many actions
currently underway were not recognized in the report's findings. OEI had made
progress in addressing our concerns and, therefore, the CIO suggested that we revise
specific findings or recommendations to reflect recent accomplishments.
OIG Evaluation
We made changes to the report findings and recommendations based on the CIO's
response, acknowledging recently-completed actions and planned activities. We agree
that EPA has taken significant first steps to address our report's findings and
recommendations on IT planning. However, many actions were initiated after we
finished audit field work, and some actions are still in progress.
26
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
We attempted to be as specific as possible in our recommendations to provide
appropriate direction and recognize current ongoing efforts. For example, we agreed
that some of the technical components can be addressed appropriately in a Technical
Reference Model or Technology Architecture, rather than the Enterprise Architecture,
and amended the recommendation accordingly. The CIO has established an ambitious
schedule to address this report's recommendations, and it will require a significant
amount of dedicated resources to not only complete them, but to maintain the EPA's
planning structure for IT capital investments.
27
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
28
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 5
EPA Needs To Strengthen
IT Project Management Controls
For the six EPA IT major projects reviewed, we found significant project management
control weaknesses, a lack of compliance with Agency system development policies,
and inaccurate project status information reported on the Clinger-Cohen budget
submission. EPA incorrectly reported an approved System Management Plan (SMP)
was being followed for projects. Further, SMPs were either out of date or had never
been formally approved and signed. We also found significant variability in EPA's
working capital fund expenditures, which adversely impacted the system development
project's planning and budgeting activities. Several key factors contributed to the lack
of management controls over IT projects:
• OEI had not updated IRM policies or established interim guidance to convey new
requirements, and project managers did not practice existing policies;
• managers were not using a phased, sequential system development process;
• EPA had not adopted standard tools for reliably managing IT project information
resources, schedules, products, and costs; and
• until fiscal 2002, EPA had not provided a means for project managers to track
project and contractor support costs.
The CIO needs to establish controls to monitor project managers and ensure they use
key management controls (e.g., SMPs), and maintain current cost-benefit analyses and
project cost records. Otherwise, the CIO has little assurance that IT investment
projects represent cost-effective solutions.
Primary System Guidance
OMB Circular A-130, Management of Federal Information Resources
Management, establishes requirements for:
• preparing and updating a cost-benefit analysis for each information system
throughout its life cycle;
• conducting post implementation reviews of information systems development
projects to validate benefits; and
establishing an oversight mechanism to ensure major systems development projects
proceed in a timely fashion toward agreed-upon milestones and deliver intended
benefits.
OMB Circular A-11, Preparing and Submitting Budget Estimates, required two
reports for fiscal 2002 budget submissions:
29
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Section 53. This report summarizes an agency's IT portfolio by listing major and
significant capital investments for IT system, infrastructure, and architecture
projects.
• Section 300. This is a separate planning and justification report for each major
capital investment with a useful life of 2 or more years. Agencies are expected to
establish and measure baseline costs, establish a measurable project schedule, and
ensure projects support performance goals.
OMB Circular A-127, Financial Management Systems, Parts 6 and 7, address
financial system requirements. EPA Directive 2100, Chapter 17, identifies an eight-
stage life cycle methodology, and establishes specific thresholds for formal review and
approval of an SMP for system development or enhancement projects.
Documents Incorrectly Reported
In its fiscal 2002 and 2003 CPIC project submissions, EPA managers misrepresented
the status of key management documents. We reviewed documentation for three of six
selected projects. We could not audit two infrastructure projects because, despite
repeated requests, EPA managers did not furnish adequate supporting documentation.
The sixth, which was EPA's current architecture project, Integrated Information Project,
did not have a current, approved SMP. Following are examples of what we found:
SMPs
The SMP for AIRS-AQS (Aerometric Information Retrieval System - Air Quality
System) had not been updated since originally prepared in 1996. Maintaining a
current and formally approved SMP is important because it discloses significant
changes to the system development project and ensures accountability.
• As of December 17, 2001, the SMP document for the RCRAInfo (currently
defined as the Resource Conservation and Recovery Act Information Management
System and Waste Information Needs/Informed) did not include the Assistant
Administrator's signature approving the project and key decisions, as required by
EPA Directive 2100. Project management attributed the lack of signed hard copies
to a reliance on electronic documents and e-mail to manage meeting minutes and
decision notes.
Cost-Benefit Analyses
• Project management stated that, given the modular nature of the RCRAInfo project,
cost benefit analyses were performed for each major component rather than for the
project as a whole. EPA's fiscal 2003 investment submission for this project
disclosed total life cycle costs of $70.5 million, an increase of $40.4 million over
previously projected costs. Management attributed the increase to: estimated
30
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
regional and state costs, changes to working capital fund rates, and adding years to
the system life cycle. An updated cost-benefits analysis would help determine the
most cost-effective strategy for implementing the RCRAInfo investment.
• The cost-benefits analysis for SDWIS/STATE (Safe Drinking Water Information
System/State Version Modernization Effort) had not been updated since 1992,
despite many changes in design, functionality, and plans to migrate to a web-
enabled application. The outdated analysis erroneously leads EPA management to
believe that the original return on investment will still be achieved. An updated cost-
benefit analysis should be completed as extra functionality is added to the system,
such as the planned integration of SDWIS/STATE into the Agency's Central Data
Exchange initiative.
Primary Architecture Project Lacks Plan
Although EPA's Information Integration Program is the heart of EPA's Enterprise
Architecture and planning investment strategy, EPA did not recognize the Program as a
separate architectural project until the fiscal 2003 budget submission, provided
September 2001. As such, no project plan had been finalized to define the vision,
scope, or implementation and cost schedules for this architectural project. The project
plan would help management ensure that the intended benefits of this complex endeavor
do not outweigh the projected costs, as well as provide specified time frames for
completing detailed tasks and products.
Project Managers Not Adequately Monitoring Status
EPA project managers were not adequately monitoring the execution of IT capital
investment projects. EPA's 300b IT investments reports showed that projects
consistently did not meet cost estimates, scheduled milestones, and planned
performance. We compared planned expenditures for 46 IT investment projects in
fiscal 2001 against their corresponding actual costs, and found that 37 percent showed
more than a 10 percent increase. Furthermore, the investment reports indicated that 78
percent of these projects experienced milestone slippages greater than 10 percent. The
data strongly indicates project managers need better standard management tools.
Many Factors Negatively Impact Management of IT Investments
Numerous factors contributed to the inconsistency of management controls for IT
investment projects. These concerns were voiced by many of the project managers
interviewed.
IT Project Managers Need Standard Tools
For the period reviewed, EPA had not adopted standard project management tools to
help managers plan, control, and evaluate IT investment projects and track project
31
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
costs, schedules, and resources. SDWIS/STATE is an example of a project that could
have been managed better with the help of a project management tool. Standard project
management tools help promote a consistent and uniform approach to tracking and
managing all forms of project and contractor support costs. A standard tool helps to
reduce the communication gap between contractor support activities and what the
Agency reported for this IT investment.
IT Projects Not Using A Phased Sequential Project Life Cycle
The status of a project is often unclear because project managers do not use a
sequential, phased development process to clearly distinguish where one series of system
development life cycle activities ends and another series begins.
EPA Directive 2100, Chapter 17, requires that system development projects follow a
sequential, phased systems development life cycle called the "waterfall" method. This
method consists of eight sequential stages. Any planned new functionality should be
considered a new project, and a new project also should be established when estimated
costs exceed stipulated dollar thresholds.
Industry recognizes at least three other models for systems development that are
sequentially-phased from a project perspective. These approaches are generally
referred to as: (a) spiral, (b) prototype, and (c) rapid application development models.
Spiral modeling works as a repeating waterfall approach, with a risk analysis at every
stage to determine whether cost overruns, schedule delays, or changing requirements will
impact the benefits of proceeding. Prototyping uses existing software and lets a group of
users define the system requirements for an organization. Rapid application development
is based on reusing and modifying software components until they perform as desired.
The projects reviewed did not demonstrate any of these acceptable "phased" software
development approaches. Rather, we found that EPA generally used an evolutionary
approach in which management continuously added requirements to the overall system
development project. For example, the RCRAInfo project was simultaneously in more
than one stage of the system development life cycle, and management could not
distinguish the cumulative costs associated with one set of activities versus another. The
project is very broad and encompasses five program area requirements. In 1999,
contractors completed the first system development life cycle stage (i.e., the
Requirements Analysis) for three of the five areas, while the two most critical functional
requirements remained in the first stage. Despite several years of effort, management
was still defining RCRAInfo requirements. Business needs can change based on
technology advances, so best practices suggest that requirements be defined in less than
2 years. We believe management should have split the program area requirements into
two or more distinct projects, so development efforts could progress in a timely fashion
from one stage to the next, and managers could easily track associated costs and
schedules.
32
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Evolving Nature ofEPA's Exchange Network
The evolving nature ofEPA's architecture project deterred management from finalizing
its formal project plan to ensure the cost-effective and timely execution of the Exchange
Network. What is now referred to as the Information Integration Project represents the
third iteration of the project, and the objectives and intended outcomes have undergone
several revisions. Also, the number of infrastructure projects (e.g., registries) affecting
the Information Integration Project have been evolving, and management must clarify the
role these supporting projects play.
Minimal Assurance that IT Investments
are Cost-Effective and Controlled
The absence of key decision documents and senior management approval (e.g., cost
benefit documents, management decision papers, system management plans) increase
the risk that funded IT projects will evolve in an unstructured, untimely, and costly
manner. Furthermore, expanding and/or changing original project objectives to
incorporate evolving business functions results in confusion, complications for proper
cost accumulation, and slipped project development time lines and even system
development projects that never come to closure. In addition, if projects are too broad
in scope to progress through the life cycle in a timely manner, then what originally was
thought to be a cost-effective solution may become a bad return on investment. Further,
the lack of project management tools inhibits project managers' ability to provide reliable
data on a project's status, and contributes to unjustified delays and unsupported cost
overruns on IT projects. Chapter 2 contains additional effects relating to EPA's
inadequate oversight processes.
Recommendations
We recommend the Chief Information Officer:
5-1. Monitor IT investments to ensure that SMPs are prepared in accordance with
Agency requirements, and that they appropriately link the Enterprise
Architecture and other planning documents to the Clinger-Cohen Act submission
documents.
5-2. As part of a monitoring process, re-evaluate funding for IT investments at least
quarterly, to determine if they have exceeded budgeted costs or project
milestone schedules by more than 10 percent, and ensure that written
justifications sufficiently support continuing the project.
33
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
5-3. Prescribe that standard tools, such as I-TIPS and project cost accounting, be
used for managing projects for software development changes to IT systems and
project management. The selected tools should be approved by the Chief
Financial Officer as being compatible with the Agency's cost accounting and
financial systems.
We recommend the Air Quality System Project Manager:
5-4. Update the SMP for the Air Quality System project and obtain the signature of
approval of the Assistant Administrator for Air and Radiation at the conclusion
of the analysis stage for major and significant enhancements adding new
functionality.
We recommend the RCRA Information Project Manager:
5-5. Update the Project Management Plan for the RCRAInfo project to make it
equivalent to an SMP, for planned system design changes and enhancements
adding functionality. In addition, the SMP should be formally approved by the
Assistant Administrator for Solid Waste and Emergency Response to authorize
the IT investment and to ensure a system of accountability.
We recommend the SDWIS/STATE Project Manager:
5-6. Establish an SMP for the SDWIS/STATE project and obtain the signature of
approval from the Assistant Administrator for Water at the conclusion of the
analysis stage and for major and significant enhancements adding functionality.
We recommend the Project Managers for the Air Quality System, RCRAInfo, and
SDWIS/STATE:
5-7. Manage project development efforts in accordance with the SMP, as updated,
throughout the life cycle of the system, and retain the SMP for reference and
review by the CIO or the CIO's designated review official.
Agency Response
We received comments from several Agency officials in response to this chapter's
findings and recommendations. The CIO agreed to monitor IT investments and
expected to also establish a preselect phase. However, the CIO stated we had not
recognized that the current review process required monitoring a project as part of an
annual review. Further, the CIO did not agree that one set of project management tools
would be cost effective or meet all projects needs. The Assistant Administrator for
Solid Waste and Emergency Response and the Director of the Office of Air and
Radiation's Information Transfer and Program Integration Division both disagreed with
our conclusion that project management controls were inadequate.
34
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
OIG Evaluation
We made changes to this chapter based on the Agency's responses, as well as further
discussion with management officials. We had used a judgmental sample of the six
different kinds of major IT investment projects, and the sample accounted for over half
of the fiscal 2002 major IT projects' budgeted funding. We had completed a limited
survey, requested supporting documents, and interviewed key project managers.
However, we were unable to complete the survey and had to limit our scope of review
because three major system projects did not provide requested information. For the
three major system projects completing the survey, we did not (1) review all the
individual project's management controls, or (2) determine whether the individual
project accomplished the objectives identified in the budget submission.
Our review concentrated on project management controls and documentation
requirements in OMB Circulars and existing EPA System Development Life Cycle
policy. We were able to document inaccurate and/or unsupported information being
incorrectly reported by the three major IT system projects in the fiscal 2002 budget.
For example, the projects (1) did not adequately address OMB requirements by
consistently accumulating costs from year to year; (2) could not support total costs from
inception of the project; and (3) could not provide current cost-benefit studies
addressing costs, needs, and expected benefits. We also found that the projects could
not document compliance with existing Agency and Federal system requirements, such
as the development and top management approval of a current cost-benefit analysis.
Each project was using a different set of project management procedures for the day-to-
day execution of the project. We did not evaluate these local project controls. Still, we
believe that if EPA was monitoring the projects' execution (at least quarterly) and
evaluating completed IT projects, individual project managers would address these
critical management controls. Furthermore, if program managers are compelled to
report accurate data for critical management controls (e.g., emerging cost and schedule
overruns), then the CPIC peer review process can more accurately assess the risk of
successful completion for susceptible IT projects.
35
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
36
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 6
Project Cost Accounting System Vital for
Planning and Managing IT Investments
Although EPA implemented an IT project cost accounting methodology in fiscal 2002,
EPA managers previously relied on an inconsistent variety of informal cost accumulation
processes and records to oversee and measure progress on individual IT system
development or enhancement projects. Even now, the accuracy of captured IT costs
depends largely on the ability of non-technical staff to consistently and accurately
distinguish how IT costs fit into system life-cycle categories, and to appropriately code
funding documents. Accuracy also depends on contractors adequately identifying
specific software development costs.
Cost Accounting a Federal Requirement
Cost accounting data is required by Federal laws, standards, and Agency policies. The
Clinger-Cohen Act notes that before an IT investment is made, it is to be evaluated using
a risk-adjusted return on investment as well as other specific quantitative and qualitative
criteria. OMB Circular A-l 1 defines the life cycle phases to be used for reporting IT
costs and budgets. EPA Directive 2100 requires system managers to prepare a needs
assessment and SMP before a new system development or enhancement project can be
approved. Statement of Federal Financial Accounting Standard No. 10 requires
agencies to capitalize the full costs of internal use software.
Managers Did Not Have Necessary Project Information
Prior to the start of fiscal 2002, EPA did not have a standardized project cost
accounting methodology for managers to use in overseeing IT projects and systems
covered under the IT CPIC process. In the projects reviewed, we found that managers
relied on an inconsistent variety of informal cost accumulation processes and records to
identify expenses, assess changes to baseline costs and schedules, and measure progress
of individual IT development or enhancement projects. In addition, managers needed a
standard project management system to allow them to establish reasonable baselines for
projects, including tracking and managing project contractors' costs; accumulating labor,
working capital fund, and project hardware purchase costs; and controlling changes to
system milestones and documentation.
Effectiveness of Interim Accounting Practices Untested
EPA's Office of the Comptroller issued interim policies and procedures on accounting for
IT activities through Policy Announcement No. 01-10, New Information Technology
Accounting Requirements. Effective October 1, 2001, this announcement established a
37
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
standard agency-wide method of tracking IT costs using the site/project field in the
existing accounting code structure. The announcement also defined three life cycle
categories, as well as IT activities, goods and services, and established processes for
capitalizing the full cost of internal use software.
The majority of EPA's IT project costs are based on contractor and grant costs.
Whether the captured IT costs are accurate will depend largely on the ability of IT
Project Officers, Delivery Order Project Officers, and Contracting Officer's Technical
Representatives to accurately assemble supporting cost documents, accumulate
appropriate project life cycle costs, and input the project costs into the Agency's
accounting system by life cycle phases. Especially in the early implementation stages,
individuals may not have enough knowledge of the IT projects they manage to
consistently and accurately distinguish between the significant and major cost categories
(i.e., the preliminary design, development, and maintenance phases). Our concern is
compounded by the fact that the three system life cycle categories set forth in the Policy
Announcement are inconsistent with the phases described in EPA Directive 2100. OEI
and the Office of Chief Financial Officer are participating in an agency-wide workgroup
to revise and identify acceptable systems development approaches, resolve current
differences in life cycle phases, and develop common definitions across various
management programs (e.g., accounting, systems development, Enterprise Architecture,
and CPIC process).
Until the new practice is audited, we cannot be certain that actual Agency practices will
conform with the Policy Announcement, or that successful implementation of the policy
will result in effective tracking of FT costs for capitalizing the full costs of internal use
software.
Ability to Assess and Manage IT Projects Impaired
The absence of a project cost accounting system impaired IT managers' ability to
efficiently and reliably estimate, manage, and report IT project costs. For example,
system managers could not perform reliable cost-benefit analyses of technical alternatives,
which is useful for developing a sound system/project management plan. Likewise, IT
managers could not maximize the value of or perform risk-adjusted Return on Investment
analyses. Furthermore, neither the CIO nor Chief Financial Officer could reliably verify
or validate the accuracy or completeness of IT expenses reported by program offices and
regions. Therefore, IT investment amounts previously reported via OMB Exhibits 53 and
300b were at significant risk of being incomplete, inaccurate, or inconsistent with prior
year disclosures.
EPA Asserts System Complies with Standards
Despite previous OIG report recommendations to implement a managerial cost
accounting system, the Office of the Chief Financial Officer had maintained that EPA's
financial management system met Federal accounting standards. While Statement of
38
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Federal Financial Accounting Standard No. 10 prompted the Agency to create a
methodology to capture IT costs for "internal use" software capitalization purposes,
EPA's current interim cost accounting and related management systems still cannot
provide managers with enough basic cost information to accomplish objectives associated
with planning, decision making, control, and reporting for their respective IRM program
activities. However, on September 24, 2002, the Office of Chief Financial Officer
submitted an action plan for Expanding Cost Information at EPA. We will continue to
monitor the Agency's achievements as they work with program offices to promote the
use of cost information in managing for results.
Recommendations
Implementing appropriate definitions and controls will require the combined efforts of
several EPA program offices. We recommend the Chief Information Officer, Chief
Financial Officer, and Director for Acquisition Management work together to:
6-1. Institutionalize consistent definitions of systems life cycle stages and IT costs in
Agency policy to be used for contracting, accounting, IT systems, project
management, and the capital planning investment control process.
We recommend the CIO and Chief Financial Officer work together to:
6-2. Institutionalize in Agency policy consistent systems life cycle and IT costs
definitions for revising EPA Directive 2100, and the interim IT activities policy
guidance.
We recommend the Chief Financial Officer lead an effort to:
6-3. Complete a needs and feasibility assessment of alternatives to determine what
types of project cost information and supporting documentation are needed for
the capital planning investment control process and managing FT projects.
Agency Response
Responding for EPA's Chief Financial Officer, the Comptroller agreed in general with our
recommendations and pointed out that Policy Announcement 01-10, effective October 1,
2001, implemented IT project cost accounting, which is a new way of conducting
business for EPA. Both the Comptroller and the CIO did not agree with a proposed
recommendation to amend all current system development contracts to identify system
development costs by Agency system development life cycle phase. The Comptroller
stated that the policy already requires Project Officers, Delivery Order Project Officers,
and Contracting Officer's Technical Representatives to code project costs for projects
and systems under their control.
39
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
OIG Evaluation
Despite Agency assurances, we still have concerns about whether accurate cost
information will be available to permit Project Officers, Delivery Order Project Officers,
and Contracting Officer's Technical Representatives to accurately code costs for projects
and systems. As the Comptroller pointed out, this is a new process that only was
established at the end of our field work. As a result, no information was available to
complete a detailed evaluation of operational cost accumulation controls. We have
dropped our prior recommendation to amend requirements for existing software
development contracts until the fiscal 2002 financial statement audit evaluates the
adequacy of this new cost accounting process for accumulating software development
costs by project.
40
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 1
Details on Scope and Methodology
We performed our audit in accordance with Government Auditing Standards, as issued
by the Comptroller General of the United States. The audit included tests of the program
records and other necessary auditing procedures. We began preliminary research on
January 16, 2001, and an in-depth review on August 21, 2001. We issued a draft report
in April 26, 2002. We conducted this audit at EPA Headquarters in Washington, DC.
At the time of our audit, our scope was limited because the Agency could not provide a
final work plan for the Information Integration Program project, also known as the
National Environmental Information Exchange Network project. Also, we could not
substantiate how the Working Capital Fund process integrates with the IT investment
process (see Scope Limitations section below).
To accomplish the audit objectives, we attended hearings on July 11, 2001, on Senate
Bill 803, and documented Testimony before the Senate Governmental Affairs Committee.
This bill was to address the need for a Federal CIO to manage IT investments under the
Clinger-Cohen Act. We compiled a list of public laws related to IT acquisition and
management that affected implementation of the Clinger-Cohen Act. This included the
Electronic Government Act, the Paperwork Reduction Act, and the Federal Acquisition
Regulation. We reviewed Congressional Reports and noted the problems Federal
agencies were experiencing implementing the Clinger-Cohen Act. We reviewed OMB
Circulars pertaining to implementation of the Act, and feedback provided by OMB to
EPA concerning Agency IT budget submissions.
We reviewed the Agency's Enterprise Architecture dated March 29, 2001, and
summarized the Federal requirements for developing Enterprise Architecture documents.
We researched and reviewed documents issued by the Federal CIO Council relating to
the implementation of the Clinger-Cohen Act. EPA has actively participated in the
Council's survey and study projects.
We reviewed EPA IRM policies related to implementation of the Clinger-Cohen Act We
met with Agency personnel knowledgeable of and responsible for writing IRM policies.
At the time of our review, EPA had established an Agency work group to address the
needed revision of System Development Life Cycle polices to support the requirements
of the Clinger-Cohen Act.
We reviewed Agency delegations dealing with implementation of the Clinger-Cohen Act
to ascertain whether appropriate authority had been delegated to the CIO by the
41
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Administrator, and whether the CIO had delegated appropriate authority to program
officials. We consulted with the OIG Counsel on this matter.
To gather information on the implementation of the Clinger-Cohen Act in other Federal
agencies and determine potential benefits that could be implemented by EPA, we
interviewed personnel at three other agencies: Treasury, Housing and Urban
Development, and Agriculture. For example, I-TIPS was a tool used by management at
these agencies.
We interviewed personnel responsible for implementing and managing EPA's CPIC
process, including the OEI Director; and personnel in the Office of Technology
Operations and Planning and its Information Technology Policy and Planning Division.
Division personnel interviewed included the Chief of the IT Strategic Planning Branch and
CPIC Team Leader. We also attended various OEI meetings related to the CPIC
process.
We reviewed EPA's IT budget submissions for fiscal years 2002 and 2003, including
various budget proposals. Our review included a comparison of the proposals for the 2
years to determine any proposed changes, the differences in budgeted and actual costs,
and the cost variances. We also noted whether the proposal indicated a Cost Benefit
Analysis and a Security Plan had been completed.
We examined various documents provided by OEI, including budget call letters,
instructions for preparers, the organization of the peer review, instructional material for
reviewers, proposal evaluation criteria, peer review scoring, ranking and comments,
notes, agendas, and actions of the Investment Subcommittee. We reviewed the agenda,
notes, and actions of the Quality Information Council.
For three IT investment projects, we reviewed the adequacy of information and
documentation in support of their Clinger-Cohen Act submission documents for fiscal
2002. This included an evaluation of the related project management controls and a
comparison of the information provided for fiscal 2003. We used control questionnaires
and follow-up interviews with IT project managers to ascertain information about project
management practices, as well as Agency infrastructure and architecture projects.
Scope Limitations
We could not substantiate how internal controls for EPA's Working Capital Fund
process integrate with both the IT investment process and the Enterprise Architecture,
despite repeated efforts to obtain relevant policy or procedural information from OEI
officials. The Working Capital Fund is used to fund various aspects of IT projects. We
were advised that responsibility for the Fund recently shifted from OEI to the Office of
the Chief Financial Officer. The Working Capital Fund concept is described in the
42
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
narrative for the Agency's IT Architecture Roadmap, but the Roadmap does not
elaborate on the Fund's relationship to the Agency's IT investment process.
We attempted to audit two infrastructure project proposals: the National Centralized
Computing and Information Processing Initiative and the proposal for the Scalable
Computing and Information Infrastructure. The Agency could not provide any support
for the proposals, including support for why $13 million in work included in initial
proposals was no longer in the total costs of a subsequent proposal. Consequently, we
could not audit what happened with the $13 million. Following our inquiries, the Scalable
Computing and Information Infrastructure proposal was withdrawn from the investment
review process and included as part of the National Centralized Computing proposal.
Other projects also showed significant variability in Working Capital Fund expenditures,
and we could not verify the nature of these variabilities.
Congressional Concern
One of the reasons for our conducting this review was the concern expressed by
Congress in a report from the U.S. Senate's Governmental Affairs Committee,
Investigative Report of Senator Fred Thompson on Federal Agency Compliance
with the Clinger-Cohen Act, dated October 20, 2000. The report indicated that
Federal agencies had not taken adequate actions to implement the Act, and noted that
EPA did not produce evidence of any specific mission-related review of assessments
based on programmatic or operational goals. EPA acknowledged shortcomings in its IT
investment proposals, such as milestones being too general, projects being planned and
managed in a stovepipe fashion, priorities not being established agency-wide, and the
IRM strategic plan not being updated since the implementation of the Government
Performance and Results Act. Further, when the Committee asked for a status report on
EPA's top 10 IT investment projects, EPA could not provide any information on the
status of 4 of those 10 projects. The Committee made numerous recommendations to
executive departments (including EPA) for making improvements.
Prior Audit Coverage
In OIG Report No. 2001 -P-00013, Water Enforcement: State Enforcement of
Clean Water Act Dischargers Can Be More Effective, dated August 14, 2001, we
reported that although the modernized Permit Compliance System was estimated to cost
more than $10 million in life cycle costs, the required system charter and system
management plan decision papers had not been prepared or approved by appropriate
levels of management.
In OIG Report No. 001000239, Financial Management: EPA 's Fiscal 1998 Working
Capital Fund Financial Statements, dated March 29, 2000, we found internal control
weaknesses that would impact the overall management of Working Capital Fund
operations, and resulted in managers not having accurate or timely financial information
43
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
on the Fund's operations. This Fund provides EPA with computer and
telecommunication services on a cost-reimbursable basis.
In OIG Report No. E\1^MF3-15-0072-5W0240, Management of Application
Software Maintenance at EPA, dated March 31, 1995, we noted that while EPA was
creating the Working Capital Fund to more cost effectively administer services, it was still
questionable whether EPA could separate application software maintenance activity from
operations activity. EPA did not develop, review, and update software maintenance
costs by individual systems throughout their life cycles, which would prevent informed
budget decisions from being made.
In OIG Report No. El SKG3-15-0098-4400038, Special Review of EPA 's
Information Systems Program, dated March 24, 1994, we noted that management did
not treat information as a strategic resource nor IRM as a core function and valuable tool.
EPA did not have an information data architecture, data standards, or administrative
structure to facilitate data sharing Agency-wide, and data quality problems existed.
Also, a National Academy of Public Administration report, Transforming
Environmental Protection for the 21st Century, dated November 2000, noted the
nation needs authoritative information about environmental conditions, and discussed
various steps being taken by EPA to do so. The report also emphasized that OEI had
not begun to draft a strategic plan to guide its activities, and had no direct authority over
the budget or staff that support EPA's systems.
44
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 2
Office of Environmental Information's
Response to Draft Audit Report
July 2, 2002
MEMORANDUM
SUBJECT: Response to the Draft Report: EPA 's Management of Information Technology
Resources Under the Clinger-Cohen Act, Audit Number 2001-0591
FROM: Kimberly T. Nelson /s/ Rick Otis for
Assistant Administrator
and Chief Information Officer
TO: NikkiTinsley
Inspector General
This memorandum provides a response to the Office of Inspector General (OIG) findings outlined
in the Draft Report: EPA 's Management of Information Technology Resources Under the Clinger-
Cohen Act, Audit Number 2001-0591, dated April 26, 2002. Overall, the Office of Environmental
Information agrees with your emphasis on the critical importance of an effective IT resource investment
management program that 1) delivers real benefits to the Agency's mission and 2) properly manages the
risks across our enterprise portfolio. It is my intent to aggressively address the key issues raised in the
report and I appreciate the work of your staff in providing us with this critical input to our planning and
operation of the Clinger Cohen CIO program. We will provide a complete action plan for improvements
upon receipt of the final report.
There are some findings and recommendations in the draft report that my staff finds are not totally
accurate in their characterization of the past accomplishments, current status and strategic directions of
our program. We previously provided comments correcting some items which provided the basis for this
draft report, but the report does not reflect any changes for those issues. We have also made much
progress as an Agency during and following the audit. I would appreciate your review of our attached
comments. Please adjust the final version of the report to incorporate changes to the introduction,
findings and recommendations based on this information to ensure the final report provides the most
accurate view of the program and where the Agency should focus attention and resources to help it
improve in the future.
45
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
If you have any questions regarding this response please have your staff contact Mark Day,
Director of the Office of Technology, Operations and Planning at (202)566-0300.
Attachments
cc: Mark Day, Director, Office of Technology Operations and Planning
Debra Stouffer, Chief Technology Officer
Kathy Petruccelli, Director, Office of Planning, Resources and Outreach
Mike Flynn, Deputy Director, Office of Information Analysis and Access
Brion Cook, Director, IT Policy and Planning Division
Rick Martin, Director, National Technology Services Division
Kevin Phelps, Associate Director, IT Policy and Planning Division
Barbara A. Chancey, Chief, IT Strategic Planning Branch
Chuck Cavanaugh, Program Lead for Investment Management
John Sullivan, Chief Architect
John Moses, Office of Information Collection
Joe Dillon, Comptroller
Juliette McNeil, Director, Financial Management Division
John Gherardini, OAM
Tom McEntegart, OAM
Ed Lillis, OA
Edward Cottrill, OW
Tony Jover, OSWER
Michael Mundel, OECA
Jeffrey Worthington, OEI Audit Coordinator
Brigid Rapp, OCFO Audit Coordinator
Christa Eckel, OAM Audit Coordinator
Greg Marion, OECA Audit Coordinator
Judy Hecht, OW Audit Coordinator
Johnsie Webster, OSWER Audit Coordinator
Patricia H. Hill, OIG
James Rothwell, OIG
46
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Draft Report: EPA's Management of Information Technology Resources Under
the Clinger-Cohen Act, Audit Number 2001-0591
Executive Summary
While we agree with the overall goal of the report, in many cases findings do not adequately reflect status and
accomplishments, so recommendations are not as helpful as they might be. We request adjustments to findings and
recommendations to focus attention more effectively on where additional effort and resources would benefit the Agency.
The following comments address statements in the Executive Summary "Results in Brief which contains content outlined
from each chapter. Additional specific comments on findings and recommendations are identified separately in relation to
the respective chapters.
"Since established in 1998, EPA's CIO has not taken adequate actions to implement and institutionalize the Agency-wide
authority and responsibilities for IT capital investments"
EPA CIO's have made major advancements in ensuring Agency-wide compliance with Clinger-Cohen
responsibilities. EPA established the Quality Information Council (QIC), chaired by the CIO and comprised of
Agency senior resource management officials. The QIC formally approves IT investment decisions, and has done
so since Clinger-Cohen has been in place. Under CIO's leadership, EPA senior resource managers have engaged
in substantive investment reviews and direction. Their joint efforts have lead to restructuring of portfolio
components, as well as substantive change/improvement of specific proposals.
"Several key factors continue to inhibit the realization of a successful program..."
OEI has made significant advances on each of the factors specified. Specifically the CIO has taken steps to:
• establish a substantive range of new policies, procedures, and guidance on priority areas (security,
investment) and is in the process of moving forward on a new comprehensive policy framework;
• promulgate a new information strategic plan reflecting the Clinger-Cohen framework (in CIO review);
• officially establish a chief architect and elevate the Agency profile for enterprise architecture
development;
• hire a Chief Technology Officer to champion Clinger-Cohen compliance within EPA
• employed risk-based assessments for capital IT projects reflecting the evolving nature of OMB
guidance under Clinger-Cohen;
• establish new IT cost-tracking structures and requirements, and begin integrating investment and cost-
tracking.
"CIO had minimal assurance that IT investments reported to OMB would maximize their value"
CIO recommendations for IT investments reflected senior Agency decisions on strategic program direction and
value, based on then applicable Agency needs and available OMB guidance. Further, OEI continues to strengthen
the investment review process to maximize value, including regular investment reviews of all OEI investments to
review cost, schedule, and performance.
47
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Executive Summary
Recommendations
OEI/OTOP Response
Revise outdated policies to remove
unauthorized IT business practices and
add new requirements.
Suggest restating to acknowledge OEI process underway since Ql/02 to:
1. Identify, from a best practices perspective, what EPA's IT policy
collection should be (recommendations to be forwarded for CIO review
in August, 2002);
2. Catalog EPA's current IT policy collection (completion in August,
2002);
3. Identify the gaps between the "should be" and "current" states i.e.,
those IT policies needing to be created, updated, or canceled
(September 2002);
4. Develop a multi-year plan for how to address the gaps and bring
EPA's IT policy collection to the "should be" state referencing
Enterprise Architecture, CPIC, and IT acquisition processes (November,
2002).
Finalize the IRM Strategic Plan.
Agreed and underway. A "Strategic Information Plan" document is in CIO
review. The goals and direction put forth in this document are being
incorporated as drivers in the architecture development.
Formally establish a Chief Architect
position with sufficient authority.
Please correct. On February 22, 2002, the CIO established the Enterprise
Architecture Program and named John Sullivan as Chief Architect for EPA.
Implement an automated project
management system (I-TIPS).
Please restate: "Continue efforts to implement I-TIPS". OEI is implementing I-
TIPS successfully and will be using it to generate OMB reports this September
for budget year 2004. EPA completed a security vulnerability assessment and
developed risk mitigation plans prior to production as required by OMB, and is
now moving forward agressively.
Implement monitoring and evaluation
processes for IT investments.
Please provide greater specificity. EPA senior management and the CIO do
monitor and evaluate IT investments, reviewing all OEI investments for cost,
schedule, and risk. Further, the CIO is taking steps to integrate investment,
enterprise architecture, system life-cycle and fiduciary management processes in
partnership with OCFO. A general statement expressing support for these efforts
would be useful.
Postpone funding for IT projects that
have been identified as "materially
deficient"
The CIO and the QIC review investments prior to funding. Funding has never
been recommended for an investment determined to be "materially deficient."
48
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 2 - CIO Needs to Fully Implement Clinger-Cohen Act Requirements.
Findings
Response
2.1- Five years after implementation of
the Act, EPA's CIO still had not
established an adequate structure with
the policies and guidance needed to
sufficiently implement the Act.
EPA through actions by the Administrator and the CIO has taken steps to
implement critical Clinger-Cohen functions, and to direct IT resources in a
manner that will deliver increasing value to our program mission. CIO leadership
has been highly visible in enterprise architecture, investment management,
critical policy, and workforce development. Please amend this finding to
highlight the specific areas where the CIO and Agency leadership should direct
additional attention and resources.
2.2 - Overall, EPA's program managers
are treating the Clinger-Cohen Act
requirements as little more than a paper
exercise to satisfy the annual OMB
budget call.
Inaccurate. There is evidence that program offices do take the CCA seriously.
Agency managers at multiple levels have actively participated in investment
reviews. Management attention is reflected in: consolidation and elimination of
duplicate projects; the number of program offices seeking OEI's consultation on
preparing proposals; more refined reporting of budget numbers; linking IT
investments to GPRA goals and agency priorities. It would be helpful, if you
could expand the recommendation to identify the specific manner in which
program managers should be involved beyond the roles that they currently fulfill
(proposal preparation, approval, participation in Agency-wide portfolio
development).
2.3 - Numerous examples demonstrated
the use of inconsistent criteria and a
general lack of objective, quantitative
investment criteria (e.g., cost-benefit
analysis)
For the past five years, criteria has been based on the OMB's eight Raines Rules,
plus additional Agency policy and programmatic criteria that was approved by
the QIC's Information Investment Subcommittee (IIS), CIO, CFO, and the QIC, as
such was both consistent and objective. This year, we plan to revisit selection
criteria and approved revised criteria (including applying weights) through the
QIC.
2.4 - EPA has not formally appointed a
Chief Architect to oversee the
development and execution of its
Enterprise Architecture Plan.
Inaccurate, please remove. On February 22, 2002, the CIO has appointed a Chief
Architect for EPA. The Enterprise Architecture baseline, target and sequencing
approach is scheduled to be delivered to OMB on October 15, 2002.
2.5 - The fiscal 2002 budget did not
identify an architecture project.
Inaccurate, please remove. For the fiscal 2002 budget, the architecture project
was included as a component of integration proposals and for F Y02 it was
reported separately on the Exhibit 53 - Section 3.- Architecture.
2.6 - In 2001, EPA purchased a SLA to
use the off-the-shelf software I-TIPS....
However, when requested, EPA could
not provide any evidence to support
that they were assigning resources or
providing milestones for implementing
the software.
Please restate. The Investment Management Team has assigned resources to I-
TIPS implementation, developed milestones for production, proceeded with
implementation, and will be using I-TIPS to generate automated OMB reports for
this investment cycle. Further, I-TIPS will be expanded agency-wide in 2003.
49
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 2 - CIO Needs to Fully Implement Clinger-Cohen Act Requirements.
Findings
Response
2.7 - In addition, the following effects
are likely to occur: 1) IT investments
will no be driven by business priorities
and mission goals; 2) Stovepipe
systems will continue to operate; 3)
EPA will continue to invest in duplicate
IT system; 4) IT investments will not
take advantage of technology
advances and reduced costs; 5)
reporting processes will not be made
efficient for states and private industry;
6) application systems will not comply
with environmental data and
interoperability standards; and 7)
increased public access and security
requirements will not be met.
Please restate to acknowledge the following:
All IT investments in the CPIC process are linked to the Agency's strategic
goals. Significant reductions in stovepipe systems have been made through
consolidation and / or modernization to align these systems to the architecture.
Duplicate systems have been identified through the CPIC process by the
technical and executive management review. Proposals (e.g., Records and
Document Management, and GEO and GIS) were combined last fiscal year to
reduce redundancies and maximize efficiencies. For the past four years, data
standards questions have been required, evaluation criteria has been
established, and a data standards team has reviewed proposals to ensure that
programs are complying with data standard requirements.
Over the next couple years, Central Data Exchange (CDX) will be implemented.
As CDX grows and gains wider acceptance, it will reduce the reporting burden
on the states and private industry. Also, as the National Environmental
Information Network is being constructed with input from the states and
industry. The new network will greatly enhance the reporting and information
exchange between the states, industry, tribes and the agency.
2.8 - During recent years, the CIO
should have used an IT investment
control process to solve key Agency-
wide problems such as integration of
environmental data, electronic
reporting,, duplicate systems,
Geospatial Information, and data
management.
Please restate to acknowledge those very issues targeted and addressed during
the CPIC process. For example, the GEO investment was stopped from receiving
operating plan funds in F Y01 due to CPIC process findings. Also, duplicate
systems were identified and requested to coordinate development strategies and
present before the IIS.
Recommendations
Response
2.1 - Assign sufficient resources and
expertise to ensure timely and effective
implementation of report
recommendations; and use objective,
risk-based criteria to decide whether
proposed and ongoing IT investments
will help resolve key Agency-wide
problems and advance EPA's IRM
vision.
Agreed. Request for an increase in resources (extramural, FTE) has been
submitted for the F Y03 and F Y04 budgeting years.
From the inception of this process under Clinger-Cohen, management reviews
have been risk-based. The initial method referenced the "Raines rules" following
the approach which was then applicable on a government-wide basis.
In the FY 02 CPIC process, the technical review team is using objective, risk-
based criteria by identifying weaknesses and working with program offices in
producing strong business cases, cost/benefit analysis, results-oriented
performance measures, cost and schedules, and presenting proposal evaluation
results to the QIC/Information Investment Subcommittee in a portfolio
management enterprise perspective. Increased resources would enable more
frequent investment reviews.
50
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 2 - CIO Needs to Fully Implement Clinger-Cohen Act Requirements.
Findings
Response
2.2 - Revise EPA Directive 2100 and
related guidance to remove outdated
and unauthorized IT business
practices. Incorporate appropriate
policies and procedures for the
Enterprise Architecture, CPIC process,
and IT acquisitions addressed in the
Clinger-Cohen Act, OMB guidance,
and EPA Delegation 1-84.
Please restate to acknowledge that OEI has had a process underway since Ql/02,
anticipated for completion in Q2/03. The process is to:
1. Identify, from a best practices perspective, what EPA's IT policy
collection should be (recommendations to be forwarded for CIO review
in August, 2002);
2. Catalog EPA's current IT policy collection (completion in August,
2002);
3. Identify the gaps between the "should be" and "current" states i.e.,
those IT policies needing to be created, updated, or canceled
(September 2002);
4. Develop a multi-year plan for how to address the gaps and bring
EPA's IT policy collection to the "should be" state referencing
Enterprise Architecture, CPIC, and IT acquisition processes (November,
2002).
2.3 - Work with the Director for
Acquisition Management to (a) direct
contracting officers and other
procurement personnel to only accept
procurement requests with a formal CIO
approval or officially re-delegated
procurement authority; and (b)
establish interim delegations, policies
and procedures for IT procurement,
until formal redelegations are revised
and implemented.
Please restate to acknowledge that OEI (and previously OIRM) has historically
worked with OARM to ensure appropriate review/concurrence for IT
acquisitions. The CIO has initiated the establishment of delegations under the
Clinger-Cohen framework to ensure all IT procurements have formal management
official approval (either CIO or someone with formal authority delegated by the
CIO) before consideration by procurement personnel.
Chapter 3 - Weaknesses in CPIC Process Place EPA's FT Investments at Risk
Findings
Response
3.1- However, EPA's CPIC process
does not monitor each project's
execution during a Control phase nor
evaluate the adequacy of completed
projects in an Evaluation phase, as
recommended in Figure 2.
Please restate. EPA's CPIC process has incorporated the Control phase since its
inception. The evaluation phase is currently being implemented. Refer to
Report on Management Options for Implementing the Evaluation Phase of IT
Capital Planning and Control, dated January 7, 2001 and white paper entitled
Implementing the Select/Control/Evaluate Phases of Review, dated April 12,
2001. To be helpful, please identify, in the final report, specific aspects of control
/ evaluate phases which OIG believes require further attention.
51
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 3 - Weaknesses in CPIC Process Place EPA's IT Investments at Risk
Findings
Response
3.2 - The peer review risk assessment
was the only substantive process used
to control IT investments, and we
found no evidence of a quality
assurance process to ensure
investment proposals were accurate.
This finding is inaccurate as stated. The CPIC process has four levels of
qualitative reviews: 1) staff level - a thorough review of proposal format and
content is conducted; 2) technical peer review - evaluation criteria based on the
Raines Rules is applied and proposals are evaluated based on technical merit,
then grouped and ranked; and 3) IIS - executive management level review to
address funding and policy issues, grouped and ranked red, yellow or green.
4)QIC review.
3.3 - Agency management planning and
budgeting recommendations for fiscal
2002 were based on IIS opinion, rather
than objective peer review risk
evaluations.
This finding is inaccurate as stated. The F Y 2002 recommendations were based
on the technical peer review analysis and the discussions and deliberations of
the IIS. The IIS depends heavily on the technical review results.
3.4 - Table: EPA Major Investment
Proposals, Key Project Risk Factors
(Fiscal 2002)
The percentages in these findings do not match reports and OMB statements
given by EPA and OMB. It would be helpful if the OIG presents the document
which states "OMB Assessment" amounts.
Please also include the statement from OMB "We think a great deal of BCA has
been performed on the majority of the portfolio."
3.5 - Nevertheless, the IIS
recommended to the Quality &
Information Council and the CIO that all
48 projects be recommended for
funding in the fiscal 2002 budget
submission.
This finding is misleading and should be restated or removed. Projects were
recommended for funding only after substantial corrective actions were taken to
make the business case, and a fourth review of the project proposal was
conducted. Five projects were required to address the IIS to explain and defend
their business cases.
3.6 - Major projects were found to have
material deficiencies by the peer review
process, yet the IIS recommended to
fund these projects in fiscal 2002. In
fiscal 2003, the peer review process
once again stated these projects
contained significant weaknesses, but
IIS still recommended them for funding.
This is inaccurate. In 2002, the IIS red-lighted five projects, initially not flagged
by the technical peer review team. These projects were required to go before the
IIS for further scrutiny and extensive review of the project's business case
occurred.
The finding should also state that for 2003, following extensive project/portfolio
revisions per senior management direction, OMB subsequently found
deficiencies to the business case for only 2 of 48 proposals, which they then
accepted after minor revisions.
3.8 - Paragraph on CIO Needs to
Formalize and Institutionalize a CPIC
Process - The CIO has yet to establish
policies and guidance, and implement
key Clinger-Cohen Act requirements by
formalizing the CPIC process in Agency
Directive 2100.
The process is formalized, so please restate. For the past five years EPA has
been conducting a Capital Planning and Investment Control process (CPIC),
which includes a Select and Control phase, appropriate guidance, training,
evaluation criteria based from the Raines Rules, a formal technical review process
and executive management review to evaluate proposals.
In June 2002, a final CPIC policy was issued, formalizing the process in Agency
Directives.
52
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 3 - Weaknesses in CPIC Process Place EPA's IT Investments at Risk
Findings
3.9 - Implementing the Federally
sponsored I-TIPS software, an
automated investment control and
reporting system, would provide EPA
with a valuable tool for monitoring and
managing its IT investment portfolio.
While EPA has been using a peer
review process to evaluate risks,
management has not employed a
structured CPIC process to maximize
the value of investments and manage
the risks of IT acquisition projects.
Response
Please restate. There are two separate issues - implementation of I-TIPS and a
structured CPIC process.
OEI is implementing I-TIPS and will be using it to generate OMB reports for this
cycle. OIG should also note that EPA's schedule for I-TIPS implementation
reflects the fact that I-TIPS does not conform to the Agency's existing technical
architecture and employs web-based functions with security vulnerabilities
which required careful risk assessment and mitigation plans, prior to production.
OEI has developed methods to address vulnerabilities and is moving forward.
From a process perspective, EPA has consistently followed a highly structured
approach involving project and program managers at key decision points. The
process continues to evolve and next year will integrate enterprise architecture
with investment to provide further structure to the process of establishing
management priorities and decision making.
If OIG believes additional structure is required, specific recommendations would
be helpful.
Chapter 3 - Weaknesses in CPIC Process Place EPA's IT Investments at Risk
Recommendations
3.2 - Formally establish objective, risk-
based criteria for the IIS to use in
selecting and funding all IT
investments (e.g. Chart of EPA System
Development Risk Factors). Based on
the criteria, management should not
fund proposals or projects that classify
as high risks.
Response
Please restate. EPA has in fact employed specific evaluation criteria for review of
CPIC proposals for past CPIC cycles. The evaluation criteria was released as part
the annual Exhibit 300 data call so that preparers and reviewers were aware of the
criteria each proposal would be evaluated against.
Also, the IIS will be given a technical peer review summary of each of the
proposals prepared in accordance with the evaluation criteria and with that
information will be following the OMB scoring guidelines provided in this year's
A-l 1 guidance. As part of the Strategic Direction for Investment Management,
the IIS plans to identify business and architectural criteria for investments,
Ql/03.
With this established, the Agency will be able to make even more thorough,
objective, risk-based evaluations of all proposals than in the past. Additional
specific suggestions from OIG on how to further enhance criteria would be
welcome.
53
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 3 - Weaknesses in CPIC Process Place EPA's IT Investments at Risk
Recommendations
Response
3.3 - Postpone funding for current IT
projects identified by the Peer Review
process as materially deficient or high
risk for 2 consecutive years, until
critical deficiencies are resolved and the
resolution steps adequately
documented.
Agreed. However, no "materially deficient" project has been recommended for
investment by the CIO. The Agency's Information Investment Subcommittee is
responsible for recommending funding to the CIO regarding major IT
investments. Those projects identified in the Technical Peer Review process as
deficient are afforded the opportunity to make revisions to their proposals prior
to the Subcommittee's review and, time permitting prior to the QIC's review.
Forty-eight proposals submitted for funding last year to OMB were approved.
For example, in FY01, the IIS advised the Acting CIO to charge a task force to
develop a strategic direction and architecture for electronic records, dockets, and
document management applications. The IIS made a recommendation to
suspend funding for seven systems. The Acting CIO followed through on that
recommendation (memo from Margaret Schneider, dated October 12, 2001,
Management Task Force for Agency Document Management Systems,
"...suspend spending for design and development work for all new and existing
document management systems.")
3.4 - Direct the Information Investment
Subcommittee to monitor the execution
of IT projects during the fiscal year (at
least quarterly) to identify emerging
cost or schedule problems and initiate
corrective actions.
Agreed. As part of ITPPD's Investment Management strategic planning efforts,
and in conjunction with the use of I-TIPS, it is OKI's vision to evolve the
Agency's capital planning process to do a continues update and review process
in the next two years. This continues process will involve Program Offices
updating their business cases as their systems develop (i.e. moving from
different life cycle or CPIC phases). Rather than relying on annual data calls for
updates, this will allow the Program Office management, the Subcommittee, the
QIC and the CIO's office access to the most current information possible, thus
providing them the ability to address cost or performance issues as they are
identified, not just once a year.
3.5 - Initiate a formal process with
written evaluations of ongoing,
completed, and terminated information
technology projects to evaluate
whether the projects or systems are
successfully delivering promised
benefits at an acceptable cost.
Agreed. As ITPPD prepares its Strategic Planning for IT Investment
Management, one of the areas being examined is the formalization of processes
and evaluations in all phases of the CPIC process, including the possible
inclusion of Pre-Select and Steady State phases to provide management with on-
going evaluation monitoring.
3.6 - Implement an automated project
management system (e.g., I-TIPS) to
provide timely, reliable information for
investment decisions.
This recommendation should clearly define the difference between a portfolio
management system and a project tracking system - I-TIPS is a portfolio
management system, not a project management system.
ITPPD is currently piloting the use of I-TIPS in EPA. ITPPD plans to populate
Exhibit 300 data in I-TIPS and submit electrically to OMB (09/02). Additionally,
ITPPD plans to develop an Agency-wide deployment strategy for I-TIPS in
Ql/03 forFY 03-04 implementation.
54
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Findings
Response
4.1 - During 2001, EPA completed many
actions towards establishing a baseline
enterprise architecture for IT planning
purposes. In April, EPA provided
OMB with documentation of EPA's
first Enterprise Architecture, dated
March 29, 2001. However, by October,
neither the Agency's IT Contracting
Officer nor the Procurement Office had
been provided a copy of the proposed
Enterprise Architecture.
This is incorrect. The Agency's Enterprise Architecture is posted on the EPA
Intranet and program offices were notified of its availability. The Office of
Administration and Resources Management (OARM) was notified that the
architecture had been published.
4.2 - Moreover, the document was not
provided timely to the EPA program
offices for use in developing IT
investment proposals for the fiscal 2003
budget submission.
Please restate. This finding does not accurately reflect that appropriate guidance
was provided from the EA Team to proposal preparers on developing their 2003 /
2004 investments. The EA Team also worked one-on-one with program offices
requesting assistance. The current enterprise architecture being developed will
contain a baseline, target and sequencing approach, which will assist preparer in
the 2005 exercise.
4.3 - Also, OMB reviewed the
Agency's fiscal 2001 IT Investment
Portfolio and noted that they could not
match the projects in the proposed
Enterprise Architecture to the portfolio.
In August 2001, OEI established a
workgroup to identify and verify EPA's
business processes for the Enterprise
Architecture baseline. The work
group's efforts occurred after
completion of our field work; as such,
we do not know fully what they have
accomplished.
Please acknowledge that the workgroup has updated the business processes and
these processes will be aligned with the new OMB Business Reference Model.
4.4 - EPA's outdated IRM Strategic
Plan has contributed to the delay in
implementing the Enterprise
Architecture concept. In May 2001,
EPA established an agency-wide work
group to update the IRM Strategic
Plan. The work group provided the
draft plan to OEI's Quality Information
Council, but it has yet to be finalized.
Please restate to acknowledge that a "Strategic Information Plan" document is in
CIO review. The goals and direction put forth in this document are being
incorporated as drivers in the target architecture development.
55
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Findings
Response
4.5 - As of the end of field work, EPA
had yet to fully baseline and validate
the Agency's business processes
essential for establishing a portfolio for
future IT investments. EPA's draft
Enterprise Architecture document
included very high-level business
processes; however, these processes
had yet to be validated by the
responsible program offices. We were
informed that some of these business
processes have been revised, but were
unable to substantiate whether the
applicable program offices formally
endorsed the work group's
conclusions.
Agreed, however we have made progress, and plan to acquire QIC approval of
the EA. Formal validation of baseline program components by the CIO and
senior program managers, via the QIC, is occurring this year per the management
plan for build-out of the enterprise architecture.
4.6 - Therefore, the physical and
personnel security requirements of
these servers need to be added into the
baseline security architecture.
Inaccurate. The Security Architecture does address the physical, facility and
personnel security issues.
4.7 - As depicted in Figure 3, the
Enterprise Architecture conceptual
framework should consist of five
components. As such, the Enterprise
Architecture should define mission-
critical data needs to properly support
the IT investment process. However,
EPA's current Enterprise Architecture
does not adequately address (1) EPA's
existing data standards and related
metadata baseline information, and (2)
other critical data used by stakeholders
and programs business processes.
This finding is inaccurate. Data standards and critical data are both integral
aspects of EPA's enterprise architecture. The model specifically references data
standards, and will address program-specific data needs across several
dimensions.
4.8 - We had recommended that EPA
support its data standards program by
using the Environmental Data Registry
as a central repository for publishing
and recording data standards. EPA has
yet to do so, and the draft Enterprise
Architecture does not adequately
describe the registry as a critical
component of its target architecture.
This finding is outdated and should be removed. The Enterprise Architecture
does support data standards and the EDR. The document being prepared for
OMB will clearly outline this architectural component.
56
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Findings
Response
4.9 - Complete Inventory of Systems
Needed for Enterprise Architecture ...
we found that the Enterprise
Architecture document does not
include sufficient information on
Agency application interfaces. The
document states the CIO plans to
gather and document this information
as part of the Agency's ongoing
application inventory initiative,
including documentation regarding
major interfaces with applications
outside of the Agency.
The Enterprise Architecture will gather more information this year on internal
system interfaces and partner interfaces within the framework of the National
Environmental Information Network (NEIN). The target architecture and
sequencing plan will also take into account the impact of external federal Agency
interfaces and E-gov directions.
4.10- Our review showed that the
Enterprise Architecture document did
not explicitly identify minimum
response times for key transaction-
based systems and for business
application systems on the Agency's
wide area network.
This finding does not provide relevant or helpful direction. Normally, this level
of detail is not in an Enterprise Architecture document. Transaction response
requirements for critical data streams will be considered as a factor in the
development of the technical architecture, which must be scaled and engineered
to support such needs.
4.11 - We believe the VPN concept is
needed today to help the Agency
comply with existing Federal
telecommuting statutory requirements
and to satisfy current business needs.
OEI agrees with the importance of secure external communications. This year
OEI is taking the critical steps to establish secure external partner levels of
access with implementation planned to start next year and full operations to be
available on an enterprise basis in 2004 (pending continued availability of
resources).
4.12 - Also, EPA needs to define the
role and authority of its Chief Architect
for IRM. The role of this Chief
Architect is to oversee development
and coordination of the Enterprise
Architecture with other planning
elements that should materially shape
and drive the IT planning structure.
The CIO named an individual to this
role in February 2002 (via electronic
mail), but there has been no formal
definition of the position's scope and
responsibilities, nor any official
delegation of authority.
This recommendation is outdated and should be refined. On February 22, 2002,
the CIO via electronic email, established the Enterprise Architecture Program and
named John Sullivan as Chief Architect for EPA. If additional authority is
needed in the view of OIG, specific deficiencies should be noted.
57
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Findings
Response
4.13 - To EPA's credit, management
established a central Enterprise
Architecture workgroup in August
2001. However, no permanent central
organization has been established or
assigned resources to coordinate,
develop, and maintain the Enterprise
Architecture. Agency-wide Enterprise
Architecture components need to be
addressed and maintained for the
following functional areas: the
identification of EPA's major and
significant systems; defining the
security architecture; validating the
business processes with program
offices; developing the Middleware
architecture and defining baseline
telecommunication requirements;
defining Working Capital Fund capital
investments; and approving individual
IT project management plans for major
projects or systems.
This finding should be rephrased. The functional areas identified in the
recommendation are all included within the strategic activities underway this year
and planned for next year. Please restate the recommendation to acknowledge
the importance of the ongoing efforts being made to address these needs.
4.14 - In its fiscal 2003 budget
submission, EPA identified the
Information Integration Program as its
only major architectural project for
deriving and completing an enterprise
architecture. As critical as the project
is to EPA's Enterprise Architecture
development efforts, no final
management work plan has been
implemented for this project since the
draft was issued in December 2000. A
final work plan is essential to ensuring
the timely success of the individual
program, as well as the overall quality
of the Enterprise Architecture Plan and
the Agency's future technology
investments. Although EPA views the
program as key to improving the overall
integration of environmental
information, this project does not
report to the Chief Architect.
This finding is inaccurate and does not accurately reflect the continuity of
results and the connection of that project with the Enterprise Architecture
program. The products from the Information Integration Program are the basis
for the target architecture of the environmental business area. Please restate this
finding to acknowledge the intent and proposed products of the Information
Integration Program.
58
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Findings
Response
4.15- The Chief Architect provided
information that indicates EPA's
program and regional offices will be
asked to co-develop the Agency's
baseline and target elements for the
enterprise architecture. With OKI's
leadership and facilitation, the program
and regional offices will conduct their
own architectural needs analysis, and
realign their respective systems with
EPA's evolving target. We were
unable to substantiate how this will be
accomplished. The participants will
need a clear understanding of their
roles and responsibilities, as well as
their respective business processes, if
they are to play a significant role in
helping define the enterprise
architecture.
We agree that clear roles and responsibilities are essential in defining the
Enterprise Architecture. We have taken the necessary steps to ensure
participants are clearly aware of their respective roles and responsibilities. The
Chief Architect and the Enterprise Architecture team are working with program
and regional representatives at the staff level to develop requirements and
validate Agency-wide perspectives. At the same time, the Chief Architect is
preparing explicit guidance, including senior management roles, to formally
record roles and responsibilities of program and regional offices for the
architecture. This framework for Enterprise Architecture policy and practice will
be reviewed by the CIO and senior managers at a forthcoming QIC meeting in
July, per the schedule presented to the QIC on 6/26/2002.
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Recommendations
Response
4.1 - As the number one priority, we
recommend that the Chief Information
Officer formally establish:
(a) an Enterprise Architecture
program to plan, manage,
monitor, and control the
development and maintenance
of the plan.
(b) the Chief Architect position
by clearly defining the role,
responsibility and authority of
the job. The position should
ensure a system of
accountability for the overall
architectural effort. This
would include coordinating
and overseeing resources for
IRM strategic planning and
the Information Integration
Program, and reporting
directly to the CIO.
This recommendation should be rephrased to acknowledge the efforts underway
to plan, manage, monitor and control the development and implementation of the
Enterprise Architecture.
The Chief Architect, through direct and ongoing consultation with the CIO, has
been directing and coordinating the Agency' s efforts to create an architecture
and architecture program. The Chief Architect is working with the CIO and Chief
Technology Officer (CTO) to promulgate an Agency-wide framework for
managing the establishment and implementation of the Enterprise Architecture.
This framework will be a major focus for senior executive discussion and decision
at the July meeting of the QIC.
We would appreciate any subsequent OIG recommendations that focus on
additional steps required to support this effort.
59
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Recommendations
Response
4.2 - Under the leadership of the Chief
Architect, update and maintain the
Agency IRM Strategic Plan to support
EPA's Strategic Plan, its Government
Performance and Results Act
requirements, and the Enterprise
Architecture.
The Chief Architect and the Architecture Team are responsible for creating,
updating and maintaining the Agency' s architecture. As part of creating the
architecture, the Chief Architecture must coordinate and participate in the
strategic planning process, GPRA and other efforts. OEI is producing a
"Strategic Information Plan" under direction of the OEI - Office of Information
Collection (QIC). This Plan will be used as a driver for the EA development.
4.3 - Identify current major and
significant general and application
systems to establish an accurate
inventory of such systems and
integrate this information with both the
Agency's Enterprise Architecture
application component and the IT CPIC
Portfolio.
EPA agrees with this recommendation. The Enterprise Architecture will
incorporate all systems (major and significant as defined in the CPIC) and others
into the Information Resources Registry System - which will serve as the Agency
Applications Inventory. The IRRS is scheduled to be operational by the end
F Y02. A linkage between the IRRS and the EA repository is planned. All
application systems within the purview of CPIC review are included in the
baseline applications architecture.
4.4 - Develop a master project plan for
completion of all parts of the Enterprise
Architecture, including a breakdown of
the tasks and subtasks needed to
acquire, develop, and maintain the
Enterprise Architecture.
EPA agrees with this recommendation. The Enterprise Team has an overall
management plan and project plan that contains the detail tasks and subtasks to
develop the Enterprise Architecture. Additionally, the Team is in the process of
identifying a change management process for updates to the Agency's
architecture.
4.5 - Establish an information
repository, require the use of a data
registry for Agency maintained data,
map EPA's data and information
resources, and adopt life-cycle data
management principles for the
Enterprise Architecture data and
systems components.
Please restate this recommendation to reflect efforts already underway. OEI has
established an EA repository in which the Agency's business, data,
applications, and technologies are mapped and interlinked. As part of the CPIC
process, programs will be required to ensure their systems are represented in the
EA repository and applications inventory. The Enterprise Architecture Team is
coordinating efforts with the ITPPD's efforts to update the Agency's life-cycle
principles currently being developed to produce a "cook book" on systems
development that will align the Systems Lifecycle policy, the CPIC Process and
the Enterprise Architecture.
4. 6 - Use a top management
verification, validation, and approval
process to ensure program business
processes and goals are accurately
reflected and incorporated into the
Enterprise Architecture. Subsequently,
formalize the process as a discipline for
updating the Enterprise Architecture
document.
EPA agrees with this recommendation. The Enterprise Architecture is presented
to the Quality Information Council for recommendation to the CIO for approval.
An EA change management and configuration control process is being
developed to formalize the process of updating the architecture. The Chief
Architect is preparing explicit guidance, including senior management roles, to
formally record roles and responsibilities of program and regional offices for the
architecture. This framework for Enterprise Architecture policy and practice will
be reviewed by the CIO and senior managers at a forthcoming QIC meeting in
July, per the schedule presented to the QIC on 6/26/2002.
60
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 4 - EPA Needs to Organize and Integrate Planning for IT Investments
Recommendations
Response
4.7 - Coordinate the Enterprise
Architecture document with the
Agency's Office of Acquisition
Management for future IT acquisitions.
Jointly develop an approval process
that ensures the Enterprise
Architecture concept is incorporated in
IT contract activities for large and
significant IT projects.
EPA agrees with this recommendation. In addition to formal promulgation of
acquisition authority and delegations by the CIO, once the EA version 1.0 is
approved by the CIO, the EA team will work with OAM to broaden the current
contracting clauses to ensure compliance with the EA.
4.8 - Develop a Middleware
Architecture as part of the Enterprise
Architecture technology component to:
define the components that interface
among the client and server systems;
improve the overall usability of the
distributed architecture; and integrate
the information repository with the
client-server systems.
Please rephrase this recommendation. As part of the Target Architecture
(Q4/02), the data warehouse methodology and platforms will be determined. The
detailed design of the warehouse (whether it is virtual or physical) will be
contained in the Technical Reference Model, which is being developed as part of
the EA. OIG recommendations should be cautious when making specific
technical references (e.g. linking client-server systems with the repository) as the
target technical architecture is likely to move the Agency towards new models.
4.9 - Establish a comprehensive and
explicitly defined set of baseline
telecommunications requirements to
support a scalable, reliable, and secure
network infrastructure for the
Enterprise Architecture technology
component. Also, address existing
bandwidth shortages and provide for
additional network capacity to support
current business needs and take
advantage of technology advances.
OEI agrees with the importance of this recommendation and its importance for
the technical architecture. Telecommunications requirements to support a
scalable, reliable, and secure network infrastructure, bandwidth capacity, and
additional network capacity are essential components of the Technology
Architecture Segment. OEI is working with OCFO and senior agency managers
to define a fiduciary and technical management strategy that will address current
technical architecture shortfalls and provide more effective methods to maintain
the technology in the future.
Chapter 5 - EPA Needs to Strengthen IT Project Management Criteria
Findings
Response
5.1- Paragraph on No Reliance or Value
Placed on EPA's IT CPIC Process
Please discard this finding, it is inaccurate. Over the past five years of the CPIC
process, the Investment Management Team has worked with over 50 different
program mangers at one time or another. We have received positive comments
from program managers that the process has forced them to rethink their
investments and to pay closer attention to costs, schedule, and milestones. EPA
does acknowledge and place value on the need for the IT CPIC process.
61
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
5.2 - EPA had not adopted standard
tools to help managers plan, control,
and evaluate IT investment projects
and track project costs, schedules, and
resources.
Please revise this statement, it is inaccurate. First, with the development of the
CFO Comptroller Policy Announcement 01-10 and the IT Cost Tracking system,
program offices are required to track project costs. Secondly, as program offices
implement this requirement, it clearly complements and links to project planning
and work plan development.
5.3 - The absence of key decision
documents and senior management
approval increase the risk that funded
IT projects will evolve in an
unstructured, untimely, and costly
manner.
Please restate this finding to acknowledge efforts of the senior management and
decision making body of the QIC. The QIC, referencing recorded
recommendations from the IIS, formally acts on each IT investment. Formal
meeting notes are taken at each subcommittee meeting, reviewed and approved
by the co-chairs, and starting in January 2002, co-chairs signed the meeting
notes before being distributed to subcommittee members.
Chapter 5 - EPA Needs to Strengthen IT Project Management Criteria
Recommendations
Response
5.1- We recommend the Chief
Information Officer monitor IT
investments to ensure that SMPs are
prepared in accordance with Agency
requirements, and that they
appropriately link to the respective
Clinger-Cohen Act submission
documents the Enterprise Architecture
and other planning documents.
EPA agrees with this recommendation. As OEI prepares its Strategic Planning for
IT Investment Management, one of the areas being examined is the formalization
of processes and evaluations in all phases of the CPIC process, including the
possible inclusion of a Pre-Select phase. The Pre-Select phase will allow the
Agency to ensure that all proposed systems in the system lifecycle planning
process are aligned with Agency requirements on enterprise architecture,
security, etc. This Pre-Select phase will allow EPA to ensure compliance with
Systems Lifecycle Policy in advance of a system entering the Select Phase.
5.2 - We recommend the Chief
Information Officer re-evaluate funding
for IT investments that do not provide
sufficient written justifications for
projects exceeding budgeted costs or
project milestone schedules by more
than 10 percent.
Please rephrase this recommendation to accurately reflect the current process in
place. Systems without sufficient justification to cost and schedule variances
greater than 10% are not recommended for funding. As part of the Exhibit 300
submission, OMB is requiring that all major systems provide a breakdown of
costs and schedule performance from their original baseline. The Chief
Information Officer does not recommend investments for projects with
insufficient justification or those with excessive cost and schedule variances.
5.3 - We recommend the Chief
Information Officer prescribe standard
tools for managing system
development projects and for managing
software changes, as part of the
development of consistent definitions
of system life cycle stages to be used
for IT systems and project
management. The selected tool should
be approved by the Chief Financial
Officer as being compatible with the
Agency's cost accounting system.
OEI does not agree that it is necessary or appropriate to prescribe uniform tools
for managing system development projects and software changes since it is
unclear at this time that there is one set of tools which meets the needs of all
system development efforts in a cost-effective manner. However, OEI does
intend to broaden the scope and usefulness of I-TIPS with particular attention to
linkages between I-TIPS and Agency financial data for IT cost tracking. OEI is
also leading an effort to update EPA's System Life Cycle Policy. The updated
policy will provide appropriate consistent definitions, lay out the requirements
that must be met when an Agency office develops a new system, provide
appropriate system development management methodology options, and
encourage the use of "best practice" project management principles and
techniques. The selected "tools" will be compatible with the Agency's Financial
systems.
62
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 5 - EPA Needs to Strengthen IT Project Management Criteria
Recommendations
Response
5.4 - We recommend the Air Quality
System Project Manager update the
SMP for the Air Quality System project
and obtain the signature of approval of
the Assistant Administrator for Air and
Radiation at the conclusion of the
analysis stage and for major and
significance enhancements.
Please see the memo from William T. Hamett to Patricia H. Hill dated 5/28/02.
5.5 - We recommend the RCRA
Information Project Manager revise the
Project Management Plan for the
RCRAInfo project to make it equivalent
to an SMP, and update the document
for planned system design changes
and enhancements. In addition, the
revised SMP should be formally
approved by the Assistant
Administrator for Solid Waste and
Emergency Response to authorize
funding for the IT investment and to
ensure a system of accountability.
Please see the memo from Marianne Lament Horinko to Kimberly Nelson dated
6/14/02.
5.6 - We recommend the
SDWIS/STATE Project Manager
establish an SMP for the
SDWIS/STATE project and obtain the
signature of approval from the
Assistant Administrator for Water at
the conclusion of the analysis stage
and for major and significant
enhancements.
We agree with this recommendation and SDWIS/STATE has all the components
of a Systems Management Plan. However, the project has not compiled the
information into a single document for signature for the following reasons: First,
we have not been able to identify the format the agency wishes for the SMP and
second, a SMP was not specifically required when the project began.
Part of our plan for this fiscal year (may slide to early next F Y) is to compile the
document and present it to management.
5.7 - We recommend the Project
Managers for the Air Quality System,
RCRAInfo, and SDWIS/STATE link the
SMP to the Agency Clinger-Cohen Act
submission documents and the
Enterprise Architecture and planning
documents.
OAR — Please see the memo from William T. Harnett to Patricia H. Hill dated
5/28/02.
OW -- When the SMP document is completed it shall be linked to all IT
submissions (where applicable).
OSWER - Please see the memo from Marianne Lament Horinko to Kimberly
Nelson dated 6/14/02.
63
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Chapter 5 - EPA Needs to Strengthen IT Project Management Criteria
Recommendations
Response
5.8 - We recommend the Project
Managers for the Air Quality System,
RCRAInfo, and SDWIS/STATE
manage project development efforts in
accordance with the SMP, as updated,
throughout the life cycle of the system,
and retain the SMP for reference and
review by the CIO or the CIO's
designated review official.
OAR — Please see the memo from William T. Harnett to Patricia H. Hill dated
5/28/02.
OW -- We agree that the documents that go into the SMP should be updated
throughout the life-cycle of the system. We currently do this and with each new
release the following documents are updated (among others): requirements,
design, testing, and user documentation. Also, each fiscal year we produce a
new work plan. Finally, we continuously, update and track our financial reports.
OSWER - Please see the memo from Marianne Lament Horinko to Kimberly
Nelson dated 6/14/02.
Chapter 6 - Project Cost Accounting System Vital for Planning & Managing IT Investments
Findings
6.1 - Our concern is compounded by
the fact that the three system life cycle
categories set forth in the Policy
Announcement are inconsistent with
the phases described in EPA Directive
2100.
Recommendations
6.1- We recommend the Chief
Information Officer, Chief Financial
Officer, and Assistant Administrator
for Acquisition Management work
together to develop consistent
definitions of systems life cycle stages
and IT costs to be used for contracting,
accounting, IT systems, project
management, and the capital planning
investment control process.
Response
Please restate this finding to accurately reflect efforts in the Systems Life Cycle
work group and the IT Cost Tracking work group. Participants from OEI and
OCFO are on both work groups coordinating the IT Cost Tracking system
guidance, which includes policy development, and the Systems Life Cycle
development, updating our system life cycle policy. The life cycle categories
stated in the policy announcement reflect the new work that is being done to
update the systems life cycle policy.
Response
Please acknowledge the current ongoing efforts underway to meet this
recommendation. ITPPD is currently leading an effort to update EPA's System
Life Cycle Policy. This effort will develop consistent definitions that can be used,
to the extent practicable, throughout the Agency's varied processes that relate
to IT systems development. Additionally, ITPPD is supporting OCFO efforts in
developing an IT Cost Tracking system. As this system matures and focuses on
capturing "actual" budget cost data more accurately, and comprehensive training
is provided to program offices, management will be able to make better decisions
to evaluate investment priorities.
OCFO and OARM - submitting response under separate cover.
64
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Recommendations
Response
6.2 - We recommend the Chief
Information Officer, Chief Financial
Officer, and Assistant Administrator
for Acquisition Management work
together to amend all current Agency
software development contracts, and
require that all future IT software
development contracts be written to
require a contractor to break out and
separately report all IT software
development costs by the system
development life cycle.
OEI - With the following ongoing efforts - the updated Systems Life Cycle
Policy, the interim CPIC Policy (final soon to be released), architecture and the IT
Cost Tracking system - the modular contracting approach will be supported,
contractors will have better guidance on providing development costs, and
management will be able to make better decisions on investments. Please
acknowledge these efforts in your recommendation.
OCFO and OARM - submitting response under separate cover.
6.3 - We recommend the CIO and Chief
Financial Officer work together to
develop consistent systems life cycle
and IT costs definitions for revising
EPA Directive 2100, and the interim IT
activities policy guidance.
OEI — Please restate this recommendation to accurately reflect the current efforts
being developed between OEI and OCFO. ITPPD is currently leading an effort to
update EPA's System Life Cycle Policy. This effort will develop consistent
definitions that can be used, to the extent practicable, throughout the Agency's
varied processes that relate to IT systems development.
OCFO - submitting response under separate cover.
6.4 - We recommend Chief Financial
Officer lead an effort to complete a
needs and feasibility assessment of
alternatives to determine what types of
project cost information and
supporting documentation are needed
for the capital planning investment
control process and managing IT
projects.
Submitting response under separate cover.
65
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
66
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 3
Office of Chief Financial Officer's
Response to Draft Audit Report
July 19, 2002
MEMORANDUM
SUBJECT: Draft Report on Management of Information Technology Resources
Inspector General Audit Number 2001-0591
FROM: Joseph L. Dillon /s/
Comptroller
TO: Patricia Hill
Director for Business Systems (2421)
I appreciate the opportunity to respond to your draft report titled "EPA's Management of
Information Technology Resources under the Clinger-Cohen Act," Audit Number 2001-0591. The Office
of the Chief Financial Officer (OCFO) fully supports your emphasis on effective management controls over
EPA's information technology (IT) portfolio and, as you recommend, we are working closely with the
Office of Environmental Information (OEI), the Office of Administration and Resources Management
(OARM), and others.
Chapter 6 of your draft, "Project Cost Accounting System Vital for Planning and Managing IT
Investments" makes four recommendations for OCFO. A discussion of recent OCFO progress in
implementing IT cost accounting is below. Specific responses to your draft recommendations for OCFO
are attached.
As you note, Comptroller Policy Announcement No. 01-10, "New Information Technology
Accounting Requirements" (PA), has been in effect since October 1, 2001. The PA established a standard
method of tracking all IT related costs in the Integrated Financial Management System (IFMS).
As the PA states, OCFO recognizes that the IT cost accounting "procedures represent a new way
of doing business in the Agency." We are now evaluating results and have implemented a quality assurance
process to ensure the accuracy of the cost data for both large IT systems and projects, and for smaller
projects and general IT activities.
67
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
To help familiarize staff with the new information and its uses, an IT Cost Accounting section has
been added to OCFO@work at http://intranet.epa.gov/ocfo/policies/itcostacctg.htm. The section includes,
as promised in my November 19, 2001 response to your preliminary finding outlines and position papers,
several reports on FY 2002 spending for IT. OCFO plans to add instructional materials for system
owners, funds control officers, and others to this page.
To build on this year's experience, OCFO staff are working closely with OEI, the contracts
community, headquarters SIRMOs, regional IRM branch chiefs, a regional comptroller, and others. For
example, most regions are voluntarily piloting a method that uses two characters to classify their IT
investment in greater detail than required by the PA. Results of the pilot are now being evaluated, and
proposals are on the table to require a similar level of detail agency wide. Our goal is high quality cost
accounting without overly burdensome and time consuming requirements.
Sue Arnold 202-564-5192 can answer any questions.
Attachment
cc: Linda Combs
Mike Ryan
Mark Day
Terry Ouverson
Jim Rothwell
John Gherardini
Larry Wyborski
Krista Mainess
68
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
OCFO RESPONSES TO
OIG DRAFT RECOMMENDATIONS
Chapter 6 of the Inspector General's Draft Report on Management of Information Technology Resources
offers four recommendations for the CFO. OCFO's responses are below.
Recommendation 6-1 - Develop consistent definitions of systems life cycle stages and IT costs to be used
for contracting, accounting, IT systems, project management, and the capital planning investment control
process. (Joint recommendation for the CIO, CFO and Assistant Administrator, OARM)
Response - Comptroller Policy Announcement No. 01-10, "New Information Technology Accounting
Requirements" (PA) includes these detailed definitions. To help ensure consistency across the Agency,
OCFO has been an active participant in OEFs workgroup to update IRM Policy Manual 2100,
Chapter 17 - System Life Cycle Management, since the workgroup's inception in November 2001.
Recommendation 6-2 - Amend all current Agency software development contracts, and require that all
future IT software development contracts be written to require a contractor to break out and separately
report all IT software development costs by the system development life cycle. (Joint recommendation for
the CIO, CFO and Assistant Administrator, OARM)
Response - Attachment B of the PA requires that procurement documents show the life cycle phase,
allowing software development costs to be easily rolled up for capitalization. Attachment A requires
that project officers (POs), delivery order project officers (DOPO), and contracting officer technical
representatives (COTRs) ensure proper IT coding on funding documents, proper allocation of IT
activities on invoice payments, and proper classification of projects and systems under their control.
Recommendation 6-3 - Develop consistent systems life cycle and IT costs definitions for revising EPA
Directive 2100, and the interim IT activities policy guidance. (Joint recommendation for the CIO and
CFO)
Response - Please see response to Recommendation 6-1.
69
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Recommendation 6-4 - Complete a needs and feasibility assessment of alternatives to determine what
types of project cost information and supporting documentation are needed for the capital planning
investment control process and managing IT projects.
Response - As stated above, OCFO is now implementing a structured plan to evaluate the cost
information now required by the PA and to make appropriate refinements. We are working closely
with OEI in the light of OMB's new CPIC requirements, as well as with OARM, headquarters
SIRMOs, Regional IRM Branch Chiefs, representatives from the funds control and finance
communities, and others.
70
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 4
Office of Air Quality Planning and Standards'
Response to Draft Audit Report
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
Office of Air Quality Planning and Standards
Research Triangle Park, NC 27711
May 28 2002
MEMORANDUM
SUBJECT: Response to April 26, 2002 request for comments on Clinger-Cohen Act
FROM: William T. Harriett, Director
Information Transfer and Program Integration Division (MC-C304-03)
TO: Patricia H. Hill, Director for Business Systems
Office of the Inspector General for Audit (MC-2421)
This memorandum responds to your April 26 request for comments on the IG's recently released draft report
"EPA Management of Information Technology Resources under the Clinger-Cohen Act". The report primarily
discusses how Office of Environmental Information (OEI) and the Chief Information Officer have implemented this
important legislation. The report also refers to certain Agency data systems, such as Air Quality System (AQS). In
this respect, the report mentions AQS in two places.
One, on page 32, is in relation to an IG recommendation that a System Modernization Plan (SMP) be prepared
for AQS and approved by the Assistant Administrator/Office of Air and Radiation (OAR). We generally agree with
this and plan to revise the SMP and submit it for concurrence.
The other reference is on page 9. In this case, we are uncertain of the scope of the issue and have copied the
full paragraph from the draft to illustrate our uncertainty.
"The slowly evolving and decentralized approach being used to develop an IT investment control
structure has not been successful. EPA 's approach allowed IT projects to be funded without proper
justification, and in the absence of adequate management controls. EPA invested resources on outdated
systems that did not maximize the efficiency or resolve long-standing problems, such as integration of
environmental data. For example, the Air Quality System was funded $2.5 million for fiscal 2001,
although planned modifications did not include adapting the system to function in conjunction with EPA 's
Central Data Exchange portal".
71
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
From the last sentence, it appears there is a concern that AQS was not a part of Central Data Exchange
(CDX) in fiscal 01. However, given the preceding sentences, it appears there is also a concern that AQS is a project
funded without proper justification and without management controls. In addition, it could be interpreted there is a
concern that AQS is an outdated system. We do not believe the report provides an accurate characterization of AQS
if all of these concerns are intended for AQS.
With respect to the comment about AQS and the CDX, the AQS Information Technology (IT) budget
proposal submitted in FY-01 did include our intent to work with OEI on a joint CDX pilot project in FY-02. In fact,
OEI/OAR staff were actively meeting in F Y-01 to develop a work plan which was submitted to the Quality and
Information Council in late 2001 and approved in early 2002 (along with funding from the Agency's System
Modernization Fund). Work is now underway.
We also disagree with the IG comment that seems to imply that AQS is an outdated systems that does not
maximize the efficiency or resolve long standing problems such as integration of environmental data. The AQS is an
Oracle relational data base which is the Agency's recommended architecture for such applications. One benefit of
Oracle systems is their ability to be integrated with data from other Oracle data bases (such as those being developed
throughout the Agency). This technology is consistent with the Agency's approach for data integration; it is not
outdated technology.
If the report is intended to also portray AQS as a system with a lack of proper justification and absence of
adequate management controls, material support for this conclusion is lacking in the narrative. We are hopeful the
first two sentences of the above citation were not intended to apply to AQS. If they do apply, further explanation is
essential. In either case, some editing of the paragraph is recommended.
In summary, we believe this paragraph mischaracterizes the AQS system in many respects. I believe a
conference call with you or your staff would be helpful.
Again, thank you for the opportunity to comment and I look forward to discussing the matter with you at
your earliest convenience.
cc: J. Seitz, OAQPS
T. Curran, OAQPS
B. Kellam, ITPID
E. Lillis, ITPID
J. Summers, ITPID
I. Spons
R. Slade
72
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 5
Office of Solid Waste and Emergency Response's
Response to Draft Audit Report
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, B.C. 20460
JUN 1 4 2002
OFFICE
SOLID WASTE AND EMERGENCY
RESPONSE
MEMORANDUM
SUBJECT: OIG Draft Report "EPA's Management of Information Technology Resources Under the
Clinger-Cohen Act" Audit Number 2001-0591
FROM:
Vlananne Lament Horinko
Assistant Administrator
TO: Kimberly Nelson
Chief Information Officer (2823)
The Office of Solid Waste (OSW) agrees in principle with the general spirit of the OIG report and concurs
with the suggested future approaches to system development and project management in EPA. However,
contrary to its portrayal in the draft report, we believe that RCRAInfo serves as a model for modular
system development, rather than an example for how not to develop systems in our agency. The modular
approach has enabled RCRAInfo to remain flexible to the changing needs of our constituent groups and
allowed us to avoid some administrative pitfalls other projects have encountered. It has also eased the
administrative burden.
The modular approach uses the Program Area Analysis in its development of requirements for
RCRAInfo, which is then approved by senior management before actual development occurs. This
inevitably leads to RCRAInfo being in more than one stage of the system development life cycle. We
made this choice intentional to allow the system to adapt in a timely, flexible manner to changing
program requirements. Before the beginning of each major project within RCRAInfo, senior
managers agreed on the need, and benefit, of continuing with that specific project. Senior managers
also agreed on levels of funding for each project.
On page 31, the report states that, "Despite several years of effort, management was still defining
RCRAInfo requirements'. While some requirements are still being defined for a few RCRAInfo modules,
the majority of the RCRAInfo modules are well past this stage and in the development stage. OSW
believes that the use of the Information Engineering model, combined with the separation of RCRAInfo
into distinct modules that can be independently analyzed and developed, is an appropriate methodology to
use for a large, complicated, and dynamic system such as RCRAInfo.
Additionally, the report implies that work on the most crucial modules was put off while earlier modules
were developed. The report fails to mention that EPA and its State partners deliberately approached each
RCRAInfo module in a consensus order established by senior decision makers. To make the broad RCRA
73
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
analysis more usetul, EPA and the States decided which 5 areas were appropriate tor detailed analysis and
on the order in which modules would be pursued. Staging the analysis in this manner allowed a number of
improvements to move forward (e.g., one recommendation from an early module led to consolidating site
information across three different mechanisms into a single form) while appropriate expertise (e.g.,
compliance personnel) could be directed at the last two modules on a separate track. The schedule also
reflected the availability of key/personnel to work on modules.
Finally, the following comments are offered regarding recommendations made specifically for
RCRAInfo:
Recommendation 5-5: Revise the Project Management Plan for the RCRAInfo project to make it
equivalent to a System Management Plan (SMP) and update the document for planned system
design changes and enhancements. In addition, the revised System Management Plan (SMP) should
be formally approved by the Assistant Administrator for Solid Waste and Emergency Response to
authorize funding for the IT investment and to ensure a system of accountability.
Recommendation 5-7: Link the SMP to the Agency Clinger-Cohen Act submission documents
and the Enterprise Architecture and planning documents.
Recommendation 5-8: Manage project development efforts in accordance with the SMP, as
updated, throughout the life cycle of the system, and retain the SMP for reference and review by
the CIO or the CIO's designated review official.
We feel the current development and management structure in place for RCRAInfo already meets the
recommended actions and that no change is needed in that structure. RCRAInfo has a System
Management Plan (SMP) in place, as well as a change and enhancement plan. In addition to the Capital
Planning and Investment Control Proposal (CPIC) process, RCRAInfo adheres to a formal approval
process for the Assistant Administrator for the Office of Solid Waste and Emergency Response to
authorize funding for the IT investment and to ensure a system of accountability.
cc: JeffWorthington
William Ocampo
Brion Cook
Linda Travers
Linda Garrison
74
Report No. 2002-P-00017
-------
EPA Management of Information Technology
Resources Under The Clinger-Cohen Act
Appendix 6
Report Distribution
Headquarters
Administrator
Deputy Administrator
Chief Financial Officer
Assistant Administrator for Air and Radiation
Assistant Administrator for Enforcement and Compliance Assurance
Assistant Administrator for Environmental Information
Assistant Administrator for Solid Waste and Emergency Response
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Regional Operations and State/Local Relations
Associate Administrator for Congressional and Legislative Affairs
Associate Administrator for Communications, Education, and Public Affairs
Agency Followup Official (2710)
Agency Followup Coordinator (2724)
Headquarters Library
Office of Inspector General
Inspector General
Regional Offices
Regional Administrators
Regional Libraries
Other
General Accounting Office
National Academy of Public Administration
75
Report No. 2002-P-00017
------- |