U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Audit Report
EPA Personnel Access and
Security System Would Benefit
from Improved Project
Management to Control Costs and
the Timeliness of Deliverables
Report No. 08-P-0271
September 22, 2008
-------
Report Contributors: Rudolph M. Brevard
Cheryl Reid
Teresa Richardson
Abbreviations
CMM Contracts Management Manual
EPA U.S. Environmental Protection Agency
EPASS EPA Personnel Access and Security System
HSPD Homeland Security Presidential Directive
IT Information Technology
OARM Office of Administration and Resources Management
OIG Office of Inspector General
OMB Office of Management and Budget
SDLC System Development Life Cycle
SLCM System Life Cycle Management
SMD Security Management Division
SOW Statement of Work
WQX Water Quality Exchange
-------
I
a
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
08-P-0271
September 22, 2008
Why We Did This Review
We evaluated the cost
justifications for major
Information Technology (IT)
investments in the U.S.
Environmental Protection
Agency (EPA) IT investment
portfolio. We also evaluated
contracted work for IT
investments to determine
whether the work met EPA's
(1) time and budget estimates,
and (2) intended needs.
Background
EPA received $346 million in
system development and/or
maintenance funding for
Fiscal Year 2007. This
funding includes IT
acquisition costs for contract
services to develop and/or
maintain IT systems.
For further information,
contact our Office of
Congressional and Public
Liaison at (202) 566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2008/
20080922-08-P-0271 .pdf
Catalyst for Improving the Environment
EPA Personnel Access and Security System
Would Benefit from Improved Project Management
to Control Costs and the Timeliness of Deliverables
What We Found
EPA has put into place processes to adequately justify costs of projects identified
in its IT investments portfolio. However, the lack of key project management
practices prevents it from achieving many of the projected milestone and budget
estimates. In particular, EPA did not require the EPA Personnel Access and
Security System (EPASS) contractor to follow Agency procedures for system
development. EPASS did not have a Project Manager authorized to oversee the
contractor's work. EPA also paid for invoices that contained contractor labor
overcharges. These system development procedures are designed to help
management better predict and control project costs. Had EPA implemented
processes to mitigate many of the identified system development weaknesses, it
would have been better able to anticipate and possibly avoid most of the additional
$983,216 in costs for EPASS. Further, had EPA implemented formal review
procedures for contractor invoices, it would have prevented paying an estimated
$75,276 in over-billed contractor labor charges. We were unable to determine
whether the EPASS work would meet EPA's intended needs because the project is
under further development.
What We Recommend
Our recommendations to the Director, Security Management Division, Office of
Administration, Office of Administration and Resources Management, are to:
• Develop and maintain an EPASS System Management Plan that includes
the required Change Management and information security documents.
• Appoint a certified EPASS Project Manager with authority to oversee
contractor work and ensure compliance with EPA's System Life Cycle
Management guidance.
• Issue a memorandum to all EPASS Task Order Project Officers that
outlines and reinforces expectations for complying with EPA invoice
reviewing guidance.
• Follow up with the Contracting Officer to ensure EPA collects from the
contractor the amount EPA overpaid for billing rate errors in the
contractor's invoices.
The Agency indicated that it has taken actions to address many of our concerns.
However, we believe the actions taken do not adequately address our
recommendations. The Agency needs to take steps to put into place a structure to
ensure that the EPASS project progresses through the System Development Life
Cycle process as required by EPA guidance.
-------
j
I
MEMORANDUM
SUBJECT:
FROM:
TO:
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
September 22, 2008
EPA Personnel Access and Security System Would Benefit
from Improved Project Management to Control Costs and the
Timeliness of Deliverables
Report No. 08-P-0271
Patricia H. Hill
Assistant Inspector General for Mission Systems
Wesley J. Carpenter
Director, Security Management Division
Office of Administration and Resources Management
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
The estimated cost of this report - calculated by multiplying the project's staff days by the
applicable daily full cost billing rates in effect at the time - is $391,452.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days. You should include a corrective actions plan for agreed upon
actions, including milestone dates. We have no objections to the further release of this report to
the public. This report will be available at http://www.epa.gov/oig.
If you or your staff have any questions, please contact me at (202) 566-0894 or
hill.patricia@epa.gov: or Rudolph M. Brevard, Director, Information Resources Management
Assessments, at (202) 566-0893 or brevard.rudy@epa.gov.
-------
EPA Personnel Access and Security System 08-P-0271
Would Benefit from Improved Project Management
to Control Costs and the Timeliness of Deliverables
Table of Contents
Chapters
1 Introduction 1
Purpose 1
Background 1
Scope and Methodology 2
Noteworthy Achievements 3
2 EPASS Needs Improved Contract Management and
System Development Practices 4
SMD Did Not Follow Agency Procedures for System Development 4
SMD Did Not Require Contractor to Deliver Tasks by Due Dates 6
SMD Approved Contractor Invoices Containing Overcharges 7
Improved Project Management Oversight Needed 8
Recommendations 8
Agency Comments and OIG Evaluation 9
Status of Recommendations and Potential Monetary Benefits 11
Appendices
A OIG Estimate of Efficiencies 12
B Agency Response 15
C Distribution 20
-------
08-P-0271
Chapter 1
Introduction
Purpose
We sought to determine whether the U.S. Environmental Protection Agency
(EPA) justified the Information Technology (IT) investments outlined in its
Capital Investment Plan. We also sought to determine (1) what contract work was
completed, (2) was it completed within time and budget requirements, and (3) did
the work meet EPA's intended needs.
Background
During Fiscal Year 2007, EPA received $346 million to support acquiring and
maintaining its IT systems. This funding included costs to procure contract
services to develop and maintain EPA systems.
EPA offices document the system acquisition strategies and costs in the business
cases that support their systems. EPA's Chief Information Officer reviews this
information for major IT investments through the Agency's Capital Planning and
Investment Control process. This process is a Federal mandate designed to assure
that investments in IT resources achieve high value outcomes at acceptable costs.
Upon funding of the proposed business cases by the Office of Management and
Budget (OMB), EPA offices commence system acquisition plans as detailed in
the business cases.
For IT investments reviewed during this audit, EPA offices used contract services
to acquire the systems. As such, the Contracts Management Manual (CMM) and
Interim Agency System Life Cycle Management (SLCM) procedures outline
EPA's contract management and system development requirements. In particular:
• The CMM requires the Contracting Officer to (1) verify usage of the
correct contract billing rates and (2) ensure billing rate changes are
correctly applied at the end of each contract period. The CMM also
requires the Contracting Officer to verify other conditions that may result
in re-calculation or adjustment of billing rates. Further, the CMM requires
offices to perform Government surveillance of the contract. The Agency
or appointee should review the receipt of services to ensure it is getting
what it requested and needed. Contracted services should also be
monitored for compliance with established timeframes.
• The SLCM procedures require offices to complete the system definition
phase prior to starting the System Development or Acquisition Phase.
-------
08-P-0271
Most importantly, the procedures require offices to define the systems'
functional, technical, and data requirements.
Scope and Methodology
We performed this audit from February through October 2007 at EPA
Headquarters in Washington, DC, in accordance with generally accepted
government auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient and appropriate evidence to provide a reasonable
basis for our findings and conclusions based on the audit objectives. We believe
the evidence obtained provides a reasonable basis for our findings and
conclusions.
We evaluated EPA program offices' management control processes for
compliance with Agency contracting and systems development requirements. We
reviewed contract documents related to the systems reviewed under this audit.
We interviewed EPA staff responsible for contractor work and management. We
also reviewed contract invoices and schedules of deliverables.
We judgmentally selected two EPA systems that represented 20 percent of EPA's
Fiscal Year 2007 IT investment portfolio. We did not include financial and
infrastructure IT investments, as we review these systems yearly during the
Agency's financial statement audit or these are included in the Office of Inspector
General's (OIG's) annual audit plan. We reviewed the following systems:
• Water Quality Exchange (WQX) System, within the Office of Water.
WQX provides a national picture of the surface and groundwater quality
of the United States. WQX is the result of the redesigned STOrage and
RETrieval water quality system. Under the Clean Water Act, EPA is
responsible for monitoring the ambient surface and ground waters of the
Nation. The Office of Wetlands, Oceans and Watersheds within the
Office of Water is responsible for developing WQX.
• EPA Personnel Access and Security System (EPASS), within the
Office of Administration and Resources Management (OARM).
EPASS is the Agency's implementation of Homeland Security
Presidential Directive-12 (HSPD-12), Policy for a Common Identification
Standard for Federal Employees and Contractors. This standard was
signed by the President of the United States on August 27, 2004.
Provisions 4 and 5 of the standard describe the timeline for federal
departments and agencies to implement the standard. Implementation of
the standard is to include both physical access to Agency facilities as well
as electronic or logical access to Agency IT systems. The Security
Management Division (SMD) within OARM is responsible for developing
EPASS.
-------
08-P-0271
We did not find notable weaknesses in regards to WQX acquisition and
subsequently informed the Office of Water of our findings. During preliminary
research, we also did not find notable weaknesses with EPA processes that
defined costs contained in Capital Planning and Investment Control business
cases and did not pursue this area during field work. We were unable to
determine whether the work would meet EPA's intended needs because the
EPASS project is under further development.
We had no prior report recommendations to follow up on during this audit.
Noteworthy Achievements
EPA's management stated it completed many key milestones for the EPASS
project. EPASS received the authority to operate on January 25, 2007, and
implemented a physical access control system at EPA's One Potomac Yard in
Alexandria, Virginia. EPA issued its first smart card in October 2006, and EPA
has and continues to issue smart cards to employees and non-Federal workers
throughout the Agency.
-------
08-P-0271
Chapter 2
EPASS Needs Improved Contract Management and
System Development Practices
Our review disclosed that EPA did not require the EPASS development contractor
to follow Agency system development procedures. This hindered management's
ability to control project costs. Management officials stated they were unable to
follow Agency procedures because of evolving requirements. However, we found
EPA did not use a change management process to guide them in decisions for
accepting risks resulting from the effects of these changing requirements.
Although a qualified Project Manager was on the EPASS team, the Project
Manager was not authorized to oversee the contractor's work or was not
positioned within the organization to influence major decisions made related to
the development of EPASS. We further disclosed that EPA paid additional
charges for invoices that contained errors in contractor labor charges.
Management's informal processes for reviewing invoices for accuracy did not
identify discrepancies before approval and payment. Had EPA implemented
processes to mitigate system development weaknesses, it would have been better
able to anticipate the additional $983,216 in costs for EPASS. Further, had EPA
implemented formal review procedures for contractor invoices, it would have
prevented paying an estimated $75,276 in billed contractor labor charges.
SMD Did Not Follow Agency Procedures for System Development
SMD's management of the EPASS project did not conform to key system
development requirements required by EPA SLCM guidance. In particular, SMD
proceeded to develop EPASS without (1) putting in place a structure to control
undefined EPASS requirements as they are known, and (2) appointing a qualified
Project Manager who has authority to oversee all EPASS development efforts.
EPASS Needs Clearly Defined Requirements and Implemented
Change Management Practices to Control Spending
SMD did not complete the EPASS Definition Phase before entering into a
contract to develop the system, nor did the contractor complete the Definition
Phase once SMD modified the Statement of Work. The Definition Phase defines
the system's functional, system, and data requirements and System Owners must
complete this phase as required by EPA SLCM guidance. The Definition Phase is
important because it assists management to ensure the intended system will
support Agency requirements and control project costs. Management stated they
could not complete the Definition Phase requirements because of the evolving,
changing, and increasing program requirements imposed by lead Federal
-------
08-P-0271
agencies. Therefore, SMD issued a Statement of Work (SOW) that did not have
detailed tasks that defined EPASS' system requirements. SMD then modified this
SOW to include detailed tasks, which the contractor prepared. However, these
detailed tasks did not require the contractor to perform a Definition Phase.
SMD had not put into place practices to validate newly defined HSPD-12
requirements and formally introduce the new requirements into the EPASS
system development process. A change management process is a key
management control used to record management decisions regarding evolving
system changes. During our discussions with management about the change
management processes, they seemed unaware of EPA requirements. After audit
field work, management provided us the OARM/Office of Administration
Software Development Software Configuration Management Plan in response to
our request for their change management procedures. This plan outlines the
contractor procedures for making system changes to EPASS, upon receipt of a
software change request/software error notice via a trouble ticket system.
However, management did not provide evidence of the processes it uses to
evaluate and approve EPASS system changes from evolving HSPD-12
requirements. Further, the plan (1) is a proprietary document used internally by
the contractor, (2) was not related to EPA-specific SLCM system development
requirements, and (3) was not formally adopted by EPA management. Also, EPA
management had not provided proof it implemented the practices outlined in the
plan.
We found that SMD had not developed a System Management Plan, as required
by SLCM guidance. This plan is the primary managerial document and serves as
a portfolio of required documents used by System Managers to control, assess,
and document the system throughout the SLC. EPA uses this plan as the principal
tool for organizing and managing system project/program management
information throughout the system life cycle.
Since SMD had not fully defined EPASS' requirements or implemented a process
to control unexpected system requirements, further EPASS system development
efforts are at risk. SMD needs to develop a full picture of EPASS' end state.
Without this full picture, SMD cannot measure the contractor's system
development work to ensure EPASS will meet EPA's desired needs. Had EPA
implemented processes to mitigate system development weaknesses, it would
have been better able to anticipate the additional $983,216 in costs for EPASS.
This upward trend of unanticipated costs has potential to continue because SMD
projects that EPASS development and implementation will continue through
2015.
EPASS Needs a Certified Project Manager
EPASS needs a Project Manager with the skills, qualifications, and authority to
oversee a High-Risk system development project. SMD assigned a Project
-------
08-P-0271
Officer to oversee the contractors developing EPASS. However, the Project
Officer's main responsibility was to perform contract management functions and
the Project Officer does not possess the qualifications or skills needed to manage
system development activities for a high-risk project like EPASS. The EPASS
Project Officer was not familiar with the Agency's SLCM requirements and, as
such, was not familiar with system development techniques or processes to reduce
the risk to the Agency for this high-risk project.
There was a qualified Project Manager on the EPASS development team who
indicated some involvement with system development and system design;
however, the Project Manager was not given responsibility for monitoring the
contractor's progress, work, and costs. The Project Officer did not want the
assigned Project Manager to have authority or responsibility for (1) reviewing the
contractor's monthly status reports, (2) monitoring work, and (3) reviewing
invoices, because the Project Officer stated they would not be comfortable with
the Project Manager having all of these responsibilities. The Project Officer
performs some of these duties, but does not have the required training and
experience to be appointed as a Project Manager, and does not have the time to
get the certification. Therefore, management listed the Project Manager on the IT
business case submitted to OMB for funding even though the Project Manager
was not fulfilling the role as required by OMB and EPA.
EPA's SLCM procedures require assigning a Project Manager who is responsible
for managing the entire project through its life cycle. These responsibilities
include managing the project's compliance with EPA SLCM policy and
procedures, funding and resources, and system development processes.
According to OMB, skilled project managers are critical in managing contractor
activities to ensure they achieve intended outcomes. As such, it appears that
management placed the certified Project Manager on the project team to receive
funding for EPASS and not to oversee the system development processes as
intended by OMB.
After audit field work, we learned that SMD issued a new SOW, with potential
funding of $9.6 million over the life of the contract. This new SOW will be used
to continue EPASS system development and deployment. SMD officials stated
that system development costs are about 10 percent of the new SOW. If SMD
uses a system development approach as specified in EPA guidance, we estimate
EPA could better anticipate $902,530 in unplanned project costs. See Appendix
A for details.
SMD Did Not Require Contractor to Deliver Tasks by Due Dates
Tasks listed in the modified EPASS SOW were either late or lacked information
on which to determine when the contractor was required to complete the assigned
tasks. EPA's CMM requires offices to perform government surveillance of the
contract. The CMM requires the respective office to review the receipt of
-------
08-P-0271
services to ensure it is getting what it requested and needed. The CMM also
requires that contracted services should also be monitored for compliance with
established timeframes.
SMD had the contractor prepare a detailed list of tasks with the dates the tasks
were due. However, our review of the tasks and milestones revealed that
59 percent (75 of 127) of the tasks were delivered at least 1 month or more late.
Also, 27 of the 127 tasks either did not have a due date or a date delivered.
Management had not responded to our inquiries regarding these late or undated
deliverables.
The Government Accountability Office recognizes that mature and effective
management of IT investments can vastly improve government performance and
accountability. Without good management, such investments can result in
wasteful spending and lost opportunities for improving delivery of services. We
feel this lack of oversight over deliverables, coupled with the absence of basic
system development practices as previously discussed, contributed to the
unpredicted overspending on the development of EPASS.
SMD Approved Contractor Invoices Containing Overcharges
From November 2005 through July 2007, SMD did not have formal processes for
reviewing invoices and did not identify incorrect labor charges on at least 10
monthly invoices paid by EPA. EPA's CMM states the Contracting Officer
should periodically verify usage of the correct rates. This includes reviewing
rates that change at the end of each contract period and verifying rates that are re-
calculated or adjusted for any other reasons.
We learned that SMD subsequently reviewed all previous contractor invoices,
identified billing discrepancies, and notified the Contracting Officer of the
discrepancy. The Contracting Officer, in turn, issued a written request to the
contractor regarding this matter. Based on our calculations, EPA paid an
estimated $75,276 in incorrect contractor labor overcharges. See Appendix A for
details.
We further learned that after field work, the new EPASS Project Officer
appointed five Task Order Project Officers and made them responsible for
reviewing contractor invoices. Although SMD did not document this new internal
review process, this informal practice resulted in SMD disapproving an invoice
due to questions over billing.
Having documented procedures is the cornerstone of an effective internal control
environment. Formal procedures help to ensure that personnel are aware of their
responsibilities and understand the tasks that management intends to be
accomplished. Because SMD uses a distributed structure for reviewing invoices,
-------
08-P-0271
it is imperative that SMD document these procedures to ensure processes are
followed during day-to-day operations and personnel turnover.
Improved Project Management Oversight Needed
In discussions with OARM management regarding these findings, management
indicated that:
• Although EPASS had not been able to comply with EPA's SLCM policy
for the definition phase, it has complied for management of other key
components, such as architecture planning, investment management, and
security planning.
• EPASS did, and continues to have, a Project Officer authorized to oversee
the contractor's work.
• OIG should focus on cost benefits of project accomplishments rather than
total expenses, among these, issuing 7,000 smart cards to EPA employees
and non-federal workers.
We recognize that developing an information system during a period where
federal requirements continually evolve is a significant undertaking for SMD and
its management. We further recognize that EPA is on the leading edge of federal
agencies that have issued smart cards to its civilian employees and contractors.
Although innovation involves taking risks, we feel that it is incumbent upon
management to implement practices for innovation to mitigate risks to an
acceptable level.
Developing EPASS is a high-risk undertaking. We feel that SMD chose to follow
an ambitious implementation plan, which resulted in SMD spending the total
project funding within 27 months. Our concern is that the Federal HSPD-12
requirements are now defined and SMD has yet to establish the formal processes
needed to minimize the risk to EPA and guide them in the continued development
of EPASS.
Recommendations
We recommend that the Director, Security Management Division, Office of
Administration, Office of Administration and Resources Management:
2-1 Develop and maintain an EPASS System Management Plan. The plan
should include all documentation that supports management's adherence to
all controls gates and decision points related to ensuring EPASS compliance
with prescribed EPA SLCM guidance. The plan should also include all
required change management and required information security documents.
-------
08-P-0271
2-2 Appoint a certified EPASS Project Manager as required by EPA SLCM.
The appointment memorandum should also include specific language to
reinforce expectations for that person to manage the EPASS project through
its life cycle and ensure compliance with EPA's SLCM guidance.
2-3 Issue a memorandum to all EPASS Task Order Project Officers that outlines
and reinforces expectations for complying with EPA invoice-reviewing
guidance.
2-4 Follow up with the Contracting Officer to ensure EPA collects from the
contractor the amount EPA overpaid for billing rate errors in the
contractor's invoices.
Agency Comments and OIG Evaluation
The Agency indicated that it has taken actions to address many of our concerns.
However, we believe the actions taken do not adequately address our concerns.
The Agency's complete response is at Appendix B.
In general, EPA disagrees with the report's findings. EPA indicated:
• It was not able to follow prescribed EPA system development guidance
because the requirements for the EPASS project were unknown at the
initiation of the project.
• A qualified Project Officer and Project Manager were involved in the
EPASS project from its inception. The Project Officer had overall project
responsibility while the Project Manager was to manage the IT aspects,
including the contractor's performance.
• There are no real cost overruns, savings to identify, or misspent monies.
• EPASS invoices are reviewed and paid following the guidelines set forth
in Chapter 11 of the Contracts Management Manual, and Chapter 3 of the
Recertification for Contracting Officer Representative Manual.
We found that although the EPASS requirements were not know at the initiation
of the project, EPA had not taken steps to put in place processes to control the
cost of the EPASS project. As such, EPA had not developed a System
Management Plan to manage the EPASS project and document key decisions and
control points completed as required by EPA guidance. Furthermore, OARM had
not implemented a Change Management Process to ensure that as new project
requirements occurred, there was a system in place to introduce these
requirements in the system development process.
-------
08-P-0271
Our research and interviews concluded that although the EPASS project had a
certified Project Manager listed on the project, the employee was not responsible
for ensuring the project progressed through the System Development Life Cycle
(SDLC) as required by EPA and OMB guidance. We found that the Project
Manager lacked authority to guide the EPASS project and was not receiving cost
information necessary to monitor the contractor's performance. We believe that
had OARM assigned a Project Manager with authority to guide the EPASS
project, OARM would have had a better handle over the unanticipated additional
costs for EPASS. Additionally, OARM would have been able to put into place
processes that would have minimized the risk to EPA when undertaking a high-
risk project with evolving requirements. Furthermore, our research and
interviews revealed that the assigned EPASS Project Officer lacks the knowledge
and experience necessary to provide system development guidance on a project of
this magnitude. Therefore, we believe that in order for EPASS to successfully
progress through the required SDLC stages, OARM should assign a certified
Project Manager with authority to guide the project.
With respect to OARM's invoice payment processes, although OARM assigned
five Task Order Project Officers responsible for reviewing the contractor
invoices, our subsequent interviews revealed that some personnel had not
received the invoices to review until August 2008. Furthermore, even though
OARM cites that it follows invoice review procedures outlined in EPA's Contract
Management Manual, we found that OARM had not issued guidance to the five
Task Order Project Officers outlining their specific responsibilities for
documenting invoice reviews. The documentation of invoice reviews is required
by EPA guidance, and because OARM has a distributed process for reviewing
invoices, it is incumbent upon management to set the standards for this process to
ensure consistency.
OARM also provided a status of its actions to address the report's
recommendations. OARM indicated that it has taken sufficient action to address
the report recommendations. However, for the reasons cited above, we believe
OARM has not taken action to address the report's recommendations. OARM
should take steps to put in place a structure to ensure that the EPASS project
progresses through the SDLC process as required by EPA guidance.
10
-------
08-P-0271
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (in $OOOs)
Rec.
No.
Page
No.
Subject
Status1
Action Official
Planned
Completion
Date
2-1 8 Develop and maintain an EPASS System
Management Plan. The plan should include all
documentation that supports management's
adherence to all controls gates and decision points
related to ensuring EPASS compliance with
prescribed EPA SLCM guidance. The plan should
also include all required change management and
required information security documents.
2-2 9 Appoint a certified EPASS Project Manager as
required by EPA SLCM. The appointment
memorandum should also include specific
language to reinforce expectations for that person
to manage the EPASS project through its life cycle
and ensure compliance with EPA's SLCM
guidance.
Director, Security
Management Division,
Office of Administration,
Office of Administration and
Resources Management
Director, Security
Management Division,
Office of Administration,
Office of Administration and
Resources Management
Claimed
Amount
Agreed To
Amount
$902.5
2-3 9 Issue a memorandum to all EPASS Task Order
Officers that outlines and reinforces expectations
for complying with EPA invoice-reviewing
guidance.
Follow up with the Contracting Officer to ensure
EPA collects from the contractor the amount EPA
overpaid for billing rate errors in the contractor's
Director, Security
Management Division,
Office of Administration,
Office of Administration and
Resources Management
Director, Security
Management Division,
Office of Administration,
Office of Administration and
Resources Management
$75.2
0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
11
-------
08-P-0271
Appendix A
OIG Estimate of Efficiencies
I. Estimated Efficiencies for Recommendation 2-1
The condition found involves:
Reduction in Outlays
De-obligation of Funds
Avoidance of Unnecessary Expenditures
Increase in Revenue (e.g., Uncollected Fees)
X Other
Based on SMD's anticipated costs for the current SOW, the OIG estimates SMD spent
approximately $1,321,946 more than anticipated for the first 2 years. SMD has prepared a new
SOW to continue system development and deployment. It estimates 10 percent of the new SOW
will be for system development. If SMD follows OIG recommendations, the estimated efficiencies
will total $902,530 for the new SOW's base year and 4 option years as described below.
Estimate involves efficiencies/savings related to:
a one-time event
X the current and following year for operations of a continuing nature
the next 5 years for reductions in a long-term program or program terminations
Calculation of Gross Savings
The OIG estimates that SMD could avoid project costs escalating over budget on the new
EPASS contract by an amount similar to what was underestimated on the EPASS contract that
ended in January 2008. Management indicated that approximately 10 percent of the new EPASS
$16,936,737 contract is related to system development efforts by the contractor. The OIG's
calculation of Gross Savings is as follows:
Current SOW
The first calculation relates to the base period and option period 1. Each period is 12 months,
beginning in November and ending in October.
Amount Budgeted for Base Period $ 765,863
Amount Budget for Option Period 1 + 622,037
Total Budgeted for Base Period and Option Period 1 $ 1,387,900
Paid Invoices through July 2007 (21 invoices) $ 2,371,116
Total of Budget Base Period and Option Period 1 - 1.387.900
Amount Underestimated through July 2007 $ 983,216
12
-------
08-P-0271
The following calculation estimates the cost of invoices not yet approved (August-October 2007)
for the current period. We did this to project an amount for a full 12 month period. We
calculated a monthly estimate by averaging the total amount of all invoices received.
Paid Invoices through July 2007 (21 invoices) $ 2,371,116
Average amount per invoice ($2,371,116 721 invoices = $112,910)
Estimate for 3 Months of Invoices (August-October 2007)
($112,910 X 3 months) + 338.730
Total Estimated Project Costs $ 2,709,846
Total Amount Unanticipated ($2,709,846 - $ 1,387,900) S 1.321.946
Percentage of Unanticipated Costs on Current SOW
($1,321,9467 $1,387,900) 95%
New SOW
Amount Budgeted for New SOW $9,611,890
Percentage of SOW Identified as System Development 10%
Amount Attributed to System Development ($9,611,890 X 10%) $ 961,189
Percentage of Historical Unanticipated System Development Costs 95%
Estimated Unanticipated Costs if
Recommendation 2-1 is Not Implemented ($961,189 X 95%) $ 913.130
(a) Gross Estimates of Efficiencies S 913.130
Calculation of Cost to Implement Recommendation 2-1
The OIG estimates it will take SMD 10 days to comment on the OIG's estimate; 5 days to draft
the technical direction memorandum; and 2 days for the Contracting Officer to review the
technical direction memorandum and issue it to the contractor. The cost to implement is
estimated as follows:
Estimated 7 days by GS-15 at $700 per day $ 4,900
Estimated 7 days by GS-14 at $600 per day + 4,200
Estimated 3 days by GS-13 at $500 per day + 1.500
(b) Total estimated costs to implement $10.600
Estimate of Net Efficiencies/Savings
(a-b) or ($913,130-$10,600) S 902.530
13
-------
08-P-0271
II. Estimated Efficiencies for Recommendation 2-5
The condition found involves:
Reduction in Outlays
De-obligation of Funds
Avoidance of Unnecessary Expenditures
Increase in Revenue (e.g., Uncollected Fees)
X Other
Management approved contractor invoices that contained overcharges. The contractor
overcharged on at least 10 monthly invoices for incorrect labor rates or incorrect labor
categories. As a result, EPA overpaid an estimated $75,275.66 in contractor labor charges.
Estimate involves efficiencies/savings related to:
a one-time event
X the current and following year for operations of a continuing nature
the next 5 years for reductions in a long-term program or program terminations
Calculation of Gross Savings
SMD identified 10 invoices in which the contractor over-billed EPA for incorrect labor charges.
The calculation of gross savings is as follows:
Invoice Month Amount Overcharged
September 2006 $ 9,959.08
October 2006 11,504.21
November 2006 4,232.42
January 2007 5,548.55
February 2007 3,369.80
March 2007 4,764.20
April 2007 3,718.40
May 2007 5,112.00
June 2007 10,663.05
July 2007 16.403.95
(a) Gross Estimate of Efficiencies S75.275.66
Calculation of Cost to Implement Recommendation 2-5
The OIG estimates it will take SMD 1 hour to follow up with the Contract Officer to ensure
EPA has received payment from the contractor for overcharges.
Estimated .0125 day by GS-15 at $700 per day 87.50
(b) Total estimated costs to implement $ 87.50
Estimate of Net Efficiencies/Savings
(a - b) or ($75,275.66 - $87.50) $75.188.16
14
-------
08-P-0271
Appendix B
Agency Response
August 5, 2008
MEMORANDUM
SUBJECT: OARM Response to Draft Audit Report:
EPA Personnel Access and Security System Would Benefit
From Improved Project Management to Control Costs and the Timeliness
of Deliverables
Assignment No. 2007-000557
FROM: Wesley J. Carpenter, Director /s/
Security Management Division
TO: Rudolph M. Brevard, Director
Information Resources Management Assessments
OARM appreciates the opportunity to comment on the latest version (June 24, 2008) of
the Draft OIG Audit Report of EPASS, Assignment Number 2007-000557. We believe that
most of our comments pertaining to the earlier drafts are still valid; therefore, we have attached
and are resubmitting them for inclusion in the final report.
We thank you again for your consideration and hope that we can reach a satisfactory
resolution of these issues.
Attachment
cc: Renee Page
Dennis Bushta
Cheryl Reid
15
-------
08-P-0271
OARM'S COMMENTS
Our comments are organized by the four themes highlighted in the latest version (June 24, 2008) of the
OIG discussion draft audit report on EPASS. Per the OIG's request, ancillary comments have been
added to each theme to better depict and summarize previous comments submitted by OARM during its
review of the three previous draft reports.
1. OIG Theme No. 1: OARM did not follow EPA's interim System Life Cycle Management (SLCM)
procedures, which require proposed IT systems be defined in terms of functional, technical,
and data requirements prior to project initiation, development, or acquisition.
OARM's Comments: In order to maximize the effectiveness of the SLMC in developing new IT
applications, a clear knowledge of functional, technical and data requirements is essential prior to
project initiation, development, or acquisition. Unfortunately, such complete knowledge was not
available by the time the EPASS project had to be initiated. If EPA had delayed initiation until all up-
front information had been available, the Agency would not have been able to meet federally
mandated implementation deadlines.
• The IG report does not mention that the EPASS project was mandated by the White House and
was the first of its kind ever undertaken by the Federal government, EPA, or the private sector.
Because of HSPD-12's stringent implementation deadlines, Agency activities had to be initiated
amid many uncertainties and unknowns, changing requirements, and equipment and technology
use restrictions.
• At the time of contract award, final HSPD-12 PIV standards had not been issued nor had the
relevant equipment or software been properly tested and approved by NIST and GSA for
inclusion on the government's approved procurement list (APL).
• Over the life of the project, additional or supplemental OMB policy and NIST technical documents
have been published adding either new requirements or amending those already in place. In fact,
between March 2006 and August 2008, a total of 11 technical documents impacting HSPD-12
configuration and specifications were issued creating additional work for all agencies.
Ancillary Comments OIG Theme No. 1:
• In order to accurately portray EPASS, the report should provide a fair and equitable description of
why the program was implemented, what the program is designed to accomplish, its mandates,
timeframes, and the circumstances surrounding implementation. Insert a background statement
on EPASS in the report's introduction to provide the necessary framework to completely
understand the full complexity of the program.
• The report states that EPASS lacked a detailed statement of work (SOW). The reason the SOW
did not contain detailed tasks had nothing to do with the allegation that SMD did not follow SLCM
procedures. In the case of a project where little is known about specific requirements, it is not
uncommon for the SOW to be void of detailed tasks and deliverables. The original EPASS
contract recognized this and, upon award of the first option year, the contract was amended to
include detailed tasks and deliverables.
• The OIG report states that 59 percent (75 of 127) of EPASS' tasks were either late or lacked
information on due dates. It also states the SOW didn't contain specific tasks. These are
statements are conflicting; they need to be reconciled prior to the next iteration of the report.
• OARM strongly recommends that the OIG interview the EPASS CO to better understand the
contracting process and how the EPASS contract was advertised and awarded. This request has
continually been ignored.
16
-------
08-P-0271
2. OIG Theme No. 2: OARM did not assign an EPASS Project Manager who has the certification
and authority to oversee contractor performance and compliance with EPA's interim SLCM.
OARM's Comments: A qualified Project Officer and Project Manager (IT) were involved in this
project from inception. The Project Officer had overall project responsibility while the Project
Manager was to manage the IT aspects, including the contractor's performance.
• Since inception of this project in late 2005, all monthly reports and invoices were shared with the
PM.
• The PM played a key role in monitoring the ongoing performance of the contractor as well as
providing oversight and direction for the technical aspects of the contract.
Ancillary Comments OIG Theme No. 2:
• This conclusion is not supported by the facts. No such restriction was ever placed on the PM
(IT).
• OARM has strongly recommended that the OIG interview the EPASS PM to better understand
the details of EPASS contract administration and management. This request has continually
been ignored and neither the original PM, nor the CO, have ever been interviewed.
3. OIG Theme No. 3: Costs were more than expected and unanticipated; unnecessary
expenditures could have been avoided.
OARM's Comments: Due to the many uncertainties and unknowns that existed at the inception of
this project, total costs and time frames were underestimated. However, this does not support the
OIG's implication that funds were wasted or misused. The report's references to potential monetary
benefits, estimates of efficiencies, gross savings, and avoidance of unnecessary expenditures are
unsubstantiated and should be deleted.
• The IG Report continues to imply that OARM overran costs on the contract, which is misleading
as is the potential cost savings based on this notion.
• Any increase in costs was due to evolving, changing, and increasing program requirements
imposed by lead Federal agencies resulting in an expanded level of effort.
• The follow-on contract was awarded March 19, 2008, and includes a base year and four one-year
option periods with a total contract ceiling amount of $9.6 million.
• The best way to measure EPASS cost benefits is to evaluate project accomplishments against
total expenditures (i.e., OMB and internal EPA approvals of the HSPD 12 implementation plan;
meeting executive mandate to issue smartcards by October 26, 2006; implementing a federally
compliant physical access control system at Potomac Yard; and issuing almost 14,000
smartcards to EPA employees and non-Federal workers).
Ancillary Comments OIG Theme No. 3:
• There are no real cost overruns, savings to identify, or misspent monies; therefore, remove any
references to these unsubstantiated issues.
• If the OIG really feels that there is legitimate cost savings to capture, then the way to do it is by
means of a bona fide cost benefit analysis.
17
-------
08-P-0271
4. OIG Theme No. 4: OARM has no formal procedures for reviewing and approving contract
invoices or addressing overpayments.
OARM's Comments: EPASS invoices are reviewed and paid following the guidelines set forth in
Chapter 11 of the Contracts Management Manual and Chapter 3 of the Recertification for
Contracting Officer Representative Manual. It was this review that led to SMD identifying the
contractor's overbilling after receipt of the invoice from the contractor.
• Each month every invoice is reviewed by all TOPOs (IT, ID Proofing/ Registration, and PACS)
before final PM approval.
• Currently, the $75,276 overpayment has been suspended by the CO and COTR. The
contactor's request for the funds has been denied by the CO.
Ancillary Comments QIC Theme No. 4:
• This theme implies SMD has no process for reviewing invoices. This is not true; review of
contractor invoices follow the guidelines set forth in Chapter 11 of the Contracts Management
Manual and Chapter 3 of the Recertification for Contracting Officer Representative Manual. Each
month every invoice is reviewed by all TOPOs (IT, ID Proofing/ Registration, and PACS) before
final PM approval.
• The OIG report states that the EPASS project paid $75,276 in erroneously billed contractor labor
overcharges. What it fails to mention is this issue was raised by the EPASS PM prior to
approving the first invoice containing overcharges.
• Subsequent invoices containing overcharges were also paid. At issue was the contractor's ability
to increase its rates whenever a contract option period was exercised early.
• The EPASS PM was compelled to pay subsequent invoices pending the outcome of discussions
between the CO and contractor.
• Once a formal CO decision was rendered, all overcharges were recovered.
Status of Recommendations and Potential Monetary Benefits
2-1 Develop a Technical Direction memorandum that specifies how the contracting firm must
implement system development processes compliant with EPA's SLCM. Technical Direction
memorandum should specify that no system development should begin until the company
defines, and EPA approves, the requirements for the system under development. The Technical
Direction memorandum should be approved by the EPASS Contracting Officer and issued to the
company awarded the new EPASS contract.
Status: Section C.2, Compliance with EPA Policies for Information Resources Management
(EPAAR 1552.211-79, Oct. 2000), part (b) (1) of the newly awarded EPASS contract requires the
contractor to comply with the 2100 Series (2100-2199) of the Agency's Directive System which
contains the requirements for SLCM compliance.
Planned Completion Date: Complete on contract award date, March 16, 2008.
2-2 Develop and implement a formal Change Management process that meets the requirements of
EPA's SLCM guidance.
Status: Section C.2, Compliance with EPA Policies for Information Resources Management
(EPAAR 1552.211-79, Oct. 2000), part (b) (1) of the newly awarded EPASS contract requires the
contractor to comply with the 2100 Series (2100-2199) of the Agency's Directive System which
contains the requirements for SLCM compliance.
18
-------
08-P-0271
Planned Completion Date: Complete on contract award date, March 16, 2008.
2-3 Assign a Project Manager who has the certification and the authority to oversee the EPASS
project as required by EPA's SLCM guidance.
Status: We already have a certified PM with authority to oversee the contractor's performance.
Planned Completion Date: Since inception of the original contract.
2-4 Develop and document formal procedures for reviewing contractor invoices.
Status: EPASS invoices are reviewed and paid following the guidelines set forth in Chapter 11 of
the Contracts Management Manual and Chapter 3 of the Recertification for Contracting Officer
Representative Manual.
Planned Completion Date: Since inception of the original contract.
2-5 Follow up with the Contracting Officer to ensure EPA collects from the contractor the amount EPA
overpaid for billing rate errors in the contractor's invoices.
Status: The cost associated with the overpayment of $75,276 was previously suspended by the
CO, so the Agency has already recovered the money. The EPASS CO has officially disapproved
the contractor's request for a refund of these funds.
Planned Completion Date: Complete on January 16, 2008.
19
-------
08-P-0271
Appendix C
Distribution
Office of the Administrator
Assistant Administrator for Administration and Resources Management
Director, Office of Administration, Office of Administration and Resources Management
Director, Security Management Division, Office of Administration and Resources Management
Agency Follow-up Official (the CFO)
Agency Follow-up Coordinator
Office of General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Audit Follow-up Coordinator, Office of Administration and Resources Management
Deputy Inspector General
20
------- |