Earned Value Management
Fiscal Year 2004 -
Lessons Learned
Prepared by the Office of Environmental Information
Office of Technology Operations and Planning
Information Technology Policy and Planning Division
Contact for more information:
Anne Mangiafico, CPIC Team Lead
(202) 564-9483
mangiafico.anne@epa.gov
December 17, 2004
-------
INTRODUCTION:
During FY 2004, EPA implemented its Earned Value Management System
(EVMS) based on the ANSI/EIA-748 Standard and OMB requirements
identified in Circular A-11. Earned Value Management (EVM) provides a
standard means of objectively measuring work accomplished based on the
budgeted value of that work - it is "What you got for what it cost." EVM is a
project management technique that integrates cost, schedule, and
technical performance measures to monitor and control project resources
and compile results into one set of metrics so that effective comparisons
can be made. It also helps evaluate and control project risk by measuring
project progress in monetary terms.
In the process of implementing its EVMS, EPA learned several valuable
lessons that will contribute to positive improvements in its EVMS and
related policies, processes, and procedures in FY 2005. These lessons
learned are presented in this document to help assist information
technology (IT) project managers in their efforts to fully implement EVM on
their IT projects in a manner that is consistent with both the letter and spirit
of the ANSI/EIA-748 standard, and in line with the requirements of the
OMB Exhibit 300 process. The lessons are grouped into the following
three major areas:
1. Refinement of Earned Value Management Methods
2. Increasing Consistency of Project Reporting
3. Facilitating Management Analysis of EVM Data
1. Refinement of Basic Earned Value Management Methods
As is often the case when attempting to implement new policies, procedures, and
methods Agency-wide, basic competency and understanding are often more crucial
than demonstration of highly refined skills and sophistication. While all EPA projects
were able to successfully implement EVM methods and demonstrate key EVM
competencies, work remains to be done in increasing the overall sophistication and
refinement of the EPA EVMS on an organizational level. The following lessons learned
will help to further expand and refine the EVMS capabilities of the Agency as a whole.
• Use EVM Across all Phases of Mixed Life-Cycle Projects: Many EPA projects
are mixed life cycle - i.e., they include resources for Planning, Acquisition
-------
(Development, Modernization, or Enhancement (DME)), and Maintenance activities
and milestones. In many cases there are Development activities ongoing at the
same time that Operations and Maintenance (O&M) activities are occurring. Though
OMB only requires the usage of EVM for the resources and milestones in the DME
phase, the Agency will encourage the use of EVM for the O&M resources and
milestones of mixed life cycle projects as well. Doing so is in line with the true spirit
of EVM, and in many cases, doing so does not impose a greater burden on these
projects. In some cases it actually simplifies reporting through usage of a common
approach across all system life-cycle (SLC) phases and activities, and provides the
Project Managers (PMs) with a more complete picture of the health of the entire
project, not just certain segments of the project.
Separate Milestones and Associated Resources by System Life-Cycle Phase:
In some projects, milestones include both DME and O&M activities and resources.
Though there is nothing inherently wrong with organizing project milestones and
resources in such a fashion, it can make reporting of costs in the OMB Exhibit 300
difficult, and can potentially obscure the root cause of variances that may be
associated with a particular SLC Phase and not another. Program Offices should
separate milestones and their associated resources by SLC phase wherever
possible.
Separate Milestones and Associated Costs by Contractor Whenever Possible:
Many EPA projects utilize multiple contractors on the same project. Ensuring that
milestones are specific to a single contractor -rather than multiple contractors - helps
to identify root causes when problems arise. This will also make corrective action
planning and subsequent measurement of future progress easier. Project Managers
should try to set up their projects so that each contractor has a separate set of
milestones. Individual contractor milestone data can then be aggregated for EPA
quarterly reports or for the Exhibit 300 if need be after the monthly evaluation is
complete.
Keep Milestones from Getting Too Large in Duration, Cost, or Scope: While
having too many detailed, micro-level milestones can make reporting burdensome
and time consuming, projects should avoid aggregating activities and resources to
such a degree that it impacts the PM's ability to identify variances, and develop
corrective actions. EVM is most effective when implemented using a "bottom-up"
approach that dictates that information is planned and managed in small increments
which can be quickly and accurately cumulated to view and manage the project as a
whole. Examining small, manageable "chunks" is a more efficient process for
identifying problems and root causes, and allows the PM to assess the health and
risks of a project more accurately. Generally, small milestones are easier to plan for
(their scope can be defined more specifically) and can be measured more
objectively than large ones. Project Managers should ensure that milestones (or
sub-milestones) are as small and specific as possible, in terms of scheduling.
-------
Attempt to Limit Milestones Durations to a Single Fiscal Year (or Less),
Whenever Possible: Many EPA projects had milestones that spanned one or more
Fiscal Years (FY). This makes the task of reporting cost, performance, and EVM
data in the OMB Exhibit 300 context much more complex. For milestones that are
expected to run across FYs, breaking the milestones into phases (Phase 1, Phase
2) that correspond to a period not longer than the FY, or that start or stop at the
beginning or end of a FY, often helps in evaluating performance and success given
the FY-driven nature of Federal funding decisions and processes. Also, as
mentioned previously, EVM is more effective when implemented from a "bottom-up"
rather than a "top-down" approach. Milestones should be as small in duration as
possible, to help in the identification and resolution of issues that might otherwise be
obscured or go unnoticed within the context of large, multi-year milestones.
Establish Objective Measures for Determining Earned Value: Many projects
that did not have previous experience using EVM lacked truly objective methods for
determining percent complete for their milestones. In some cases this resorted to
more subjective methods/determinations for calculating percent complete and
Earned Value. Establishing objective measures for earning value on specific
milestones or portions of milestones is a crucial step in using EVM as a tool for
improving project performance and results. For example, if one of the project's
milestones is "requirements specification completion," determine how the value for
this entire milestone will be earned over time, i.e., how will the percent complete for
this milestone be determined? If there are five separate requirements specification
documents/deliverables that when completed, constitute overall completion of the
milestone, then assign a value to each of these five documents/deliverables. This
will allow you to correlate the completion with a definitive percent complete. For
example, Specification A, when completed, will constitute a 30% overall completion
percentage for the requirements milestone, while completion of Specification B
would correlate to a 20% completion percentage.
2. Increasing Consistency of Project Reporting
Using a standard reporting format and implementing a standard reporting cycle, EPA
has improved the EVM data for portfolio analysis.
• Use Standard Template for Reporting: In the initial stages of the EVMS
implementation process at EPA, projects were asked to provide information on key
EVM metrics and data for evaluation by management. The format of this information
was largely left up to the discretion of the Project Manager. It quickly became
apparent that data submitted in a variety of different formats, e.g., Microsoft Word,
Lotus 1-2-3, Microsoft Excel, made it very difficult to compare or aggregate data
across projects.
-------
As a result, EPA established a standard template for reporting EVM data for FY
2005 and beyond that will help ensure consistent data across projects for use in
management analysis. This standardized Microsoft Excel template is now available
on the Capital Planning and Investment Control (CPIC) Web site for all projects
required to use EVM. Using the template will ensure that EVM calculations will be
accurate and complete. It also allows data from multiple projects to be more easily
aggregated for purposes of generating the Agency's EVM portfolio package. In
addition, the template reduces the burden on the PM, as key metrics are auto-
calculated for projects based on basic EVM data inputs. This allows the PMs to
focus less on data entry and manipulation, and more on understanding the overall
performance and direction of the project, and developing strategies to resolve any
cost, schedule, or performance variances.
Institute Standard Reporting Cycles: The Agency also implemented a quarterly
review process for EVM data. Project data are submitted to the CPIC EVM team
each quarter for purposes of analysis. An initial review of the data is performed, and
any issues and/or questions related to the data are discussed and resolved prior to
being presented to EPA Senior Management for analysis. This standardized
process - in conjunction with the standard EVM template used to report EVM data -
greatly enhance the effectiveness of the Agency EVMS.
3. Facilitating Management Analysis of EVM Data
While several of the lessons learned highlighted under Section 1 and 2 above have
helped facilitate management analysis of EVM data, several additional lessons learned
in FY 2004 have made analysis of EVM data by Agency Management easier.
• Provide Both Numerical and Graphical Depictions of EVM Data: Initially, much
of the Agency-level data provided to EPA Senior Management for review was textual
or numerical in nature; the only graphical display of EVM data was on a single
project basis. As the information reported each quarter became more consistent
from project to project, additional Agency-level graphical depictions were provided to
assist in analysis of the Agency's IT portfolio as a whole (e.g., "Bulls-Eye" chart).
These graphical outputs allow for quick recognition of problem projects and trends,
and enable management to focus its resources on those projects or issues that
require the most attention. Over time the Agency hopes to continue these efforts
through use of automated tools and software suites that will enable a broad range of
data presentation options, including a variety of graphical depictions for key EVM
data.
• Use Color-Coded Standardized Scoring System: EPA has developed a color-
coded, standardized scoring system - based on the OMB E-Government scorecard -
-------
that quickly allows Senior Managers responsible for reviewing project performance
to know where that project stands. Projects with a score of "Green" are considered
in good standing, while projects scoring "Yellow" and "Red" indicate concerns that
merit additional attention from Agency Management.
CONCLUSION:
By building upon these lessons learned in FY 2004, EPA will further strengthen its
EVMS over the course of FY 2005. The Agency also will continue its commitment to
continuous improvement in the area of Earned Value Management by focusing on
routine quarterly training as well as expansion and improvement in documentation and
guidance materials such as these.
------- |