or
V
OFFICE OF INSPECTOR GENERAL
**t PrO^
Catalyst for Improving the Environment
Audit Report
Office of Acquisition
Management Can Strengthen
Its Organizational Systems
Report No. 2005-P-00006
February 17, 2005
-------
Report Contributors: Stephen Burbank
Andres Calderon
Mike Davis
Sanjay Prakash
Denton Stafford
Patricia Brooks Taylor
Abbreviations
EPA Environmental Protection Agency
OAM Office of Acquisition Management
OARM Office of Administration and Resources Management
OIG Office of Inspector General
-------
^eosr^v
¦a* ** U.S. Environmental Protection Agency 2005-P-00006
| \ Office of Inspector General February 17,2005
" At a Glance
Catalyst for Improving the Environment
Why We Did This Audit
This audit was conducted to
determine whether EPA's
Office of Acquisition
Management (OAM) had the
seven fundamental components
of a high-performing
organization:
• Leadership
• Strategic Planning
• Customer Focus
• Information and Analysis
• Human Capital
• Process Management
• Performance Results
Background
High performing organizations
are those that have the
necessary systems and
processes in place for achieving
their missions. In 2002, the
EPA Office of Inspector
General published its
"Assessing Organizational
Systems: A User's Guide."
This document is intended in
part to assist Agency managers
in determining whether EPA
has the components necessary
for mission accomplishment.
For further information, contact
our Office of Congressional and
Public Liaison at (202) 566-2391.
To view the full report,
click on the following link:
www.epa.gov/oig/reports/2005/
20050217-2005-P-00006.pdf
Office of Acquisition Management
Can Strengthen its Organizationai Systems
What We Found
OAM's management systems include various components necessary for
organizational success. We found that OAM:
• Communicates its vision, values, and strategic goals to employees and
customers.
• Focuses on its customers" needs.
• Emphasizes the development of its workforce.
While OAM has taken those positive steps to achieve its vision and goals,
opportunities for strengthening systems exist:
• OAM leadership created its vision and goals without taking all the actions
necessary to accomplish its vision. OAM should establish measures to
determine whether the vision and goals were accomplished.
• OAM needs to complete work load and work force analyses to identify full-time
equivalent and skill gaps.
• The information in OAM's Integrated Contracts Management System can
measure the timeliness, but not the quality and cost, of its services, and
improvements in this area should be pursued.
• OAM does not have data to measure its progress toward achieving its vision of
being the preferred business partner for all EPA contracts. It cannot determine
the percentage of EPA contracts handled by OAM. Further, OAM does not
obtain sufficient feedback on the extent contracts contributed to Agency
environmental and performance goals.
What We Recommend
We recommend that the Director, OAM, develop an action plan with milestones
for establishing measures and means of measuring progress against its goals,
complete a workload and workforce analysis, and capture data needed to analyze
short- and long-term performance in achieving its vision and goals.
The Agency generally agreed with the recommendations and indicated that certain
corrective actions would have to be taken over the long term.
-------
3
<
3
Q
V
v> ^
T>
Z
LU
MEMORANDUM
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
February 17, 2005
SUBJECT: Office of Acquisition Management Can Strengthen Its Organizational
Systems
Audit Report No. 2005-P-00006
FROM:
Robert Mitchell, Director for Contract Audits /s/
Office of Inspector General (242IT)
TO:
David J. O'Connor, Acting Assistant Administrator
Office of Administration and Resources Management (3101 A)
This is our final report on the subject audit conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This audit report contains findings that
describe the problems the OIG has identified and corrective actions the OIG recommends. This
audit report represents the opinion of the OIG and the findings contained in this report do not
necessarily represent the final EPA position. Final determinations on matters in this audit report
will be made by EPA managers in accordance with established audit resolution procedures.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days of the date of this report. You should include a corrective action
plan for agreed upon actions, including milestone dates. We have no objection to the further
release of this report to the public. For you convenience, this report will be available at
http://www.epa.gov/oig.
If you or your staff have any questions, please contact me at mitchell.robert@epa.gov. or
Stephen Burbank, Assignment Manager, at burbank.stephen@epa.gov.
cc:
Judy S. Davis, Director, Office of Acquisition Management (3801R)
-------
Table of Contents
At a Glance
Purpose 1
Scope and Methodology 2
Results of Review 2
OAM Has Taken Various Beneficial Actions 3
Opportunities for Strengthening Systems Exist 4
Recommendations 6
Agency Comments and OIG Evaluation 6
Appendices
A OAM Vision Statement 8
B Agency Response to Draft Report 9
C Distribution 12
-------
Purpose
The Environmental Protection Agency (EPA) relies heavily on contractors to
accomplish its environmental goals. During fiscal 2003, EPA processed 122,056
contract actions (contract awards, modifications, delivery orders, etc.) with a
maximum potential value of $1.2 billion. The Office of Acquisition Management
(OAM) is the acquisition arm of the EPA Office of Administration and Resources
Management (OARM), and the OAM Director is EPA's Procurement Executive.
Because effective contracting is crucial to accomplishing the Agency's
environmental mission, EPA's OAM must have the necessary systems and
processes in place if it is to provide program offices with the contract support
needed.
In November 2002, the EPA Office of Inspector General (OIG) published its
"Assessing Organizational Systems: A User's Guide." This document is intended
to assist the OIG and Agency managers in determining whether EPA
organizations possess the necessary components for mission accomplishment.
Drawing on the President's Management Agenda, the Federal Managers'
Financial Integrity Act, the Government Performance and Results Act, and
Malcolm Baldrige and high-performing organization theories, the Guide identifies
the components of organizations which, when working well as an integrated
system, lead to mission accomplishment. The following table includes the seven
components and provides a brief description of each.
Key Area
Description
Leadership
How senior leaders define the organization's vision and mission,
address values, and monitor mission accomplishment.
Strategic
Planning
How an organization develops and deploys strategic objectives and
action plans.
Customer
Focus
How preferences, requirements, and expectations of customers are
determined, and relationships with customers built.
Information
and Analysis
How an organization selects, gathers, analyzes, manages, and
improves its data, information, and knowledge assets.
Human
Capital
How systems and employee learning enable employees to develop
and utilize their full potential in meeting organizational objectives.
Process
Management
How an organization examines the key aspects of its processes,
including product, service, and business activities.
Performance
Results
How an organization examines performance in key business
areas, such as customer satisfaction, and function performance.
1
-------
Due to the significant role OAM has in supporting the Agency's mission, we
established an audit objective that asked whether OAM had the seven
fundamental components of a high performing organization.
Scope and Methodology
We performed this audit from April to October 2004 in accordance with
Government Auditing Standards, issued by the Comptroller General of the United
States. Our work included site visits to OAM Headquarters and its five divisions
(three in Washington, DC, and one each in Research Triangle Park, North
Carolina, and Cincinnati, Ohio).
The "Assessing Organization Systems: A User's Guide" provides a series of
questions that can be used to assess the health of an organization in each of the
Guide's seven areas. We applied questions in the Guide to determine whether
OAM had the necessary systems and processes in place for organizational
success.
To determine the extent to which OAM had the seven components of a high-
performing organization, we reviewed its vision, values, and strategic goals;
OARM customer surveys; OAM internal surveys; employee performance
agreements; monthly OAM performance requirements reports; the type of data
captured in OAM management information systems; OAM's draft workload
analysis; Quality Assessment Plans; and Contract Guidance Documents. We
interviewed the OAM Director, Deputy Director, all five Division Directors, and
employees in all divisions. Using a structured questionnaire, we judgmentally
selected 53 OAM personnel to interview, which included Service Center
Managers, Team Leaders, Contracting Officers, and Contract Specialists.
There have been no recent audits performed regarding similar OAM management
areas.
Results of Review
OAM's management systems include components necessary for organizational
success. OAM communicates its vision, values, and strategic goals; focuses on
its customers' needs; and emphasizes workforce development. However,
opportunities for improvement exist, because leadership created a vision and
goals without taking or planning all the actions needed to accomplish its vision.
Additional actions are necessary for OAM to determine whether it has achieved
its vision of EPA contracting offices being ". . . the preferred business
management partners for the protection of public health and the environment."
While what we found applies to all seven components of the Guide, we do not
separately address what we found for each component because they often overlap.
2
-------
OAM Has Taken Various Beneficial Actions
OAM Communicated its Vision, Values, and Strategic Goals
Appendix A provides OAM's vision statement. OAM managers developed the
vision, values, and goals, and established OAM workgroups to facilitate
accomplishing the vision. The strategic goal workgroups were responsible for:
Proposing performance standards for employees defining what goals meant in
terms of individual performance.
Providing examples of successful achievement of goals.
Examining whether new goals warranted changing OAM's awards structure.
OAM communicated its vision statement to its workforce and others through staff
meetings; its newsletter; and incorporating the vision statement, strategic goals,
and examples of successful performance in employee performance agreements.
OAM Focused on Customers' Needs
OAM focused on the needs of its customers (those within EPA in need of
contracts) in a variety of ways. OAM used OARM customer surveys, long range
acquisition plans, and transaction customer survey cards to assist in determining
customer needs and satisfaction. OARM periodically administered customer
surveys, and OAM intends to use the fiscal 2004 survey results to gain insight
into where and how it could improve customer satisfaction. OAM customers
completed long range acquisition plans, and OAM used them in its workload
analysis and planning efforts. Transaction customer survey cards were used,
albeit sporadically, to seek feedback on specific contracting actions so that OAM
could identify and correct potential problems.
OAM said it conducts market outreach to all EPA program offices and among all
levels of staff and management. For example, OAM holds monthly Contractor
Customer Relations Council meetings with its program office partners to discuss
upcoming acquisition issues, and the potential impact of new or changed
acquisition policies, procedures, and practices. OAM also participates in various
Agency conferences and meetings with program personnel, and conducts annual
outreach sessions with Senior Resource Officials.
OAM Emphasized Workforce Development
OAM's values and goals emphasize developing its workforce. Two of OAM's
four values and one strategic goal relate to human capital (see Appendix A).
OAM invested in its people through training, details, and rotations within and
outside of OAM; empowerment initiatives; retention incentives; and a mentor
program. Of the 53 people interviewed, 45 (or 85 percent) said they were
3
-------
satisfied with the relevance of training to the work they perform. In fiscal 2003,
30 percent of the OAM workforce had participated in details or rotational
assignments. OAM's new employee handbook discusses a retention incentive
and a mentor program.
Further, OAM used the Empowerment in OAM report, issued in September 2000,
as a catalyst for more investment in its workforce. For example, Quality
Assessment Plans were implemented as part of OAM's initiatives to empower its
divisions and regional contracting operations. The Plans, which are self
assessments of office operations, were created with the ultimate goals of
identifying and sharing best practices, and collaborating with other offices to
identify any weaknesses and formulate corrective actions.
Opportunities for Strengthening Systems Exist
OAM's leadership created a vision and goals without taking or planning all
actions necessary for accomplishing its vision. OAM needs to determine whether
there are gaps in the skills of its workforce and act on those gaps, and also needs
to address information system weaknesses. Program offices sometimes go to
other organizations for contracting services, but OAM does not know that
percentage and therefore the extent to which its vision and goals are being
achieved. Further, to improve its method for measuring performance
effectiveness, OAM needs better information on the quality and cost of its
services and better feedback on contract effectiveness.
OAM Needs Better Measures for Goal Accomplishment
OAM has not established measures for goal accomplishment. Accomplishing
strategic goals (see box) should not only help OAM achieve its vision but lead to
continuous improvements in the efficiency
and effectiveness of its operations. For
example, "improving internal or external
communication" and "attending, participating
in customer service visits" were used,
respectively, to describe successful
accomplishment of optimizing business
processes and providing business leadership.
However, OAM did not establish measures, or measurement methodologies, to
determine whether these goals were accomplished. Better defining what
successful accomplishment of each goal entails, and developing a means for
monitoring performance, will assist OAM in accomplishing its vision and lead to
continuous operational improvements, even after the vision is realized.
OAM's Strategic Goals
• Investing in our people
• Providing business leadership
• Optimizing business processes
• Strengthening our link to the
Agency mission
4
-------
OAM Needs to Identify Skill and Full-Time Equivalent Gaps
Ensuring that the organization has the right people, in the right place, at the right
time is critical to organizational success. EPA's human capital strategy,
"Investing in Our People II, EPA's Strategy for Human Capital, 2004 and
Beyond, " requires EPA offices to identify and address skills/competency gaps and
surpluses.
OAM began a workload analysis in the summer of 2003 to review quantitative
workload among its four operating divisions, standardize metrics, and compare
full-time equivalents usage versus workload processes. However, OAM indicated
that the poor quality of data in its information systems (see below) presented
obstacles to completing its workload analysis. Management also applied several
rounds of subjective weighting to the data to distinguish between simple and
complex work and to recognize work not captured, which also caused delays.
Once OAM performs a work load analysis, it should also perform a workforce
analysis. Specifically, once OAM identifies the skills on hand, it should
determine the skills actually needed, and address any skill gaps that may exist as
well as surpluses. In addition to addressing the skills of contracting officer
positions, which are cross functional positions, the analyses should be extended to
other positions within OAM, such as those related to budgeting, auditing, and
computer operations.
Information Systems Need Improvement
OAM identified its information systems as a major weakness in its fiscal 2002
and 2003 Annual Assurance Letters, and created a business case (written
justification) to replace them. OAM's information systems do not enable
managers to effectively measure, analyze, and improve all aspects of short- and
long-term program performance. The information in OAM's Integrated Contracts
Management System can measure the timeliness, but not quality and cost, of its
services. While OAM is considering an automated "executive dashboard" as a
tool for capturing performance data for operational, management, and budget
monitoring purposes, its usefulness will be limited until the underlying
information systems aggregate cost and quality data against standards.
Results Need to Be Measured
OAM does not have data to measure its progress toward achieving its vision of
being the preferred business partner for all EPA contracts. Agency offices
sometimes use the contracting services of other organizations, such as the General
Services Administration's Federal Supply Schedule, rather than OAM contracts.
OAM managers told us they have no means to determine what portion of EPA's
5
-------
contracting business they have, and until it does so it will not know if it is
achieving its vision of always being EPA's preferred business partner.
OAM also does not receive sufficient feedback from program offices on the
extent to which contracts have contributed to Agency environmental and
operational performance goals. OAM believes its role is to support the programs'
missions by providing contract administration services, while it is the program's
responsibility to determine whether contracts are supporting the Agency's
mission to protect public health and the environment. However, OAM's vision
statement implies shared responsibility, and OAM needs better feedback from
EPA offices on the benefits received from Agency contracts.
Recommendations
We recommend that the Director, Office of Acquisition Management, develop an
action plan with milestones for:
1. Establishing measures for, and a means of measuring progress against,
OAM strategic goals.
2. Completing its workload analysis and performing a workforce analysis.
3. Capturing data needed to measure, analyze, and improve short- and long-
term program performance in achieving its vision and goals, including:
(a) Cost and quality of its contract services;
(b) The extent to which OAM contracting offices are the preferred
business partner for EPA offices; and
(c) Environmental and other benefits received from EPA contracts.
Agency Comments and OIG Evaluation
OARM was in general agreement with the findings and recommendations in the
draft report. The Agency suggested two minor changes to the report, which we
made. Although the Agency generally concurred with all the report
recommendations, OARM explained it faced two serious challenges - staffing
constraints and the replacement of the Agency's automated acquisition system -
to overcome before it could completely implement the recommendations. As a
result of these challenges, OARM, in responding to the draft report, was not
prepared to develop milestones for completing a workforce analysis or for the
third recommendation.
We understand that OAM maintains that staffing shortages and information
system shortcomings make it difficult to identify milestones for completing the
6
-------
necessary actions. Recognizing that OAM may not be able to address these issues
in the short term, we continue to believe that milestones are necessary for OAM
to monitor its progress.
7
-------
Appendix A
OAM Vision Statement
or sp
13
o
LU
<9
\
-------
Appendix B
Agency Response to Draft Report
January 14, 2005
MEMORANDUM
SUBJECT: Draft Audit Report: The Office of Acquisition Management Can Improve
Its Organizational Systems, Project Number 2004-000560
FROM:
David J. O'Connor, Acting Assistant Administrator /s/
for Administration and Resources Management
TO:
Robert Mitchell
Director for Contracts Audit
Office of Inspector General
Thank you for the opportunity to comment on the draft report entitled "The Office of
Acquisition Management Can Improve Its Organizational Systems," dated November 24,
2004. The Office of Acquisition Management (OAM) is very proud of its accomplishments
over the past several years in achieving organizational success and excellence, and we
believe the draft report recognizes these efforts. We are in general agreement with the
findings and recommendations. However, we offer the following comments, in order of
appearance in the draft report.
General
- We believe the report's title may be a bit harsh. We strongly suggest using the word
"strengthen" instead of "improve" as we believe this more accurately reflects the positive
findings of the draft report.
- While we generally agree with the Office of Inspector General's recommendations listed in
the draft report, OAM is facing two very serious challenges that will preclude us from
accomplishing all of the recommended action items in the near-term - staffing constraints,
and the continuing need to replace the Agency's automated acquisition systems. Due to
these two factors, we are not ready at present to develop detailed milestone plans for all of
the recommended actions. In our comments below under Recommendations, we have
separated the actions into short- and long-term goals for OAM.
9
-------
Results of Review
- Throughout the draft report, you state that OAM needs to take additional actions if it is to
achieve its vision of being the preferred business management partner. In fact, OAM may
already be the Agency's preferred business partner. We are currently working on developing
the measures, data and information needed to document this determination. Accordingly, we
recommend that your statement be amended to show that OAM needs to take actions to
determine (or measure) if the vision is being achieved.
- Page 4 - While we agree with the findings under "OAM Needs Better Measures for Goal
Accomplishment," we would like to point out that individual OAM Divisions and Service
Centers track various initiatives and accomplishments directly related to our strategic goals
throughout the year. These accomplishments are annually listed and summarized by strategic
goal in OAM's Federal Managers' Financial Integrity Act (FMFIA) Assurance Letter to the
Office of Administration and Resources Management (OARM), and we believe they
demonstrate our commitment to achieving each goal. We are in the process of establishing
detailed measures for many of these initiatives, so we can centrally track them. OAM
recently took the first step to develop such measures at the OAM managers' training meeting
on November 30 - December 1, 2004. See our comments under Recommendation 1 below.
- Page 5 - We agree with the findings under "OAM Needs to Identify Skill and Full-Time
Equivalent Gaps." See our comments under Recommendation 2 below.
- Page 5 - We strongly agree with the findings under "Information Systems Need
Improvement." As part of the Administration's E-Government strategy (one of the five key
elements in the President's Management Agenda), numerous automation-related initiatives
related to integrating agency operations have been identified that should significantly
improve customer service, as well as the effectiveness and efficiency of the Federal
Government. While we are currently in alignment with e-gov initiatives such as the GSA
Integrated Acquisition Environment (IAE), there is a limit in the ability of the current
acquisition systems to remain in alignment with e-gov. A lack of funding to replace outdated
technology will cause us to fall increasingly behind as e-gov moves along its intended
maturity path. For EPA to remain involved with e-gov initiatives, our current automated
acquisition systems need to be replaced by systems using commercial, off-the-shelf software.
- Page 6 - We generally agree with the findings under "Results Need To Be Measured."
However, we are uncertain if accurate data on EPA contract actions not processed by OAM
is available in any Agency system or source. We will work with the Office of the Chief
Financial Officer (OCFO) to attempt to identify and collect such data.
Additionally, while we agree that feedback from programs on how their contracts help
support EPA's mission would perhaps give us a better picture of OAM's effectiveness, we
question whether the benefits received would outweigh the significant effort needed to
collect such information. OAM would also have to take such feedback at face value, as it
would be very difficult to verify the accuracy of such information.
10
-------
Recommendations
RECOMMENDATION 1 - That the OAM Director develop an action plan with milestones
for establishing measures for, and a means of measuring progress against, OAM strategic
goals.
OAM agrees with this recommendation, and such a plan can be developed shortly. OAM
managers received 2 days of training in strategic planning, performance measurement, and
organizational alignment on November 30 to December 1, 2004. At this training session,
preliminary OAM performance objectives and measures were established and discussed. We
expect to develop an action plan with prioritized milestones by the end of March 2005.
RECOMMENDATION 2 - That the OAM Director develop an action plan with milestones
for completing its workload analysis and performing a workforce analysis.
OAM agrees with this recommendation. However, while developing a plan for completing
the workload analysis is something we can do in the short-term, we cannot presently commit
to a milestone plan for the workforce analysis. Performing a workforce analysis is, at this
point, a long-term project. We will develop an action plan and milestones for the workload
analysis by the end of March 2005.
RECOMMENDATION 3 - That the OAM Director develop an action plan with milestones
for capturing data needed to measure, analyze, and improve short- and long-term program
performance in achieving its vision and goals, including: (a) cost and quality of its contract
services; (b) the extent to which OAM contracting offices are the preferred business partner
for EPA offices; and (c) environmental and other benefits received from EPA contracts.
OAM generally agrees with this recommendation, but these are long-term goals. Also, we
believe it may not be physically possible to accurately identify and collect some of this data
using currently available Agency systems and sources. OAM will work with OCFO and
programs to try to determine how to capture this data in the most efficient and effective
manner. We presently cannot commit to developing a milestone plan for these long-term
actions.
We appreciate the opportunity to provide comments on the draft report. While OAM
strongly supports the need to effectively measure its performance, we cannot overemphasize
that replacing the Agency's automated procurement systems is needed to accurately capture a
significant share of the required information. If you have any questions or comments, please
call Judy Davis, Director, Office of Acquisition Management, at 564-4310, or John Oliver,
Policy Training and Oversight Division, at 564-4399.
11
-------
Appendix C
Distribution
EPA Headquarters
Office of the Administrator
Acting Assistant Administrator, Office of Administration and Resources
Management (3101 A)
Director, Office of Acquisition Management (3801R)
Agency Followup Official (the CFO) (271 OA)
Agency Audit Followup Coordinator (2724A)
General Counsel, Office of General Counsel (401 OA)
Audit Followup Coordinator, Office of Administrator (1104)
Audit Followup Coordinator, Office of Administration and Resources Management (3102A)
Associate Administrator for Congressional and Intergovernmental Relations (1301 A)
Associate Administrator for Public Affairs (1101 A)
Inspector General (2410)
EPA Regions
Region Administrators (1-10)
Audit Followup Coordinators, Regions (1-10)
Office of Federal Procurement Policy
Associate Administrator
12
------- |