U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Evaluation Report
Office of Research and Development
Needs to Improve Its Method of
Measuring Administrative Savings
Report No. 11 -P-0333
July 14, 2011
-------
Report Contributors: John Bishop
Dan Howard
Tiffme Johnson-Davis
Geoff Pierce
Rick Beusse
Abbreviations
AEP Administrative Efficiencies Project
EPA U.S. Environmental Protection Agency
FTE Full-time equivalents
FY Fiscal year
ITIP Information Technology Improvement Project
OIG Office of Inspector General
OMB Office of Management and Budget
OMIS Office of Research and Development Management Information System
OPM Office of Personnel Management
ORD Office of Research and Development
SAB Science Advisory Board
SEE Senior Environmental Employees
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail: OIG Hotline@epa.gov write: EPA Inspector General Hotline
phone: 1-888-546-8740 1200 Pennsylvania Avenue NW
fax: 703-347-8330 Mailcode 8431P (Room N-4330)
online: http://www.epa.gov/oig/hotline.htm Washington, DC 20460
-------
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
11-P-0333
July 14, 2011
Catalyst for Improving the Environment
w
hy We Did This Review
We conducted this review to
determine whether the Office
of Research and Development
(ORD) manages its indirect
and overhead costs
appropriately to maximize
available funding for research
and development activities.
Background
The goals of ORD's
Administrative Efficiencies
Project (AEP) and
the Information Technology
Improvement Project (ITIP),
which are two separate
initiatives, include reducing
costs by improving efficiency
and effectiveness. In a 2006
draft report, ORD estimated
that the AEP would save up to
$13 million in administrative
staffing costs annually when
fully implemented in 2012,
and that the ITIP saved
$2 million in 2007.
For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202)566-2391.
The full report is at:
www.epa.qov/oiq/reports/2011/
20110714-11-P-0333.pdf
Office of Research and Development
Needs to Improve Its Method of Measuring
Administrative Savings
What We Found
ORD's efforts to reduce its administrative costs are noteworthy, but ORD needs to
improve its measurement mechanism for assessing the effectiveness of its
initiatives to reduce administrative costs. ORD used a detailed methodology for
the two surveys it conducted during 2005-2010, which provided a manager's
perspective of the amount of time staff spent on administrative duties. However,
we identified some concerns with ORD's mechanism for assessing its initiatives.
Only two surveys have been completed in 5 years, and these surveys only obtained
the manager's perspective on administrative costs and did not obtain data directly
from individual employees, including staff whose time was spent on
administrative activities. Also, the surveys only considered a select number of
ORD staff rather than all ORD staff. Further, ORD used more detailed definitions
for administrative functions for the second of the two surveys, which may have
impacted the comparability of results between the two surveys.
More frequent collection of data and additional data collected directly from staff
related to what they are working on would better measure the effectiveness of
ORD's efforts to reduce costs. Also, by reducing the time elapsed between
surveys, ORD could identify and address issues that may impact ORD in meeting
its goal of reducing administrative costs and, in turn, maximize available funding
for research and development activities.
What We Recommend
We recommend that ORD establish a more timely and accurate system to measure
its effective use of resources and to allow ORD to better manage its initiatives to
reduce administrative costs. ORD generally agreed with our recommendation and
is taking action to implement the recommendation. ORD's planned actions and
timeline meet the intent of our recommendation.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
FROM:
TO:
July 14, 2011
Office of Research and Development Needs to Improve its Method of
Measuring Administrative Savings
Report No. ll-P-0333
Arthur A. Elkins, Jr.
Inspector General
Paul Anastas
Assistant Administrator for Research and Development
This is our report on the subject evaluation conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
the problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
The estimated direct labor and travel costs for this report are $250,290.
Action Required
In accordance with EPA Manual 2750, we are closing this report upon issuance in our tracking
system since your response to the draft report provided corrective actions that meet the intent of
the draft report's recommendation. In accordance with OIG policy, we will periodically follow
up to determine how well the Agency's corrective actions have addressed the report's
recommendations. We have no objections to the further release of this report to the public. We
will post this report to our website at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact Wade Najjum,
Assistant Inspector General for Program Evaluation, at (202) 566-0832 or
najjum.wade@epa.gov: or Rick Beusse at (919) 541-5747 or beusse.rick@epa.gov.
-------
Office of Research and Development Needs to 11 -P-0333
Improve Its Method of Measuring Administrative Savings
Table of Contents
Purpose 1
Background 1
Noteworthy Achievements 4
Scope and Methodology 4
Results of Review 5
Conclusions 9
Recommendation 9
Agency Comments and OIG Evaluation 9
Status of Recommendations and Potential Monetary Benefits 11
Appendices
A Agency Response to Draft Report 12
B OIG Evaluation of Agency Response 17
C Distribution 18
-------
Purpose
The purpose of our review was to determine whether the U.S. Environmental
Protection Agency's (EPA's) Office of Research and Development (ORD)
manages its indirect and overhead costs appropriately to maximize available
funding for research and development activities.
Background
ORD's Organizational Structure
ORD is organized into three national laboratories, four national centers, and six
offices located in 14 facilities around the country and in Washington, DC. ORD
also operates 12 national research programs each headed by a National Program
Director. The national research programs provide the science to support EPA's
goals as outlined in EPA's strategic plan and ORD's strategic plan.
ORD is in the process of realigning its research and development activities in an
effort to improve efficiency and effectiveness. In recent years, ORD created two new
offices, the Office of Science Information Management in 2008 and the Office of
Administrative and Research Support in 2009, and plans to consolidate its 12 national
research programs into 6 national programs by October 2011.
Allocation of EPA's Science and Technology Resources
EPA's fiscal year (FY) 2010 budget for Science and Technology totaled
$846 million. These funds were divided among ORD and other EPA offices. A
substantial portion (over 30 percent) of EPA's Science and Technology budget
goes to other EPA offices, including the Office of Air and Radiation, Office of
Administration and Resources Management, and Office of Water. ORD does not
have control over spending of the Science and Technology funds allotted to other
offices.
ORD's budget for FY 2010 totaled $594.7 million. Ninety-five percent of this
amount comes from EPA's Science and Technology appropriation. About
5 percent, or $27.8 million, comes from other appropriations, largely Superfund.
ORD allocated approximately $291 million, or 49 percent of its budget, for
grants, cooperative agreements, contracts, and interagency agreements. Further,
ORD was allocated funding for about 1,911 full-time equivalents (FTEs)1 in
FY 2010. ORD's personnel compensation and benefits funding is about
$250 million, or 42 percent, of ORD's budget, which pays for these 1,911 FTEs.
1 FTEs are calculated based on the number of full-time and part-time employees in an organization. FTEs represent
these workers as a comparable number of full-time employees.
11-P-0333
-------
Diminishing Resources for Research and Development
The Science Advisory Board (SAB) has advised the EPA Administrator on EPA's
strategic research directions and budgets for many years. In its review of EPA's
FY 2010 research budget, SAB identified concerns that rising personnel costs are
diminishing the actual research that EPA can support. SAB stated:
As personnel costs have increased each year and other categories
of expenditures have not, the funds that are available to support
extramural research, as well as those available to fund procurement
of the things that are needed to conduct intramural research,
diminish. Without significant overall research budget increases, the
"wedging" effect of personnel costs diminishes the actual research
that can be supported by EPA. This dynamic is increasingly
jeopardizing the strength and balance of ORD's combined
intramural and extramural research program.
The EPA Administrator responded to this comment by SAB on September 14,
2009:
ORD's increased personnel expenses reflect the Agency's policy
of maintaining personnel compensation and benefits to keep pace
with the rising cost of living. The SAB is correct in its observation
that rising personnel costs, without attendant increases in ORD's
overall budget, can have a wedging effect on the resources that are
available to conduct both extramural and intramural research. We
are aware of this challenge and will continue to pursue creative
solutions.
Administrative Efficiencies Project and Information Technology
Improvement Project Initiatives
ORD began the Administrative Efficiencies Project (AEP) in 2005 and the
Information Technology Improvement Project (ITIP) in 2006. While two separate
initiatives, the goals of both initiatives include reducing costs by improving the
efficiency and effectiveness of ORD's administrative and information
technology/information management activities. For the AEP, ORD estimated that up
to $13 million, or 24 percent, of overall administrative service staffing costs could be
saved annually under its recommended realignment2 once fully implemented in 2012.
ORD stated in its response to our draft report that it now plans to monitor progress
through December 2015 or until targets have been reached. ORD also estimated costs
savings for the ITIP.
Draft Report: Environmental Protection Agency's Office of Research & Development, Administrative Efficiencies Project
(AEP) (Formerly Goal 1), November 1, 2006.
11-P-0333
-------
According to ORD, based on its analysis in the 2008 AEP Cost Savings Report, the AEP
has reduced administrative service delivery expenses by about $7.9 million annually, or
11.5 percent, since its start in 2005. This savings is about 48 percent of ORD's long-term
goal of 24 percent in savings for 2012. ORD stated in its 2008 AEP Cost Savings Report
that the overall administrative workforce was reduced by 73 work years3 from 2005 to
2008. According to this report, the federal workforce showed the largest decline, with a
reduction of 63 FTEs. Senior Environmental Employees (SEEs) and contractor staffing
decreased by 10 work years, according to the report.
ORD also estimated substantial cost savings resulting from the ITIP. For example,
the Office of Management and Budget's (OMB's) 2007 Program Assessment
Rating Tool review of ORD's Ecological Research Program stated that the ITIP
achieved a savings of $2 million in FY 2007 by investing in a more powerful
shared platform for high-performance computing and reducing storage costs.
Further, according to ORD, the Total Cost of Ownership Initiative, a predecessor
to the ITIP, created a standard desktop platform, established a centralized call
center, and consolidated aspects of ORD's core computer infrastructure and
maintenance, to achieve an annual savings of $2 million beginning in FY 2005.
Consolidation of ORD Research Programs
EPA's Assistant Administrator for Research and Development announced in
September 2010 that ORD's latest realignment would consolidate its research and
development programs into 6 national programs. ORD stated that this realignment
is to strategically align its research and technical support into broad program areas
that cut across national labs, centers, and offices, to allow ORD to work across
projects and disciplines. As part of ORD's Path Forward initiative,4 one of ORD's
goals of this realignment is to increase its mission effectiveness. ORD estimated
that this realignment would be completed by October 2011.
EPA's Accounting for Costs
ORD's labs, centers, and offices are responsible for determining the appropriate
direct charges for personnel costs at the national program level in accordance with
Agency policy.5 ORD does not track costs by individual research projects. ORD
stated that, according to this policy, it accounts for its personnel costs at the
national program level.
3 In its 2008 AEP Cost Savings Report, ORD stated that it used the term "work year" to indicate the total workforce
population—federal, SEEs, and on-site contractors. "FTE" indicates full-time equivalent for federal employees only.
4 EPA's Assistant Administrator for Research and Development developed a set of principles to guide ORD's work
going forward, to provide the scientific and technological basis for advancing EPA's mission to protect human
health and the environment.
5 Office of the Chief Financial Officer's PRC Policy & Procedure Document Number: 27334.04001
11-P-0333
-------
Noteworthy Achievements
ORD has recognized the need to reduce administrative costs and has taken steps to do so
by initiating the AEP and ITIP. Although only about halfway toward its long-term goal,
ORD estimates that its AEP has already reduced its administrative service delivery
expenses by nearly 12 percent since its start in 2005, including reducing ORD's
administrative workforce by 73 work years. ORD also estimated cost savings for the ITIP
of $2 million in 2007.
Scope and Methodology
To determine how ORD manages its indirect and overhead costs, we reviewed
information on ORD's initiatives to reduce its administrative costs,6 particularly
ORD's AEP and ITIP initiatives. We reviewed ORD's intranet site for AEP and
ITIP reports to identify efforts made by ORD to address indirect and overhead
costs. We focused our review on personnel costs, since personnel costs represent
the largest amount of indirect and overhead costs under ORD's control. Further,
we analyzed ORD's budget data for the last 3 years, the current year, and the
proposed 2011 budget.
We reviewed SAB reports that advised the EPA Administrator on EPA's strategic
research directions and budget for recommendations related to properly managing
indirect costs and overhead. We also reviewed pertinent guidance issued by
OMB.
We interviewed representatives from ORD located in Washington, DC, and
Research Triangle Park, North Carolina, and the Office of the Chief Financial
Officer in Washington, DC, to identify policies, procedures, and guidance
documents related to budgeting and accounting for indirect and overhead costs.
We also interviewed management and science representatives from various
offices within ORD to identify efforts made by ORD to address indirect and
overhead costs.
Further, with data obtained from ORD, we calculated the number of employees
for each job series in three occupational groups that included employees with
administrative responsibilities for FYs 2005-2010 to determine whether there
were reductions in administrative employees within these fiscal years. The Office
of Inspector General (OIG) methodology to calculate estimated reductions in
administrative positions differed from the one used by ORD. The OIG calculated
the number of employees for each job series in the following occupational groups:
General Administrative, Clerical and Office Service Group (job series 301-399);
Indirect and overhead costs are generally costs not directly attributable to a single cost objective or activity. For
the AEP, ORD focused on staffing cost related to administrative service delivery at ORD. Examples of indirect or
administrative costs include personnel costs for budgeting, costs for information management activities, and general
management functions.
11-P-0333
-------
Accounting and Budget Group (job series 501-560); and Business and Industry
Group Gob series 1101-1105) for FYs 2005-2010. The OIG considered the job
series in these occupational groups to generally be full-time administrative
positions for purposes of our analysis. Our comparison only included federal
employees and did not address federal employees outside of these three
occupational groups, SEEs, or on-site contractors. ORD's estimated costs savings
for the AEP were largely based on two ORD surveys. The ORD survey covered
more occupational groups and covered SEEs and on-site contractors. Because of
the amount of time that has passed since ORD completed its surveys and the
difficulty in verifying estimates obtained during those time periods, our analysis
did not include a review of the ORD managers' estimates of time ORD employees
spent on administrative functions.
We conducted our evaluation from June 2010 to April 2011 in accordance with
generally accepted government auditing standards. Those standards require that
we plan and perform the evaluation to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our
evaluation objectives. We believe the evidence obtained provides a reasonable
basis for our findings and conclusions based on our evaluation objectives.
Review of Management (Internal) Controls
Generally accepted government auditing standards require that auditors obtain an
understanding of internal controls significant to the audit objectives and consider
whether specific internal control procedures have been properly designed and
placed in operation. We reviewed policies and procedures, including EPA ORD,
and Office of the Chief Financial Officer policies, procedures, and guidance
documents relating to budgeting and properly managing indirect and overhead
costs. We reviewed vulnerability assessments, assurance letters, and related
documents under the Federal Managers' Financial Integrity Act. We also
reviewed OMB's Program Assessment Rating Tool assessments for the 12 ORD
programs reviewed by OMB from 2004 to 2007 for information related to
budgeting, indirect and overhead costs. Our findings pertaining to specific internal
and management controls are discussed below.
Results of Review
ORD needs to improve its mechanism for assessing the effectiveness of its
initiatives to reduce administrative costs. ORD only completed two surveys in
5 years and did not obtain data directly from individual employees, including staff
whose time was spent on administrative activities. The surveys only reported on a
selected number of ORD staff, not all ORD staff. Further, ORD used more
detailed definitions for administrative functions for the second of the two surveys,
which may have impacted the comparability of results between the two surveys.
The effectiveness of ORD's efforts to reduce costs by improving efficiency and
effectiveness could be better measured if data on staff assignments were collected
more frequently and directly from the staff involved. Additional data may also be
11-P-0333
-------
useful, taking into consideration the costs involved with their collection. Also, by
surveying more frequently, ORD could identify and address issues that may
impact ORD in meeting its goal of reducing administrative costs and, in turn,
maximizing available funding for research and development activities.
OMB Circular A-123 states that management controls are the organization,
policies, and procedures used to reasonably ensure that "programs achieve their
intended results"; "resources are used consistent with agency mission"; and
"reliable and timely information is obtained, maintained, reported, and used for
decision making." Without such reliable and timely information, ORD lacks an
adequate measure of how its resources are actually used.
ORD Should Improve Method of Measuring Savings
ORD measured its progress toward achieving targeted savings by comparing
periodic snapshots of ORD administrative service staffing performing the
functional areas identified in the AEP report. Estimated costs savings for the AEP
were largely based on two ORD surveys, one conducted in 2005 and the other in
2008. The surveys were completed by each lab, center, and office Program
Operations Staff Director (approximately 15 managers) for their respective
administrative workforces. According to ORD:
A small team of 5 Program Operations Staff Directors (called the
Goal 1 team) developed a list of functional categories to better
understand the level of resources devoted to providing
administrative and research support services. The data included
support provided by Federal employees, SEE enrollees and
contractors. Each Program Operations Staff Director gathered data
for their respective organization, for review by the Goal 1 team to
ensure consistency and completeness.
While ORD used a detailed survey methodology, and the two surveys ORD
conducted provided the manager's perspective of the amount of time staff spent
on administrative duties, we identified the following concerns:
• ORD has only conducted two surveys in 5 years, and estimated that
the next survey will not be conducted until late in calendar year 2011.
• The staff members whose time was spent on administrative activities
did not complete the survey.
• The surveys only reported on a selected number of ORD staff. For
example, ORD's 2005 survey estimated time for 721 people (including
certain federal, SEE, and contractors). By comparison, ORD reported
having 1,975 employees in FY 2005.
11-P-0333
-------
• ORD used more detailed definitions for administrative functions for
the second of the two surveys, which may have impacted the
comparability of results between the two surveys. For example, ORD
changed the number of administrative functions from 10 in the 2005
survey7 to 14 in the 2008 survey, and changed the number of
administrative subfunctions from 22 in 2005 to 41 in 2008. While this
helped ORD managers by providing the managers with more precise
definitions for administrative functions and subfunctions in the 2008
survey, the change made assessing the effectiveness of its initiatives
more difficult. This difficulty occurred because we could not measure
the extent to which the different definitions impacted the ORD
managers' estimates of time their employees spent on administrative
activities.
ORD reports for the ITIP initiative have acknowledged problems with
determining actual personnel savings of the IT efforts, including stating, "It is
generally not clear who is involved in IT/IM (information technology/information
management) support within ORD." ORD also reported:
Even with mandatory time management multi-million dollar
human resource systems, and federal requirements to categorize
individuals and maintain job descriptions, the only reliable way to
understand who is engaged in IT/IM (information
technology/information management) functions in ORD is to
release a survey to the organization. However, even after this is
performed, there is widespread acknowledgement that the data
received does not accurately portray reality.8
A more timely and accurate measurement mechanism would better enable ORD
to provide reasonable assurance of its effective use of resources in accordance
with the intent of OMB Circular A-123, as well as allow ORD to better assess the
effectiveness of its initiatives to reduce administrative costs.
OIG's Limited Comparison of Three Job Series
We compared the number of ORD administrative employees for each job series in
three occupational groups of ORD employees with administrative responsibilities
for FYs 2005-2010. The reduction in ORD administrative employees from 2005
to 2008 was 22 FTEs, or approximately a 6 percent reduction in administrative
employees. The 2008 AEP report estimated a 14.4 percent reduction in federal
administrative FTEs during this same time period. However, it should be noted
that our comparison only included federal employees and did not cover ORD
Draft Report: Environmental Protection Agency's Office of Research & Development, Administrative Efficiencies Project
(AEP) (Formerly Goal 1), November 1, 2006.
Q
Office of Research and Development, IT Improvement Project Workgroup, Current State Findings,
July 28, 2006.
11-P-0333
-------
employees outside of these three occupational groups, nor did it cover SEEs or
on-site contractors. The ORD survey covered a larger number of occupational
groups and covered SEE employees and on-site contractors. Table 1 shows the
year-to-year changes in each of the three Office of Personnel Management (OPM)
occupational groups that contain job series with administrative responsibilities,
and the cumulative changes for the three series over the last 6 fiscal years.
Table 1: OIG analysis of three ORD occupational groups of employees with administrative
responsibilities, FYs 2005-2010
Fiscal
year
2005
2006
2007
2008
2009
2010
Number of
employees
in 300 job
series a
342
333
329
321
303
308
Number of
employees
in 500 job
series b
35
35
33
30
33
41
Number of
employees
in 1100 job
series °
7
7
7
11
8
6
Total number
of employees
in the three
series
384
375
369
362
344
355
Percent
change
(year to year)
n/a
-2%
-2%
-2%
-5%
+ 3%
Percent change
(cumulative) d
n/a
-2%
-4%
-6%
-10%
-8%
Source: OIG-created analysis of EPA ORD data.
aThe 300 series represents job positions in OPM's General Administrative, Clerical and Office Services Group.
b The 500 series represents job positions in OPM's Accounting and Budget Group.
c The 1100 series represents job positions in OPM's Business and Industry Group.
Minor cumulative differences exist due to rounding.
The OIG methodology provided a trend analysis of the number of ORD
administrative positions for these three categories for each year. Our analysis
showed that the number of positions classified as administrative for the three
occupational groups decreased from 384 to 355 from FY 2005 through FY 2010.
We identified an increase of 11 administrative employees from FY 2009 to
FY 2010. We acknowledge that this increase may not be indicative of the trend
for all ORD personnel performing administrative tasks during these time periods.
However, we believe that tracking the number of ORD administrative employees
annually or more frequently can help ORD better identify potential impediments
to meeting its goal of reducing administrative costs.
Recent ORD Measurement Efforts
ORD managers stated that ORD has taken steps that would allow it to identify
administrative employees and to track the number of employees in administrative
positions. For example, each month EPA's Human Resources Division provides a
report of personnel changes, and ORD updates its organization roster based on
this report. Administrative personnel are identified in the administrative roster. In
December 2010, ORD decided to put the information in the updated organization
rosters in its Office Management Information System Human Resources module
(database). ORD completed a reconciliation of administrative personnel in this
11-P-0333
-------
database and the ORD administrative roster. ORD plans to reconcile its database
and administrative roster every 6 months as part of its quality assurance process.
In response to our draft report, ORD stated that "tagging" (identifying) federal
administrative personnel in its database and reconciling this result with personnel
rosters will continue through December 2015 or until targets have been reached.
There are limitations to both ORD's and OIG's methodologies. While we cannot
verify ORD's estimates of administrative savings, we believe that shorter
intervals between surveys would allow ORD to conduct annual or more frequent
trend analysis that could help it determine whether it is on track to meet its goals.
Further, these trend analyses would allow ORD to more promptly identify issues
affecting progress and allow ORD to make appropriate changes toward goal
accomplishment. For example, our analysis identified an increase in
administrative positions in FY 2010 that may not have been identified under
ORD's measurement methods that were in effect for the time period we reviewed.
There may also be increases in administrative costs, due to an increase in the
number of administrative employees, which should be more promptly disclosed
and addressed. ORD, by more frequently surveying these activities, would
enhance its ability to more quickly identify lapses in progress toward goal
attainment.
Conclusions
ORD does not have sufficient data nor a timely or accurate system for assessing
the effectiveness of its initiatives to reduce administrative personnel costs. Until it
does, ORD will not have the information needed to provide reasonable assurance
that it is managing its administrative personnel costs appropriately so as to
maximize available funding for research and development activities. To assess the
effectiveness of it initiatives, ORD needs to be able to accurately and timely
measure its administrative costs.
Recommendation
We recommend that the Assistant Administrator for Research and Development:
1. Establish a more timely and accurate system to measure its
effective use of resources and to allow ORD to better manage its
initiatives to reduce administrative costs.
Agency Comments and OIG Evaluation
ORD agreed with our finding that the savings from its efforts to reduce
administrative costs could be better measured had the Agency collected data
from staff on a more frequent basis. ORD also agreed with the recommendation.
However, ORD stated that it had several reservations about other findings
included in the report. ORD's complete written response is in Appendix A. We
11-P-0333 <
-------
made changes to the final report based on these comments, as appropriate. Our
evaluation of the Agency's response is in Appendix B.
In response to Recommendation 1, ORD stated that it will "tag" federal
administrative personnel, SEEs, and on-site contractors in its Office of Research
and Development Management Information System (OMIS) and reconcile this
data with personnel rosters on a monthly basis. In addition, ORD's senior
management will meet twice each year to review current status and outline plans
to attain organizational administrative staffing targets. ORD stated that this
process would continue through December 2015 or until targets have been
reached. ORD's planned actions and the timeline for completion of the
recommendation meet the intent of our recommendation. As such, we are closing
this report upon issuance in our tracking system.
11-P-0333 10
-------
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS BENEFITS (in SOOOs)
Planned
Rec. Page Completion Claimed Agreed To
No. No. Subject Status1 Action Official Date Amount Amount
Establish a more timely and accurate system to 0 Assistant Administrator for December
measure its effective use of resources and to allow Research and Development 2011
ORD to better manage its initiatives to reduce
administrative costs.
0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
11-P-0333 11
-------
Appendix A
Agency Response to Draft Report
MEMORANDUM
SUBJECT: Office of Research and Development (ORD) Response ORD Needs to Improve Its
Method Of Measuring Administrative Savings., Project No. OPE-FY10-0018
FROM: Lek G. Kadeli
Deputy Assistant Administrator
Office of Research and Development
TO: Wade Najjum
Assistant Inspector General
Office of Program Evaluation
Thank you for the opportunity to comment on the Office of Inspector General (OIG) draft
audit report, ORD Needs to Improve Its Method Of Measuring Administrative Savings (Project
No. OPE-FY10-0018), dated April 12, 2011. We appreciate the OIG's recognition that "AEP has
already reduced [ORD's] administrative service delivery expenses by nearly 12 percent. "
The ultimate goal of ORD's Administrative Efficiencies Project (AEP) was to achieve
significant efficiencies in delivering administrative support services. We did this by taking steps
to identify ORD-wide organizational alignment options and implementation approaches. We also
developed common business practices, standard operating procedures, and leveraged existing
technology. In addition, we took advantage of industry and government best practices for
administrative support delivery management.
In the "At a Glance" section, your draft report states:
"ORD used a detailed methodology for the two surveys it conducted during 2005-2010,
which provided a manager's perspective of the amount of time staff spent on administrative
duties. However, this was not an effective mechanism for accurately assessing its initiatives for
several reasons. First, only two surveys have been completed in 5 years. "
ORD agrees with your finding that the savings could be better measured had we collected
data from staff on a more frequent basis. We also agree with the subsequent recommendation
that addresses this particular finding. However, we have several reservations about the other
11-P-0333 12
-------
findings you included in your report, some of which we shared in response to your preliminary
outline of assignment results.
As noted in your report, OIG's methodology to calculate estimated reductions in
administrative positions was limited to "... federal employees and did not address federal
employees outside of these three occupational groups. " You acknowledge that "the ORD survey
covered more occupational groups and covered SEEs andon-site contractors. " However, there
is no explanation in the report on why ORD used a more comprehensive methodology as
opposed to limiting our methodology to certain occupational groups. Your staff did not audit our
methodology or cost savings data but your report gives the impression that ORD's methodology
and cost savings data was flawed. Although we appreciate the additional insight that OIG
provided, we believe a better understanding of our comprehensive methodology would have
influenced your approach.
ORD's review evaluated each position and determined what percentage of their time was
spent on administrative duties. This evaluation looked at employees, SEE enrollees, as well as
contractors. We believed that, due to inefficiencies that existed at the time, scientists and other
staff were performing some administrative activities even though they were not classified in
certain occupational groups. We wanted this to be a comprehensive review, so we did not limit
our focus to specific occupational groups. We believed that the approach we used gave an
accurate assessment of the total cost of ownership as it relates to our administrative support
footprint.
See Appendix B, Note 1, for OIG Response.
In your draft report, you state "Second, the surveys only obtained the manager's
perspective on administrative costs, and did not obtain data directly from individual employees,
including staff whose time was spent on administrative activities.'" We believed that a manager's
perspective would add more value to the process. A key responsibility of a manager is to oversee
their employees' work responsibilities. When we needed more information on what positions
were performing administrative duties, we believed managers would provide a more accurate
assessment of the administrative work being performed by their employees. Many of the
managers reached out to their employees in order to provide the information that was requested
in the 2005 and 2008 survey.
See Appendix B, Note 2, for OIG Response.
In your draft report, you further state "Third, the surveys only considered a select number
of ORD staff, rather than all ORD staff'' Our decision to not survey all employees was
11-P-0333 13
-------
intentional and calculated. We did not believe a full survey of all of the ORD staff was
appropriate, especially since many of the positions were not within scope of the AEP.
See Appendix B, Note 3, for OIG Response.
In your draft report, you further state "Fourth, ORD used different definitions for
administrative functions for the two surveys". We would like to restate that ORD did not use
different definitions of for administrative functions for the two surveys. We used more
specificity, not different definitions. The goal of the follow-up survey was to do more of a deep
dive in order to develop a new baseline. The definitions were clarified to ensure that we were
getting the detailed information needed in our re-baseline effort. Our clarifications were intended
to limit the interpretation needed in understanding the definitions of administrative functions.
See Appendix B, Note 4, for OIG Response.
In addition to the comments above, we would also like to offer the following technical
clarifications:
• On page 5, you reference contractors as employees. Contractors that work in ORD
are employees of the contracted company, not employees of the U.S.
Environmental Protection Agency.
See Appendix B, Note 5, for OIG Response.
• On Page 1, you mentioned that ORD has two offices. We would like to clarify
that ORD has six offices including:
o IOAA: Immediate Office of the Assistant Administrator
o OARS: Office of Administrative and Research Support
o ORMA: Office of Resources Management and Administration
o OSA: Office of the Science Advisor
o OSIM: Office of Science Information Management
o OSP: Office of Science Policy
See Appendix B, Note 6, for OIG Response.
• The information technology improvement project (ITIP) and AEP are two
separate initiatives; the report seems to merge the two together which could
mislead the reader.
11-P-0333 14
-------
See Appendix B, Note 7, for OIG Response.
The report does not seem to acknowledge that the AEP stand up is not yet
complete.
See Appendix B, Note 8, for OIG Response.
In response to your recommendation, we have attached a table outlining ORD's
corrective actions and projected completion dates. If you have any questions, please contact
Norman Adkins at (919) 541-0872.
cc: Arthur Elkins
Rick Beusse
Lek Kadeli
Amy Battaglia
Jerry Blancato
11-P-0333 15
-------
Attachment I
ORD Corrective Actions and Projected Completion Dates
Rec
No.
1
OIG Recommendation
Establish a more timely and
accurate system to measure its
effective use of resources and
to allow ORD to better manage
its initiatives to reduce
administrative costs.
Lead
Responsibility
Director, Office
of Administrative
and Research
Support, ORD
ORD Corrective Action
ORD will "tag" federal administrative
personnel, SEEs, and on-site contractors in
ORD's administrative system (OMIS) and
reconciling this data with personnel rosters
on a monthly basis. In addition, ORD's
senior management will meet twice each
year to review current status and outline
plans to attain organizational
administrative staffing targets.
Planned Completion Date
Tagging of federal administrative
personnel in OMIS and reconciling with
personnel rosters will continue through
December 2015 or until targets have
been reached. The first ORD senior
management meeting will be completed
by December 201 1 and will continue
twice a year thereafter through 2015 or
until targets have been reached.
11-P-0333
16
-------
Appendix B
OIG Evaluation of Agency Response
Note 1: As noted on page 5, because of the amount of time that had passed since ORD
completed its surveys and the difficulty in verifying estimates obtained, our analysis did
not include a review of ORD managers' estimates of employee time spent on
administrative functions. Our analysis was intended to identify whether assessing
yearly trends in ORD's administrative personnel costs may reveal information not
captured by ORD's two surveys.
Note 2: We agree that the manager's perspective is important, but we also believe that the
effectiveness of ORD's efforts to reduce costs would be better measured if data on staff
assignments were also collected more frequently and directly from the staff involved.
Note 3: The intent of this statement was to indicate that since the surveys only reported on a
selected number of ORD staff, some administrative costs might not be captured in the
survey.
Note 4: We acknowledged on page 7 that this provided the managers with more precise
definitions for administrative functions and subfunctions in the 2008 survey. We
revised the final report to say that ORD used more detailed definitions for
administrative functions for the second of the two surveys, which may have impacted
the comparability of results between the two surveys.
Note 5: We revised the final report to make clear that we were talking about contractors and not
EPA employees.
Note 6: ORD's recommended revision made on page 1 in the final report. However, as of June
27, 2011, ORD's website at http://www.epa.gov/aboutepa/ord.html continued to state:
"ORD is organized into three national laboratories, four national centers, and two
offices located in 14 facilities around the country and in Washington, D.C." We have
requested that ORD update its public website.
Note 7: We added clarifying text to the final report to make it clear that these are two separate
initiatives. Text was added to the At a Glance and on page 2 under "Administrative
Efficiencies Project and Information Technology Improvement Project Initiatives.
Note 8: We acknowledged in several places in the draft report that the AEP project is not yet
complete. For example, on page 2, we provided ORD's estimated savings for the AEP
once fully implemented in 2012, and on page 3 we compared the estimated savings in the
2008 report to ORD's long-term goal of 24 percent in savings for 2012. Additionally, we
updated this section by adding that ORD plans to monitor progress through December
2015 or until targets have been reached. We also stated on page 7 that ORD estimated that
the next survey will not be conducted until late in calendar year 2011.
11-P-0333 17
-------
Appendix C
Distribution
Office of the Administrator
Assistant Administrator for Research and Development
Agency Followup Official (the CFO)
Agency Followup Coordinator
General Counsel
Deputy Assistant Administrator for Management, Office of Research and Development
Deputy Assistant Administrator for Science, Office of Research and Development
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Information
Audit Followup Coordinator, Office of Research and Development
11-P-0333 18
------- |