EPA-350-R-03-004
March 2003
OFFICE OF INSPECTOR GENERAL
Ca talyst for Improving the Environment
Annual Performance Report
Fiscal Year 2002
-------
PIG STRATEGIC MAP (FY 2001 - 2005)
OIG Vision
We are catalysts for improving the environment. We contribute
to environmental quality, human health, and good government through
problem prevention and cooperative solutions, and by striving for
continuous improvement in EPA's management and our own office.
Mission
Promote economy, effectiveness and efficiency within the agency
Prevent and detect fraud, waste, and abuse in agency programs and
Business Line Goals
1. Contribute to
improved
environmental quality
and human health
2. Improve EPA'
management and
program
operations
Corporate Goals
3. Produce timely
quality and cost
effective products
and services that
meet customer needs
4. Enhance
diversity,
innovation,
teamwork and
competencies
Objectives
• Influence significant
programmatic changes to
legislation, regulations, policy,
processes and practices that
have a positive impact on the
environment and human health
• Identify and recommend
solutions reducing the highest
environmental risks
• Identify best practices in
EPA, states and other Federal
agencies that have directly
contributed to improved
environmental quality and
human health impacts
• Identify opportunities
for improving economy,
efficiency and
accountability in EPA
programs and operations
• Improve the integrity of
EPA programs and
operations by identifying
actual and potential
vulnerabilities for fraud and
risk reduction
• Help EPA resolve its
major management
challenges
• Provide the right
products, at the right time,
to the right customers, at
the right cost
• Build infrastructure,
relationships and
partnerships to leverage
change
• Increase professional
image and demand for
products and services
• Improve organizational
systems and business
processes
• Increase recognition
for creativity, innovation
and teamwork
• Promote continuous
learning and demonstrated
competencies in EPA
programs, professional
skills, technical skills and
leadership skills
Program Audits
and Evaluations
Air
Water
Land
Cross Media
Good Government
Financial/Systems
Audits
Systems
Financial Statement
Contract
Assistance Agreements
Computer Security
Fact Finding
A dvisory/A nalysis
Legislation & Regulation
Review
GPRA Implementation
Control Assessment
Fraud Prevention
President's Council on
Integrity and Efficiency
Investigations
Program Integrity
Contract/Procurement
Employee Integrity
Assistance Agreements
Hotline
Computer Forensics
-------
\
a UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
JT WASHINGTON, DC 20460
THE INSPECTOR GENERAL
March 31,2003
FOREWORD
I am pleased to present the EPA OIG's second Annual Performance Report. This Report, in
summary and statistical format, presents progress toward the attainment of our FY 2002 Performance
Targets and Strategic Goals through FY 2005 as described in our Strategic Plan. Where applicable it
compares FY 2002 results with those of FY 2001 and presents cumulative totals in relation to our FY
2005 Strategic Performance Targets.
The net result of this report is to present a "Balanced Scorecard" of performance, progress, results,
activities, investments and challenges that tell a complete story of OIG performance, accountability
and opportunities for improvement. The Balanced Scorecard approach demonstrates a combination
of outputs, activities, outcomes, financial, resource use and customer quality /value ratings arrayed as
leading, current and lagging measures. This presents a systematic linkage of inputs and outputs for
comparative return on investment.
Based on the performance measurement and results from the first OIG Annual Performance Report
for FY 2001, we made significant improvements in the definitions and application of OIG- wide
performance measurement. We are also continuing to learn how to apply both a systems approach
and a culture of "starting with the end in mind" to interpret the implications of our influence and
results in attaining Goal 1 : Contributing to improved environmental quality and human health. Since
there is usually a significant time lag in recognizing our subsequent influence on most environmental
and human health impacts and behaviors, accounting for these results will require a dedicated
followup effort.
Beginning the OIG's third year of measuring performance based on our Strategic Plan, we are
continuing to research new and better ways to measure, collect, and report data on the results of our
operations. The information in this report will be used to help direct and plan future application of
resources for greater results.
L
Nikki L. Tinsley (}
-------
-------
EPA Office of Inspector General Annual Performance Report, Fiscal 2002
TABLE OF CONTENTS
Page
Number
Introduction 1
Summary of Annual Results 3
Results Compared to Goal Targets 4
Balanced Scorecard of OIG Results for FY 2002 5
Goal 1 FY 2002 and Cumulative Results Compared to Targets by Objective 6
Examples of Results by Objective 7
Goal 2 FY 2002 and Cumulative Results Compared to Targets by Objective 9
Examples of Results by Objective 10
Goal 3 FY 2002 and Cumulative Results Compared to Targets by Objective 12
Examples of Results by Objective 13
Goal 4 FY 2002 and Cumulative Results Compared to Targets by Objective 14
Examples of Results by Objective 15
OIG Operational Activity and Audit Resolution 16
Customer Service Results 1^
90
Financial and Resource Management ^
Looking to the Future: New FY 2003 -2005 Annual Targets 21
22
Appendix 1 - FY 2002 Report on Management Controls
Appendix 2 - Historical Perspective & Summary of FY2002 EPA Management Challenges 27
Appendix 3 - Definitions of FY 2002 Performance Measures 28
-------
Introduction
This is the EPA OIG's second comprehensive Annual Performance Report. It provides a
statistical summary of key performance results for fiscal 2002, a Balanced Scorecard of
performance results, and progress assessment of OIG performance results and measures
against fiscal 2002 Annual Performance Targets. In compliance with the Government
Performance and Results Act (GPRA), this report describes examples of specific outputs
and outcomes for each the OIG Strategic Goals, fiscal 2002 OIG financial statements,
Management Challenges from the OIG Federal Managers' Financial Integrity Act
(FMFIA) review, and revised annual performance targets for FY 2002 - FY 2005. This
report also provides additional operational performance measures and indicators about
OIG customer quality and value survey results, and advisory assistance to the Agency
and other external governmental organizations. We believe that this information
accounts for our use of resources and our performance in a way that will be used to
inform and inspire improved operations.
The information presented in this report is for OIG management use, and to provide
public accountability for the application of OIG resources. The results presented are
accumulated from the data entered into an OIG Performance Results and Measurement
System and the IGOR (Inspector General Operations Reporting System), which we are
continuing to refine.
It is from the application of these measures and technology that we will identify needed
data quality assurances, accountability controls for reliable management information and
opportunities for better data integration. This report is not intended to duplicate
descriptions of significant audits, evaluations, investigations and other organizational
activity presented in EPA OIG Semiannual Reports to Congress. We suggest that the
reader also refer to the Semiannual Reports for FY 2002 available electronically from
HTTP://WWW.EPA.GOV/OIG
-------
Environment
Impact
Linking Our Work to Outcomes
and Impacts
All of our work is planned based on the
anticipated value toward influencing resolution
of the Agency's major management challenges,
reducing risk, improving practices, program
operations and saving tax payer dollars, leading
to positive environmental impacts and the
attainment of EPA's Strategic Goals.
Creating a Nexus or Link Between OIG Products/
Services and Environmental Impacts/Goals
Our work is also planned and aligned with our Strategic Goals, and our work projects represent a
balanced approach contributing to environmental impact.
ecommendatio
Advk
Agency Action
This chart demonstrates
the distribution of OIG
project results by
Strategic Goal
Goal 3 &
13.4%
Goal
61.2%
GoaM
25.4%
Performance Presented In A Hierarchy of Related Measures
The Performance Results in this report are presented in hierarchy, starting with several summaries
and a Balanced Scorecard, followed by detailed descriptions and measures and examples of
results by Goal for this fiscal year, and as cumulative totals toward the accomplishment of the OIG
Strategic Plan 2002-2005.
The Balanced Scorecard displays results by Goal as leading, current and lagging indicators or
measures. It also presents related performance information on customer survey results, financial
accountability, human capital, product timeliness, innovation and other OIG operational activity,
together providing a Balanced picture of OIG investments and returns.
-------
Summary of OIG Results for FY 2002 - Highlights
Performance
Results
(highlights)
Goal 1. Contribute to Improved Environmental Quality and Human Health
* 15 Agency/delegated entity actions/decisions/changes/improvements/best practices
implemented or certified/validated influencing environmental impacts
> 16 Environmental risks reduced or eliminated, or certification of integrity
* 115 Environmental Recommendations, Risks, Best Practices Identified
Goal 2. Improve EPA's Management and Program Operations
> $ 55.2 million in potential costs, savings, fines and recoveries
> 80 Criminal, civil, or administrative actions
> 56 Management practice/policy changes/Best Practices implemented or Certification for action
Goal 3. Produce Timely. Cost Effective Products & Services Meeting Customer
Needs
> 96 Collaborative efforts with other agencies
* 46% OIG work is required or requested
* 29 Requests to testify/formal external presentations
Goal 4. Enhance Diversity. Innovation. Teamwork and Competencies in OIG
* 70% of products developed electronically, 48% Available electronically
* 98% Staff diversity in parity with civilian workforce
> 42 Innovative techniques/processes implemented by OIG
Operational
Activity
50 Reports issued and managed by the OIG
186 Reports performed by another Federal Agency (e.g. Defense Contract Audit Agency)
134 Single Audit Act Reviews
149 Reports resolved
133 Audits with no final action by EPA, over 365 days past due
38 Reports for which no management decision was made within 6 months (as of 9/30)
85 Investigations opened
64 Investigations closed
Customer Service
79% Combined OIG customer service/quality rating
* 76% OIG products/services rating
>• 83% OIG staff service rating
* Highest and lowest rated areas 90% Professional & courteous
85% Builds constructive relationships
72% Timeliness
73% Contributes to EPA Goals
Use of Resources
> $34,543,568 Total Mgt expenditures (FY02 appropriation + FY Olcarryover)
>• $10,695, 040 Superfund (SF) expenditures (FY02 appropriation + FY 01 carryover)
>• $11,767,797 Carryover from FY 01, $12,163,850 Carryover to FY02
*• 264.1 Mgt staff years used (97.5% FY02 budget)
73.3 Super Fund staff years used (79.6% FY02 budget)
Cost and Timeliness of
Products
3ost per staff year:
Salaries
Travel
Admin. Exp
Admin. Cont
Working Cap Fur
MGT (OOOs) SFCOOOsl
$ 97.5 $120.3
$ 7.2 $ 7.8
$ 4.4 $ 3.9
$ 7.3 $ 6.5
id$ 6.2 $ 0
Avg. Time: % Direct Elapsed, Staff on Products
Direct time charged of total available 41.2%
Average direct staff days/Investigation 59.7
Average elapsed months/Investigation 26.4
Average direct staff day s/Audit 335.7
Average elapsed months/Audit 14.7
EPA Management
Challenges
Reported by OIG
Linking mission and management (including cost
accounting & outcome based goals/measures)
Information resource management & data quality
Employee competencies
Oversight/use of assistance agreements
Protecting critical infrastructure from attacks
Air toxics program phase I & II
Working relations with states/partners
Information systems security
Backlog of NPDES permits
Management of boisolids
FMFIA Weakness
in OIG
Human resources strategy (updated)
Information technology (IT) strategy (updated)
Inspector General Operations Reporting System
(updated - IT strategy last year)
Product timeliness and quality (updated -
included under Business Systems last year)
Follow-up on corrective actions (new)
Organizational structure (new)
-------
Summary of Fiscal 2002 Results Against Goal Targets with Supporting Measures
GOAL #1. Contribute to improved environmental quality and human health
Target: 50; Actual: 12; Percent of Target: 24%
Improvements/changes/decisions/actions implemented, contributing to or
influencing positive impacts on environmental quality or human health
Target: 15; Actual: 19; Percent of Target: 127%
Environmental or health risks reduced or eliminated from solutions,
enforcement actions, certifications of integrity or for reliance or action
0 Regulatory changes/decisions
1 EPA policy, directive or process change/decision
0 Examples of environmental improvements/impacts
1 Best environmental practices implemented
16 Environmental risks reduced/elimin;
3 Certifications/validations/verifications
Target: 75; Actual: 115; Percent of Target: 153%
Recommendations, risks or best practices identified that can influence
environmental/health actions & change
48 Recommendations for environmental improvement
15 Environmental best practices identified
"2 Environmental risks identified
GOAL #2. Improve EPA's management, accountability and program operations
Target: 100% (S45.2M); Actual: $55.2 M 122%
Potential dollar return from improved business practices, savings,
questioned costs, recoveries & fines compared to investment in OIG
$15.9M Questioned costs
$19.5M Recommended efficiencies, costs saved/avoided
$19.8M Fines, recoveries, restitutions, collections
Target: 75; Actual: 177; Percent of Target: 236%
Actions reducing or eliminating risk of loss, or improving efficiency and
integrity in management of resources, operations & programs, resolving
Management/FMFIA Challenges from OIG influence
Target: 150; Actual: 269; Percent of Target: 269%
recommendations, best practices, FMFIA or management challenges
identified to promote action for improvement
80 Criminal, civil, administrative actioi
3 2 Certifications/validations/verifications
38 Examples of process/practice changes/decisions to
reduce risk & improve integrity or efficiency
12 Corrective Actions, Management/FMFIA Challenges
15 Best practices implemented
252 Recommendations for management improvement
10 FMFIA/ Management Challenge identified
n Best Practices identified
GOAL #3. Produce timely, quality & cost effective products and services that meet customer needs
Targets: 79% Overall OIG Customer Satisfaction
3 New Federal, 5 New State partnerships
35% OIG work requested/required externally
66% Milestones/plans met
teamwork and competencies in the OIG
Targets: 82% Application of technology
96% Diversity (civilian WF), innovation
20 New innovations implemented in OIG
21 Assignment using muliti-discipline teams
489 Awards of recognition (1.45 awards/employee)
.ctual: 79% Customer survey value/quality rating
96 Collaborative partnerships
4 6 % Requested assignments
66% Milestones met
GOAL #4. Enhance diversity, innovation
Actual:59% Products done electronically (avg)
9 8 % Staff in parity with civilian workforce
42 Innovative techniques/practices in OIG
65 Products by multi-discipline teams
""> Awards of staff recognition
-------
OIG BALANCED SCORECARD OF FY 2002 PERFORMANCE AND OPERATIONS
GOAL 1: Contributes To Improved Environmental Quality and Human Health
,ntal Recoi
.tions (Outp.
48
Recommendations/ Advice for Environmental
Improvement
15 Environmental Best Practices Identified
52 Environmental Risks Identified
ate Outcoi
0 Legislative/Regulatory Environmental Changes
12 EPA Environmental Policy, Directive, Practice/
Process Change or Action Implemented
% of OIG Recommendations/Advice Accepted
jental Results (Outcon.
0 Examples of Environmental or Health
Improvements or Behavior Changes
16 Environmental Risks Reduced or Eliminated
3 Certifications, Verifications, Validations
GOAL 2: Improve EPA's Management, Accountability and Program Operations
Management Recommendations (Outputs) Agency Actions (Intermediate Outcomes)
252 Recommendations/Advice for Management
Improvement
7 Best Management Practices Identified
10 Management Challenges/FMFIA
Weaknesses Identified
• # Referrals for Investigation/New Assignments
$ 15.9 M Questioned Costs
$ 19.5 M Cost Efficiencies/Savings
$ 19.8 M Fines, Penalties, Recoveries, Settlements
80 Criminal, Civil, Admin Actions
6.5 %Costs Acted on by EPA($10.4 of $160.2 M)
15 Best Management Practices Implemented
% & $s of OIG Recommendations or Advice
Accepted/Sustained
gency Management Results (Outcomes)
122 % $ Return on FY02 Investment in OIG
12 Actions Taken (by EPA) Correcting
Management Challenges/ FMFIA Weakness
38 Examples of Practice, Process, Policy Actions
Implemented Improving Economy, Efficiency
or Reducing Risks of Loss or Vulnerability
32 Certifications, Verifications, Validations
11 % EPA GPRA Rating Improvement in FY02
From OIG Involvement (6th to 4th of 24)
GOAL 3: Produce Timely, Quality & Cost Effective Products and Services that Meet Customer Needs
GOAL 4: Enhance Diversity, Innovation, Teamwork, and Competencies—OPERATIONS
Customer, Partner, Stakeholder
Timeliness
Financial/Resources
76 % Customer Value/Quality Rating of Products
83 % Customer Value/Quality Rating of Service
96 Products/Assignments done with partners
29 Requests to Testify/External Presentations
46 % of Work Requested/Required
65 Assignments done by Multi-Discipline Teams
336 Average Staff Days per Completed Audit
14.7 Average Months per Complete Audit
60 Average Staff days/Completed Investigation
26.4 Average Months per Completed Investigation
66 % Project Milestones Met
41.1% Total Time on Direct Time Product Lines
$45.2 M Total Funds Used 01/02
99% 01 Funds Used
74 % 02 Funds Used
$12 M Funds Carried Over to 03
337.4/ 25.6 staff years Used/Not Used
$124.4 K Avg "Loaded" Cost /Staff year
Staffing, Development & Planning
Innovation
Operational Activity
98 % IG Staff in Parity with Civilian Labor Force
100 % Staff Meeting Professional CPE Standard
% Staff Meeting IDP/Strategic Training Targets
140 % Staff Recognition/Number of Awards (475)
93 % Available staff time Used
% of Planned Assignments Started
% of Panned Assignments Completed
42 Innovations Implemented by/within OIG
70 % of Work Completed Electronically
48 % of Work Electronically Accessible
14 PCIE Projects Led/Developed
5 Agency /Regional Work Groups Participation
• = Prospective Measures
50 Reports Performed by EPA OIG
186 Reports Performed by DCAA/134 Single Audits
149 Reports Resolved
85 Investigation Cases Opened
64 Investigation Cases Closed
1,018 Hotline Complaints Opened/821 Closed
48 Legislative/Regulatory Items Reviewed
-------
Results By OIG Strategic Goal, Objective and Specific Measure
GOAL 1. Contribute to improved environmental quality and human health
This section displays the results for Goal 1 against the OIG fiscal 2002 and cumulative
Objective performance targets, and by specific performance measure. Also presented are
selected examples of OIG performance results and a table displaying the distribution of results
by Strategic Area.
Objectives, Annual Results and Cumulative Strategic Totals
• Influence significant programmatic changes to legislation, regulations, policy, processes
and practices that have a positive impact on the environment and human health.
Measures: # Environmental program policy, process, practices, changes/actions impacts
2002 Target 50
Cumulative Target 75
2002 Results 12
Cumulative Results 74
FY2002 Target Met 24%
Cumulative Target Met 99%
Identify and recommend solutions for reducing the highest environmental risks.
Measures: # Environmental risks reduced, eliminated, certification/validation of or for action
2002 Target 15
Cumulative Target 30
2002 Results 19
Cumulative Results 43
FY 2002 Target Met 127%
Cumulative Target Met 143%
Identify recommendations, best practices and risks (including outside EPA) that can directly
contribute to improved environmental quality and human health.
Measures: # Recommendations, best practices, risks identified
2002 Target 75
Cumulative Target 140
2002 Results 115
Cumulative Results 211
FY2002 Target Met 153%
Cumulative Target Met 150%
Goal 1 Results By Measure
FY 2002 Measures of Results and Progress
48
0
0
11
52
16
15
1
0
3
# Recommendations for Environmental Improvement
# Legislative Changes/Decisions
# Regulatory Changes/Decisions
# EPA Policy/Directive/Process Changes/Decision
# Environmental Risks Identified
# Environmental Risks Reduced/Eliminated
# Best Practices Identified
# Best Practices Transferred/Implemented
# Examples of Environmental Improvements/Impacts
# Certifications, Verifications, Validation of/for Action
34
2
3
48
53
24
9
3
6
included with
risks reduced
Cumulative Total
82
2
3
59
105
40
24
4
6
3
-------
Examples of Selected Results Demonstrating Progress On Goal 1
Objective: Influence significant programmatic changes to legislation, regulations, policy,
processes and practices that have a positive impact on the environment and human health.
In a 4/15/02 Region 3 weekly salient, Agency managers stated they reached an "innovative
solution" to correct a permit problem that is highlighted in our report. The solution is useful
to PADEP and also protective of water quality. The Region 3 Office of Watershed's salient
noted that this particular permit issue "gained notoriety" through the OIG report and went
on to discuss their solution. (Water)
Based on OIG recommendations, EPA revised its Federal Facility Agreement to improve
Regional oversight of DOE five-year reviews of remedial actions and site evaluations.
(Waste Management)
Based on OIG recommendations EPA facilitated data sharing with states by issuing:
(1) the Integrated Water Quality Monitoring and Assessment Report Guidance, (2) the
Consolidated Assessment and Listing Methodology, and (3) making 305(b) and water
quality monitoring information available on the WATERS database. EPA will also improve
collaboration with other Federal agencies. The Office of Water has instituted regular
meetings with the Associate Director of USGS. The Office of Water is also working with
the National Park Service to regularly transfer and analyze data in STORET. (Water)
Objective'. Identify and recommend solutions for reducing the highest environmental risks
The risk of contaminated drinking water will be reduced by Region 3 ensuring that all
Region 3 States are appropriately interpreting compliance reporting requirements for
secondary sources. (Water)
Region 3 formally notified States not to use a particular permit format; Region 3 stopped
using the Part A/C permit format; and the enforcement staff reported better relations with
the program staff. (Water)
Commercial Lab employee received 3 year debarment which prohibits her from
employment by a Federal contractor in a position as an agent, representative, or principal on
any Federal procurement or non-procurement transaction. (Enforcement and Compliance
Assistance)
A second Open Market Trading (OMT) violation (large number of invalid OMT credits in
New Jersey), resulted in 372,380 OMT credits (representing about 18,600 tons of air
pollutants) were retired and not used. Note Environmental risk reduced: OIG findings
contributed to the proposed termination of the OMT Program in the State of New Jersey.
The State of New Jersey stated that their OMT Program had failed due to, among other
reasons, "...the program's ostensible clean air benefits being limited by the failure to include
safeguards to ensure that the program would in fact reduce emissions." (Air)
-------
Objective'. Identify recommendations, best practices and risks (including outside EPA)
that can directly contribute to improved environmental quality and human health.
In the six states the Title V audit reviewed, three practices that contributed to the progress
that agencies made in issuing Title V operating permits were: (Air)
1) State agency management support for the Title V program.
2) State agency and industry partnering.
3) Permit writer site visits to facilities.
A Controlling and Abating Combined Sewer Overflows (CSO) report identified a total of 12
best practices. States were found to be at different stages of implementation depending on
the level of priority given and the stage of long-term control plan review/approval. Also,
there were two significant barriers that hampered efforts: funding and siting. In addition,
the audit found states and communities had demonstrated a wide variety of promising best
practices to address these barriers, including: 1) establishing a CSO grant program to help
communities pay for construction of CSO projects in a timely manner:
2) engaging and educating the public regarding CSO issues; 3) promoting understanding
that the CSO Project can result in improved community facilities; 4) ensuring that better
technical practices and approaches can be shared with other CSO communities; and,
5) ensuring that communities work cooperatively with state offices and municipal
departments. (Water)
-------
GOAL 2. Improving EPA's Management and Program Operations
This section displays the results for Goal 2 against the OIG fiscal 2002 and cumulative Objective
performance targets, and by specific performance measure. Also presented are selected examples
of OIG verformance results and a table disvlavins the distribution of results bv Strategic Area.
Objectives
Identify opportunities for improved economy, efficiency, and accountability in EPA
programs and operations
Measures: Questioned Costs, Efficiencies, Savings, Fines, Penalties, Restitution, Settlements
2002 Target $45.2 M
Cumulative Target
$86. 2M
2002 Results $55. 2M
Cumulative Results $12 7. 6M
FY2002 Target Met 122%
Cumulative Target Met 148%
Improve efficiency and integrity of EPA business and program operations by reducing
vulnerabilities for fraud and risk and by influencing better business actions or changes
Measures: Criminal, civil, admin actions; policy, practice, process actions taken, certifications
2002 Target 120
Cumulative Target 235
2002 Results 177
Cumulative Results 351
FY2002 Target Met 147%
Cumulative Target Met 149%
Identify recommendations, best practices, and management challenges that can directly
influence and promote actions for improved business operations
Measures: Recommendations, best practices, management and FMFIA challenges
2002 Target 150
Cumulative Target 150
2002 Results 269
Cumulative Results 436
FY2002 Target Met 179%
Cumulative Target Met 291%
Goal 2 Results By Measure
FY2002 Measures of Results and Progress
$15.9
$19.5
$19.8
80
7
15
38
252
10
11%
12
32
$Questioned Costs
$Recommended Efficiencies, Costs Saved or Avoided
$Fines/Recoveries/Restitutions/Collections
#Criminal/Civil/Administrative Actions
#Best Practices Identified
#Best Practices Implemented
#Examples of Process/Practice Changes/Decisions
#Recommendations for Management Improvement
#New/Existing FMFIA/Management Challenges reported
#EPA Improvement In GPRA Rating
#Corrective Actions on Management/FMFIA Challenges
#Certifications/Verifications/Validations
FY2001 Cumulative Totals
$35 M
$32. 1M
$ 5.2M
98
17
*
63
148
10
21%
4
9
$50.9M
S51.6M
$25 M
178
24
15 Included in Practice Change
101
400
20
32% (4th from 11th of 24)
16
41
-------
Examples of Selected Results Demonstrating Progress on Goals 2
Objective:
savings
A review of grants management in a Tribal Assistance Program identified $1.3 million in
questioned costs. (Assistance Agreements)
Region 7 Grants Proactive - A grant recipient received over $2 million to develop and
maintain an environmental website. All costs were questioned because recipient did not
have financial management system according to Federal requirements. EPA should recoup
all ineligible and unsupported costs. (Assistance Agreements)
A contractor issued a credit for $2,590.56 for hours billed to EPA for direct labor charged to a
contract. (Contractor)
An environmental lab agreed to pay the US Government $8,741,000 plus interest in return for
releasing them from any civil or administrative monetary claims. The terms of the civil
settlement are contained in a settlement agreement between the lab and the US Government.
An additional criminal settlement totaled $9 million. (Contractor/Scientific Research)
An EPA assistance recipient was awarded a total of 23 contracts totaling over $1.3 million
under its assistance agreement. Twenty of these contracts were awarded sole source. The
grantee did not have adequate justification to support the award of the sole source contracts.
Also, the grantee's procurement practices did not meet federal requirements. As a result,
$1,301,365 is not eligible for Federal reimbursement. (Assistance Agreements)
On May 9, 2002 an individual was sentenced to 2 years probation, 50 hours community
service, and ordered to pay $1,243.00 in fines and restitution in connection to his plea of
guilty to violating 4 counts of Title 18 of the PA Consolidated Statutes Annotated, 1 count of
Title 35 of the PA Statutes, and 3 counts of Title 30 of the PA Clean Streams Act.
(Contracts/Water)
Quality of DOCKET Data assignment identified a $1.3 mil recommended efficiency/cost
savings. (Enforcement and Compliance)
A contractor issued a credit for $2,590.56 for hours billed to EPA for direct labor charged to a
contract. (Contracts)
The Lake Wallenpaupack Watershed Management District did not follow Federal procurement
procedures when using EPA funds to hire an engineering firm. The District, based in Paupack,
Pennsylvania, received $2.2 million in assistance agreement funds from EPA Region 3 to
perform various watershed management tasks. The $2.2 million included funds to hire an
engineering firm. However, the OIG found that the District did not follow federal procurement
procedures regarding conflicts of interest and competition when it awarded a $547,000 contract
to an engineering firm. The District awarded a second contract to the same firm for $360,000
(for a total of $907,000) also without following federal procurement procedures. Further, the
District billed EPA for $71,000 in additional work done by the engineering firm that was not
part of the scope of work of either of the two contracts. (Contacts/Water)
10
-------
Objective'. Improve efficiency and integrity of EPA business and program operations by reduc
vulnerabilities for fraud and risk and by influencing better business actions or changes
Twenty-One count indictment filed against four defendants for violation of 18 USC 371
(conspiracy); 18 USC 666 (theft concerning program receiving federal funds); 18 USC 1341
(mail fraud); 18 USC 1343 (wire fraud); 18 USC 1341,1346 (mail fraud, depriving the public of
the right of honest services of a public official).
A contractor employee was terminated as the result of the OIG investigation that determined
the employee misused an EPA computer to access the Internet. (Computer Security)
Two criminal complaints were filed against two individuals regarding tampering with or
fabricating physical evidence. These charges were brought in the Sate of Pennsylvania.
Computer Forensics identified a threat against the Administrator, then based on results referred
the matter to EPA Office of Criminal Enforcement, Forensics and Training (OFEFT) which
successfully located the source in Turkey (where OIG had identified his location) and were
working with local law enforcement to deal with the issue. (Computer Security)
Objective: Id
directly influence and promote actions for improved business operations
The OIG identified seven recommendations for management improvement. Recommendations
caused Agency managers to make practice/changes when approving NPDES permits in
Pennsylvania. Also referred a situation to the OIG Office of Investigation. (Water)
The OIG recommended that the EPA Region 6 Administrator work with Louisiana Department
of Environmental Quality (LDEQ) to make the public participation process for air permits more
effective, with emphasis on such areas as records completeness and accessibility, clarification
of roles, and improving outreach. Report also recommended that the Region 6 Administrator
require staff to review public participation issues, define responsibilities within the Region,
perform a thorough on-site review of LDEQ's air permits program, establish a tracking
mechanism for permits reviewed, and review the required number of LDEQ's proposed Title V
permits prior to LDEQ issuance of final permits. (Air)
The OIG identified 10 recommendations for management improvement. California needs to
improve the overall management of it's $1.5 billion loan program. The control environment
and internal control systems are not adequate, and the state needs to establish an accounting
system that will accurately record all State Revolving Fund (SRF) activity. Also needs
improved management involvement in reporting and accounting, needs to establish
procedures that ensure monthly reconciliations and financial statements are prepared, and an
SRF program manager is also needed. (Financial Management and Cost Accounting)
The OIG recommended that the EPA Annual Performance Report be improved by making
better linkages between a performance results and strategic goals, that outcome performance
measures be better translated into public benefit, that costs be associated with performance,
that the Agency provide cross references to the applicable management challenges, and that
the Agency describe the role of partners such as states and tribes. (Financial Management)
11
-------
GOAL 3: Producing Timely, Quality and Cost Effective Products and
Services that Meet Customers Needs
This section displays the results for Goal 3 against the OIG fiscal 2002 and cumulative
Objective performance targets, and by specific performance measure. Also presented are
selected examples of OIG performance results and a table displaying the distribution of results
by Strategic Area.
Objectives
Provide the right products, at the right time, to the right customers, at the right cost
Measures: Customer value/quality survey ratings for OIG products
2002 Target 76% rating
Cumulative Target 75%
2002 Results 76% rating
Cumulative Results 77%
FY2002 Target Met 100%
Cumulative Target Met 103%
Build infrastructure, relationships, and partnerships to leverage change
Measures: Collaborative efforts with federal/state partners
2002 Target 40 (32 in
FYOl+8)
Cumulative Target 72
2002 Results 96
Cumulative Results 128
FY2002 Target Met 240%
Cumulative Target Met 178%
Increase professional image and demand for products and services
Measures: Work/assignments requested (externally); Milestones/expectations met
2002 Target 35% work
requested
Cumulative Target 33%
2002 Target 66% milestones
met*
2002 Results46% work requested
Cumulative Results 51%
2002 Results 66% milestones met
FY2002 Target Met 131%
Cumulative Target Met 154%
FY2002 Target Met 100%
* Annual Target are a cumulative 10% increase from FY 2001 baseline
Goal 3 Results By Measure
FY2002 Measures of Results and Progress FY2001 Cumulative Total
79%
76%
83%
66%
29
96
46%
% Customer Value/Quality Satisfaction Ratings Overall
% Customer Value/Quality Products
% Customer Value/Quality Staff Service
% Engagement Letter/Project/Budget Milestones Met
# Requests to Testify (Congress/judicial)/Presentations
# Collaborative Efforts, States/Feds
% OIG Work Requested/Required
80%
77.5%
85.3%
60%
4
32
56%
79.5% Avg
77% Avg
84% Avg
63% Avg
33
128
51%
-------
Examples of Selected Results Demonstrating Progress On Goal 3
Objectives: Customer satisfaction with OIG products, services and attitudes. Build
collaborative relationships to leverage change with federal and state partners. Develop
professional image, quality and demand for OIG products and services
This case resulted in the conviction of a subject. It resulted in the illegal discharge of wastewater
being discontinued. OIG had seven collaborative partnerships on this effort. This was a Joint
Case with EPA Criminal Investigation Division, FBI, and Sarasota County. The US Attorney,
Middle District of Florida requested OIG investigative assistance on this matter.
(Contracts/Water/Partnering)
The OIG conducted four cases in the area of computer security with FBI and Defense Criminal
Investigation Service.
The OIG was required to perform General Control work in accordance with the Chief Financial
Officers Act (CFO).
Management Assistance Review (MAR) of Missouri's Fiscal 2000/2001 Performance
Partnership Grant Work Plan and Grant Process was a collaborative effort between the OIG
Central Audit and Evaluation Resource Center and the State of Missouri. MAR report
recommended Region 7 and Missouri work jointly engaging in environmental priority setting
and planning. Recommended piloting a partnership approach by selecting an environmental
priority. (Assistance Agreements)
In April 2002, at the request of OMB and the House Transportation Subcommittee on Water
Resources and the Environment, OIG a presented a briefing on the results of audit, Catalog of
Federal Domestic Assistance (CFDA) number 66.606, "Surveys, Studies, Investigations, and
Special Purpose Grants." (Customer Focus)
The EPA Region 3 Administrator requested that we work with Office of Water to correct
reduced oversight within the NPDES program. The purpose of the audit was to determine
whether NPDES permits in Region 3 were written to ensure water quality protection.
OIG Quarterly Reports of Significant Activity Reports are a combined effort of auditors,
investigators, analysts and editors. It is done completely electronically, but it is printed for
distribution. (Customer Focus)
13
-------
GOAL 4: Enhancing diversity, innovation, teamwork, and competencies
This section displays the results for Goal 4 against the OIG fiscal 2002 and cumulative
Objective performance targets, and by specific performance measure. Also presented are
selected examples of OIG performance results and a table displaying the distribution of results
by Strategic Area.
Objectives
Improved organization systems and production processes.
Measures: Work done/accessible electronically; Work done by Multi-disciplined teams
*2002 Target 82% work is electronic
*2002 Target 21 Multi-teams
2002 Results 59% electronic
2002 Results 65 Multi-teams
FY2002 Target Met 72%
FY 2002 Met 3 10%
Increased recognition for diversity, innovation, and teamwork.
Measures: Innovation implemented; Staff in parity with civilian workforce; Staff recognition
*2002 Target 20
innovations
*2002 Target 96% parity
*2002 Target 489 Awards
2002 Results 42
innovations
2002 Results 98% parity
2002 Results 475 Awards
FY2002 Target Met 2 10%
FY2002 Target Met 102%
FY2002 Target Met 97%
Improved continuous learning and demonstrated competencies in EPA programs,
professional skills, technical skills and leadership skills.
Measures: Staff in compliance with professional or strategic standards
*2002 Target 100%
2002 Results 100
FY2002 Target Met 100%
* Annual FY 2002 targets are a cumulative 10% increase from FY 2001 baseline
GOAL 4 Results By Measure
FY2002 Measures of Results and Progress
98%
65
475
70%
48%
42
100%
% Staff In Parity with Civilian workforce
# Assignments By Multi-disciplinary Teams
# Awards for Creativity, Innovation & Teamwork
% Work Performed Electronically
% Products Available Electronically
# Innovative Techniques/Processes, Implemented (by OIG)
% Compliance with Professional Training Standards
FY2001 Cumulative Total
94.2%
19
515
72%
77%
18
N/A
98%
65
990 (495 Avg)
70% (71% Avg)
47% (62%)
60
100%
14
-------
Examples of Selected Results Demonstrating Progress On Goal 4
Objectives'. Improve organizational systems and processes through technology and multi-
discipline approach. Increase and recognize organizational and staff achievement, innovation,
teamwork and diversity. Improve professional competencies in EPA programs, technical, &
leadership areas
The OIG Central Audit /Evaluation Resource Center (CAERC) Newsletter is innovative in its
use of photojournalism to communicate CAERC highlights with staff throughout EPA OIG.
Articles and photographs within the CAERC Newsletter capture such items as the personal and
professional accomplishments of CAERC staff; volunteer efforts; new employee
backgrounds/biographies; achievements; training summaries; and staff creativity. CAERC staff
create and produce the Newsletter electronically. Staff are encouraged to submit articles and
photographs electronically via E-mail. The Newsletter is maintained on a shared drive accessible
by all CAERC staff. (Communications)
Training Information System (TIS) is now in beta testing. Once completed the TIS will be an
integrated training and career management system. TIS will produce the Human Capital Budget,
provide Assignment Mangers with critical information regarding staff availability, monitor the
execution of IDP's and evaluate the effectiveness of training while maintaining electronically
CPE and training records. (Human Capital/Integrated Knowledge Management Systems)
IT Security Awareness Training was provided electronically to OIG staff nation-wide.
AutoAudit was implemented in the OIG in April, 2002. AutoAudit is a Lotus Notes database
which contains, organizes, and displays all electronic working papers for audits and program
evaluations. (Integration of High Performing Culture/Knowledge Management Systems)
EPA OIG, in conjunction with President's Council on Integrity and Efficiency (PCIE),
developed a Compendium of Environmental Programs, an interactive database which catalogues
and cross references Federal environmental programs in 29 Federal departments and agencies
for collaborative planning, implementation, evaluations and resources sharing. This is the first
product of its kind to promote partnering and cross agency coordination. (Integration of
Knowledge Management Systems, Air, Water, Waste Mgt, Partnering)
EPA OIG began leadership of the President's Council on Integrity and Efficiency (PCIE),
Government Performance and Results Act (GPRA) Interest Group Round Table and quickly
expanded the scope to include the President's Management Agenda. Through monthly
meetings, this group has become one of the most active entities within the PCIE community and
is recognized by well known external organizations as a forum for sharing current information
on government performance and accountability. (Partnering/Integration of High Performing
Organizations)
15
-------
FY 2002 OIG Operational Activity and Audit Resolution
Audit/Evaluation
Reports Issued
*• Reviews Performed by OIG 50
*• Reviews by Another Federal Agency 186
*• Single Act Audit Reviews 134
TOTAL Reports 370
Audit Resolution
Millions)
Cost Disallowed to be Recovered
Costs Disallowed as Efficiencies
Agency Recoveries
Reports Resolved
(Dollars in
$ 7.9 M
$ 0
$12.8 M
149
> Audits with no Federal action as of 9/30
which are over 365 days past issuance date 133
*• Reports for which no management decision
was made within six months of issuance 38
Audit Resolution Reported by EPA
Audits with management decisions but
without final action beginning FY 02 $153.2 M
Audits for which management decision
were reached in FY02 $ 7.0 M
Total audits pending final action
during FY 02 $160.2 M
Final Action taken during FY02 $ 10.4 M
Audits without final action end of FY 02 $ 149.8 M
Investigative
Investigations Opened
Investigations Closed
Pending Investigations as of 9/30
Indictments of Persons/Firms
Conviction of Persons/Firms
85
64
181
35
20
Administrative Action EPA Employees/Firms 19
Civil Judgments 6
Other
Percent Audit $ Value Resolved in F Y 02
6.5%
Hotline Complaints Received 1,018
Hotline Complaints Closed 821
Legislative/Regulatory Items Reviewed 48
EPA's GPRA Rating Improvement
In F Y 02 from OIG Involvement
(From 6th to 4th out of 24 CFO agencies)
As rated by the Mercatus Center 11%
President's Council on Integrity and
Efficiency (PCIE) projects led/developed
Advisory participation on Agency
management improvement task force
implementing changes and
recommendations on GPRA and
President's Management Agenda issues
14
16
-------
OIG Customer Quality and Value
As Measured From FY 2002 OIG Customer Survey Results
Actual OIG External Customer Survey Results Compared to Strategic Targets
(Based on 100% scale)
Year
Questions About:
OIG Products/Services
OIG Staff
TOTAL
1998/1999
Actual
73.5%
77%
74.6%
2000
Actual
74%
79.6%
76%
2001
Actual
77.5%
85.3%
80%
2001
Target
74%*
82%*
77%
2002 2002
Actual Target
83.4% 84%
Customer/Client Attribute Survey Rating Results FY 1999-2002
2005
GOAL
80%
90%
85%
Survey Question
N Factor
(# of responses)
Question 3
Question 4
Question 5
Question 6
Question 7
Question 8
Question 9
Question 10
Question 11
Question 12
Question 13
Question 14
Question 15
Question 16
Question 17
TOTAL
Attributes
Response rate : 98/99 63%; 2001 58%; 2002 55%
Respondent knowledge of IG Act/Mission
OIG Products/Services are: Factually accurate
Objective and balanced
Relevant & significant
Useful for decisions & actions
Recommendations or advice are practical
Logical and understandable
Timely
Responsive to needs/requests
Contributes to EPA goals
OIG Staff : Are professional courteous
Are knowledgeable of programs
Communicate clearly
Seek comments & clarification
Build constructive relationships
1998/99
57
54%
75%
73%
75%
70%
70%
75%
75%
75%
73%
87%
68%
75%
77%
77%
74.6%
2000
26
X
75%
72%
70%
75%
70%
78%
78%
77%
73%
88%
75%
80%
78%
77%
76%
2001
90
50%
78%
77%
82%
78%
72%
78%
73%
77%
77%
92%
78%
85%
82%
85%
80%
2002
95*
68%
78%
77%
80%
75%
75%
78%
72%
75%
73%
90%
77%
82%
83%
85%
79%
* several responses represent groups, so total responses are over 120
17
-------
FY 2002 Narrative Customer Survey Questions and Responses
Program knowledge - technical and program knowledge; more needed by OIG;
auditors should use all program information provided
Recommendations - too broad and far reaching; too vague and unclear, and should
be more specific
Timeliness - OIG should try to produce more timely reviews; auditors need time to
work with issues until completed; Agency needs more time to provide comments
Communication - OIG audits should be clear on scope and criteria used; OIG
should follow up on leads which present themselves during review; auditors should
provide Agency with monthly email status reports; audit teams should better
coordinate their assignments with other audit teams; SRF strategy should be
overhauled; audits should focus on important Agency issues; audits should
communicate specific grant numbers in audits; auditors should communicate entire
single audits (no excerpts); audits should focus more on OIG mission; auditors
should review documents sent to OMB; should notify regional liaison when audits
are terminated; and should communicate with GAO to avoid duplication
Collaboration- OIG needs constructive relationships with Agency and should reach
agreement on final products; audits should focus on important Agency issues;
audits should follow appropriate procedures; audits should explore issues
completely
15
10
19. With what products or services, and in what program areas, can the OK
best serve EPA ?
Progress, follow up and knowledge about whether implementation of
recommendations actually took place
Financial issues - Continue SRF and internal control audit work
Proactive assistance - partner with Agency training efforts; partner in the area of
grants management; link single audits to OIG mission; balance national and regional
concerns; more attention to criminal cases
Program reviews- do more work in this area
Assistance Agreement, Contract and Single Audits - These are extremely useful
and help monitor grantees/contractors - do more and faster
Number of
Comments
3
2
9
2
4
18
-------
20. How do OIGproducts or services add value? Ifvou do not believe thev
add value, whv not?
Financial, SRF audits/investigations invaluable to EPA financial programs,
need more of this type of work
Provide independent/balanced reviews - reports do adequately recognize
accomplishments and progress
Focus on Issues - OIG helps programs focus on major issues, next steps,
solutions, need for internal controls, review tribes as sovereigns not subdivision
of states
New collaborative approach is better
Number of
Comments
Other Comments By Customers and Clients
Complaints
Report findings had minimal relative value
Staff should be more professional, courteous, knowledgeable
Baseless allegation and review was biased and had no value
Audit contained factual errors and illogical conclusions
Investigative work not thorough and conclusions were vague
Staff should listen better to understand issues, not jump to conclusions
Compliments
OIG staff were professional and able to communicate needs and goals
effectively
Final product will be useful to the program
OIG staff were extremely hardworking and dedicated to the OIG mission
OIG did excellent job. Brought things to our attention that we overlooked
Good systems review, including review of policies, procedures, and internal
controls
Excellent criminal/civil investigative agent work
Questions were always answered promptly
OIG did good job on difficult project where information was missing
OIG did excellent job of identifying problems and working with contractor to
correct them
OIG staff provided new information and ideas to the discussion
OIG staff were enthusiastic and committed to helping solve problems
19
-------
FY 2002 FINANCIAL STATEMENTS FOR OIG, (Figures From IFMS - Unaudited)
(Source: EPA Integrated Financial Management System)
Budget Object Class
Total FY 2002
Expenditures
Average Cost per
Staff Year in
OOOs
MANAGEMENT
Salaries
Travel
Program. Expenses
Administrative Expenses
Program Contracts
Administrative Contracts
Working Capital Fund
Grants
Awards
Reserves/EPA take back
TOTAL
SUPERFUND
Salaries
Travel
Site Travel
Program Expenses
Administrative Expenses
Program Contracts
Administrative Contracts
Working Capital Fund
Grants
Awards
Reserves/EPA take back
TOTAL
264.1 Staff Years
$25,754,741
1,891,905
58,606
1,166,897
1,408,595
1,934,920
1,648,197
135,816
86,984
456,907
$34,543,568
$8,818,898
570,897
1,519
14,085
287,078
460,876
474,864
33,954
32,869
$10,695,040
$97.5
$7.2
$4.4
$7.3
$6.2
.J
$122.9
73.3 Staff Years
$120.3
$7.8
$3.9
$6.5
.J
$139.0
FY 2002 GRAND TOTAL
$45,238,608 337.4 Staff Years
$126.5
20
-------
FY 2003 OIG PERFORMANCE TARGETS FY (with targets FY 2002 - 2005)
Crosswalk between OIG Strategic Plan Objectives/Measures & EPA Annual Performance Plan
Goals
OIG STRATEGIC GOALS / EPA OIG Al
FY 03 OIG GOAL 1: EPA APG 1: Improve environmental quality and human health by
identifying 80 recommendations, risks or best practices; contributing to reduction or elimination of 20
environmental risks; and 60 changes or actions influencing positive environmental or health impacts.
Objective Measures and Targets:
*• 1. Environmental Improvements/Changes/
Actions (Legislative, regulatory, policy,
directives, best practices, Environmental or
health improvements)
Intermediate Outcome & Outcome Measures
*• 2. Environmental Risks Reduced /Eliminated
Certifications, Verifications, Validations;
Outcome Measures
*• 3. Recommendations, Risks or Best Practices
Identified; Output Measures
50
15
75
60
20
80
70
25
90
80
30
100
FY 03 OIG GOAL 2: EPA APG 2: Improve EPA's business and program operations by identifying
155 recommendations, potential savings & recoveries for 150% annual investment in OIG, 75 actions
for better business operations, & 50 criminal, civil or admin actions reducing risk of loss/integrity.
Objective Measures and Targets:
*• 4. Potential Dollar Return on Savings,
Questioned Costs, Improved Business
Practices, Recoveries, Fines, Settlements
Outcome Measures
*• 5. Criminal, Civil Administrative Actions
Reducing or Eliminating Risk of Loss &
Operational/Data Integrity Intermediate
Outcome Measures
*• 6. Improvements in Business/Systems/Efficiency
(Actions Taken on Mgt Challenges,
Certifications, Best Practices, Policies, Regs)
Outcome Measures
*• 1. Recommendations Made or Weakness, Best
Practices Identified Output Measures
100%ROI
50
70
150
150%ROI
50
75
155
150%ROI
50
80
160
200%ROI
50
85
165
-------
Appendix 1
OCTOBER 7, 2002
MEMORANDUM
SUBJECT: Fiscal Year 2002 Report on Management Controls
TO: Christine Todd Whitman
Administrator
I am submitting this annual report from the Office of Inspector General (OIG), in accordance with the
requirements of Office of Management and Budget (OMB) Circular A-123, Management Accountability and
Control, and the Federal Managers' Financial Integrity Act (FMFIA). This report also complies with the internal
control requirements of OMB Circular A-130, Management of Federal Information Resources.
ASSURANCE STATEMENT
I have taken the necessary actions to assure we have evaluated our internal controls in accordance with the
guidance provided by the Office of the Comptroller. Based on our evaluation process and my personal
knowledge, I believe the internal controls in effect in the Office of Inspector General on September 30, 2002,
taken as a whole, provide reasonable assurance of compliance with FMFIA. The OIG has, however, identified
six issues this year as OIG-level weaknesses, four of which contain issues previously identified as office-level
weaknesses in Fiscal Year 2001. These areas pertain to:
1. Human Resources Strategy (updated)
2. Information Technology (IT) Strategy (updated)
3. Inspector General Operations Reporting System (updated-IT strategy last year)
4. Product Timeliness and Quality (updated-included under Business Systems last year)
5. Follow-up on Corrective Actions (new)
6. Organizational Structure (new)
We have provided a brief description of these issues and the actions we have taken or plan to take under
the Office-Level Weakness section of this report. We also plan to develop a more detailed Management Action
Plan, with milestone dates and action officials.
MANAGEMENT CONTROL ACCOMPLISHMENTS
During Fiscal Year (FY) 2002, we conducted vulnerability assessments in each of the OIG's assessable
units. This year, we also conducted a comprehensive, internal Quality Assurance Review of the OIG Resource
Centers. We designed the review to evaluate: (1) the extent to which each center was complying with
Government Auditing Standards and OIG policies and procedures; and (2) whether the office consistently
maintained documentation to support continuing professional education (CPE) completion, consistently
followed OIG and General Accounting Office guidance for independent referencing of all reports, and assured
that projects were free from indications of personal or external impairments to independence. These reviews did
not identify any issues which were significant enough to be considered material control weaknesses.
The OIG also took the following steps to improve management controls:
• Implemented a Management Action Plan to improve the OIG's systems, processes, and office
environment. The plan categorized the three office-level weaknesses identified in FY 2001;
identified goals, action items, accountability, and milestone dates; and established a process to track
our accomplishments and progress.
• Continued organizational-wide training in concepts of personal and organizational leadership and
performance excellence.
• Reviewed administrative and operational issues to clarify and reinforce policies, procedures, and
-------
management control requirements and revised policies and procedures, as needed.
• Improved OIG budgeting, analysis and reporting requirements under the Government Performance
and Results Act, FMFIA, and the Inspector General Act.
• Developed a new procedure for conducting internal Quality Assurance Reviews (QARs), completed
a comprehensive QAR review of all OIG Audit/Program Evaluation Resource Centers, and
produced a new policy document on the OIG Quality Assurance Program.
• Implemented a Performance Results and Measurement Data Base.
• Developed an OIG Annual Performance Report to measure progress in reaching our Strategic Goals
including a balanced scorecard of financial, customer service, timeliness, and effectiveness
measures.
• Designed and implemented an Annual Assignment Suggestion and Planning System Data Base.
• Implemented an OIG-wide 360° performance feedback system.
• Developed a draft OIG Needs Assessment Tool and Career Management System.
• Established a Cost Accounting Workgroup to work with the Office of the Chief Financial Officer to
develop and implement a cost accounting system within the OIG.
• Deployed a knowledge and skills inventory system (KSIS).
OFFICE-LEVEL WEAKNESSES
The OIG identified six issues as office-level weaknesses for FY 2002. In each area, we are actively
working to develop clear plans, policies, and procedures to improve upon these weaknesses.
1. Human Resources Strategy
The OIG previously identified areas for improvement in our Human Resources Strategy and this years'
assessment identified additional areas for improvement. In FY 2002, we completed a career path framework to
serve as criteria for staff development and began designing a training information system to improve the
organizational curriculum. We also deployed a knowledge and skills inventory database to provide a basis for
staffing projects, identifying skill gaps, and assessing the OIG's overall workforce capacity.
The OIG is currently performing a workforce analysis and is developing a workforce plan which
analyzes the type of work we plan to conduct and the appropriate skills necessary to conduct that work. Based
upon this analysis, the OIG will be better able to define its overall recruitment strategy including specialized
skills and competencies, diversity goals, and national and local recruitment practices. To continue our human
resource efforts we will:
• Complete and implement the OIG workforce plan.
• Task the Assistant Inspectors General (AIG) and Directors to continually identify skills and
competencies needed.
• Develop updated diversity goals based on Census 2000 information.
• Complete development of a training information system.
• Emphasize career development of staff in the Directors' performance standards.
• Disseminate the Career Path Framework to serve as criteria for staff development.
• Develop and implement a rotational assignment policy.
• Implement performance expectations agreements OIG-wide.
• Work closely with Team Vegas (the OIG personnel service provider) to expedite the development
and classification of organizational position descriptions and to produce vacancy announcements
that clearly describe the skills and competencies needed to fill positions.
• Determine that position descriptions exist for all positions, and reassign or recruit to fill vacant
positions.
• Develop a formal assessment process for determining whether individual development plan
objectives have been met and ensuring that new skills are recorded in the KSIS.
-------
2. Information Technology (IT) Strategy
In FY 2002, we completed a draft IT Strategic Plan which addressed immediate IT needs and new,
innovative technologies. We published a new records management policy and procedural guidance handbook
for hard copy documents; customized and deployed AutoAudit, a commercial, paperless audit software program
to assist auditors in creation of electronic work papers; provided training on AutoAudit to all OIG employees;
interfaced the Inspector General's Operating Reporting System (IGOR) and AutoAudit; developed an action plan
to establish an OIG-wide correspondence control system; implemented a file/records inventory in 30 percent of
OIG offices, with 100 percent implementation expected in FY 2003; implemented a new WEB policy that
includes a reporting requirement on WEB accomplishments; and began recruiting additional staff with the
expertise to address the needs of the OIG Intranet WEB system.
Advances in IT hardware, however, have not expanded at the rate of our changing organizational needs.
While there is a plan to replace aging equipment, we have not developed a long-range plan to purchase new,
innovative software and hardware technologies. We also do not have a centralized system to inventory current
hardware and software. Although we now have broader remote access capabilities, we sometimes experience
system difficulties which limit our connectivity capability. Further, we sometimes experienced long waits for
contractor assistance in response to help desk issues. Moreover, we continue to experience operational
constraints due to outdated, or improperly "linked" information on our intranet WEB site.
To address remaining IT strategy weaknesses, we will:
• Hire a Chief Information Officer as the AIG for Mission Systems.
• Complete and implement a 5-year IT strategic plan that includes immediate needs and new,
innovative technologies.
• Complete a multi-year procurement plan in conjunction with the next OIG Strategic Plan.
• Continue conducting customer surveys and implement corrective actions.
• Develop a feedback system to evaluate contractor "troubleshooting" performance.
• Consider other contractors as necessary.
• Develop a centralized hardware and software inventory system.
• Work closely with the Agency as it develops an electronic records archival system and volunteer to
pilot this system.
• Implement a files/record inventory OIG-wide.
• Expedite the posting of OIG products to the Internet and Intranet.
• Develop and disseminate an electronic records management policy and procedures when an Agency-
wide policy is established.
• Establish and implement an OIG-wide correspondence control system.
3. Inspector General Operations Reporting System
The OIG previously identified issues with our internal information system, IGOR, and this assessment
identified additional needs. In FY 2002, we implemented controls within IGOR to facilitate the thorough,
accurate and timely input of assignment data and timely timecard data review, and linked this to individual
performance agreements. We also issued an IGOR user handbook and a training/guidance document, trained
users, created a system to facilitate the use of sub-accounts across office jurisdictions to allow individuals
working on teams outside their individual organizations to charge time to the project, and provided an on-line
user handbook on master and sub-accounts.
Even with enhancements, the IGOR system remains unable to meet current management information
needs; and the system's capability and capacity still have not been fully adopted by staff. Our project data in
IGOR is inaccurate due, in part, to staff not completing or inconsistently completing IGOR project data fields.
Further, time sheet data is not always entered and approved in a timely manner. The data entry process for
projects is onerous and inconsistent because of system limitations and project categories are somewhat arbitrarily
determined. Finally, all staff have not received training in the use of IGOR, nor do all OIG employees have
access to relevant IGOR information needed to manage projects. To address these issues, we will:
-------
• Determine those projects that have data problems and hold the respective Director accountable to get
the information updated.
• Emphasize the importance of ensuring IGOR information is current and accurate by including
oversight language in AIGs' and Directors' performance agreements
• Provide access and training to all OIG employees who need and use IGOR data.
• Streamline IGOR options for data fields.
• Develop a data dictionary and a detailed users' manual.
• Improve IGOR's capacity to better track project progress and resources used.
• Develop and distribute a monthly report to project managers so they are able to identify missing or
inaccurate data.
• Improve the link between IGOR and AutoAudit to avoid gaps and inconsistencies in the information
used to manage projects.
• Incorporate into OIG Performance Expectation Agreements responsibility for maintaining and
verifying data contained in IGOR and AutoAudit.
• Implement an integrated system to replace IGOR.
4. Product Timeliness and Quality/Accountability
Over the past year we have worked to develop an integrated Multi-Year Plan which incorporates the
work of the Offices of Audit, Investigations, and Program Evaluation. We have developed an Assignment
Process Handbook, a Business Systems Guide, and worked to consolidate our report editing and distribution
functions. While these efforts will help improve management, timeliness, and cost-effectiveness of OIG
projects, we need to develop an OIG-wide accountability system including, project management, cost
accounting, and project follow-up. Currently, we do not have adequate systems in place to ensure timely
completion of projects. We need to develop an OIG-wide system to ensure milestones, due dates, and objectives
are consistently established and that changes to the initial projections are justified and documented.
Further, while the OIG completed several significant projects over the past year, we did not produce our
work plan in timely fashion. Individual projects in the plan did not always contain clear objectives which
required additional time to develop during the course of our audit and evaluation work. The work plan should
clearly identify the body of work that would help us effectively achieve our strategic goals. Delays in producing
this work plan have resulted in delays in work products, products that did not always meet the intent of our
"matrix" organization, and staff concerns about their work and their contributions to the organization.
We continue to implement our strategy to improve performance with emphasis on organizational values
and assessment criteria. However, we have not yet fully implemented project-specific performance expectations
agreements to define individual and/or team project expectations. Clearly defined expectations establish
accountability and criteria to measure performance and help ensure quality projects. To improve product
timeliness and quality, we will:
• Link accountability to individual SES performance agreements.
• Hold regular meetings with the product line AIGs and Directors to monitor progress in developing
and implementing the OIG work plan.
• Task the product line AIGs and Directors with ensuring that projects are well developed, include
clearly defined objectives, and are efficiently & effectively assigned and managed.
• Task the Product Line and Resource Center Directors as well as Project Managers with effectively
communicating to staff how their projects fit into the overall work plan.
• Conduct specific training on the Assignment Process Handbook to ensure all staff members
understand and comply with professional standards and OIG policies.
• Require the Resource Center Directors to certify that the Project Managers received project
management training.
• Implement project-specific performance expectations agreements to define individual and/or team
expectations.
• Participate in the Agency's Business Objects pilot, a business intelligence reporting tool, which, if
effective, will provide an OIG-wide cost accounting system.
-------
5. Follow-up on Corrective Actions
The OIG conducts many types of audits, program evaluations, and investigations to help identify and
recommend corrective actions for Agency improvement. Organizationally, we should evaluate whether the
Agency implements the corrective actions, and determine whether our recommendations effectively contribute to
resolution of the problem and attainment of EPA's goals. Currently the OIG does not have a follow-up policy
which would hold the Agency accountable for taking timely appropriate actions and hold the OIG accountable
for the outcome of its work. Nor have we clearly defined a process for identifying, prioritizing, tracking, and
correcting OIG Integrity Act weaknesses and other internal management assessment review findings.
While the OIG has met its responsibility to identify the key Agency management challenges, we have
not developed a systematic method for gathering this information or following up on previously reported
Management Challenges. Without follow-up reviews, the OIG has difficulty determining what actions the
Agency has taken to improve upon these challenges. To improve follow-up, we will:
• Task product line AIGs with identifying follow-up projects from previous OIG work, and projects
that will address reported management challenges.
• Create a job bank for short-term projects and work with Directors to instruct and assist them in
loading it with follow-up projects.
• Hold regular meetings with the product line AIGs and Directors to monitor progress made on these
follow-up projects.
• Develop a methodology for identifying and prioritizing key Agency management challenges.
• Implement an OIG-wide process for identifying, prioritizing, tracking, and correcting OIG Integrity
Act weaknesses and other internal management assessment review findings.
• Include language in AIGs' and Directors' performance agreements for implementing the new OIG-
wide process to recognize, prioritize, and report on Agency management challenges and OIG internal
weaknesses, and address timely follow-up on them.
6. Organizational Structure
The OIG continues to undergo significant organizational change as we plan and implement our work and
Human Capital Management strategies. These operational changes have outpaced our personnel system and have
sometimes outpaced our timely completion of re-organizational documents and related personnel transactions.
To address these issues we will:
• Submit a reorganization package to the Agency and Team Vegas that accurately represents our
current organizational structure.
• Place a high priority on filling senior level vacancies and communicate this priority to the Agency's
Office of Human Resources and Organizational Services and Team Vegas.
• Revise and monitor internal and external organizational information to ensure our organization and
contact information is always current and accurate.
Should your staff have any questions concerning this report, please have them contact Elissa R. Karpf,
Assistant Inspector General for Planning, Analysis, and Results, on (202) 566-2604.
Nikki L. Tinsley
cc: Assistant Administrators
General Counsel
Associate Administrators
Regional Administrators
-------
Appendix 2
EPA's Top Management Challenges as Reported by OIG (Historical Perspective)
Below is the list of Top Management Challenges that the OIG has reported to EPA each year since 2000.
While EPA is making progress in resolving its Major Management Challenges, several have been
longstanding problems. The following table shows which challenges have been listed from 2000 through
2002 and their relationship the President's Management Agenda (as numbered).
Management Challenge HI
Environmental Data Quality (2)
Emission Factor Development
Year 2000 Modification on Information Systems
Closeout of Construction Grant Program
Inconsistent/Oversight Enforcement Activities
Quality Assurance/Plans
Use of Inefficient Contract Types
Managerial Cost Accounting (1 & 4)
Automated Information System Security Plans/Security (2)
Oversight/Use of Assistance Agreements (4)
Agency Relationship with Contractors
Environmental Data Information Systems
Accountability (1)
Backlog; Nation'l Pollutant Discharge Elimin. System Permits
Enhance Employee Competencies/Human Capital (3)
Process for Preparing Financial Statements
Superfund Five-Year Reviews
Great Lakes Program
Quality of Laboratory Data
Information Resources Management (2)
Working Relationships with States & Other Partners (1)
Results-Based Information Technology Project Management
Protecting Critical Infrastructure from NonTraditional Attacks
Biosolids
Air Toxics Program
**
**
**
8
**
*
**
*
**
§
**
*
**
*
**
**
President's Management Agenda
1. Budget and Performance Integration
* Combined for 2001 and 2002 as "Linking Mission with Management" 2. Expanded Electronic Government
** Combined for 2001 and 2002 as 3. Strategic Management of Human Capital
"Information Resources Management & Data Quality " 4. Improving Financial Performance
-------
Appendix 3
DEFINITIONS OF FY 2002 PERFORMANCE MEASURES
EPA OIG Performance Measurement Results System Database
GOAL 1 - CONTRIBUTE TO IMPROVED ENVIRONMENTAL QUALITY AND
HUMAN HEALTH
Number of Recommendations for Environmental Improvement
Number of environmentally-related (not management process) recommendations, suggestions or advisory
items from any OIG work including briefings, memos, reports, discussions, etc. Record these
recommendations on a COMPLETED form as COMPLETED RESULTS (outputs). Rollup similar
recommendations that together address each opportunity or need for environmental improvement or
process change. Environmental implications of recommendations should be described.
Legislative Change/Decision (means to impacts)
Any OIG work which through recommendations, discussions, observations, conclusions, or risk
identification results in, or significantly contributes to drafts or actual legislative changes to improve an
environmental program. Measured in number of changes and described in terms of actions or impacts on
environmental programs and results; such as, impact time, measures of environmental improvement,
numbers of people affected, health & behavioral changes and compliance with standards. Would also be
identified in other measures listed below, within each goal.
Regulatory Change/Decision
Any OIG work which results in, or significantly contributes to, draft or actual regulatory changes to
improve environmental program implementation. Measured in number of changes and described in terms
of potential impact on environmental programs and results; such as, time, measures of environmental
improvement, numbers of people affected, health & behavioral changes and compliance with standards.
Would also be identified in other measures listed below within each goal.
EPA Policy. Directive. Process. Practice. Action. Change/Decision
Any OIG work which results in, or significantly contributes to, specific changes in definitions, purposes,
objectives, processes, requirements, controls, competencies, reporting in the formulation of environmental
policies, plans or results. Measured in number of changes or actions and described in terms of impact on
environmental programs and results, such as, time and measures of environmental improvement,
numbers of people affected, health and behavioral changes. Would also be identified in other measures
listed below, within each goal.
Examples of Environmental Improvements/Impacts
Any OIG work that results in, or significantly contributes to, an identifiable environmental or health
condition or improvement, measured in the number of improvements, and described by the prospective or
actual impacts on physical characteristics or behavioral changes.
Environmental Risks Identified (including noncompliance)
Any OIG work which identifies actual or potential environmental risks, or health risks, for future plans,
action, reduction or elimination. Described by potential/actual environmental or health vulnerabilities,
behaviors or conditions.
-------
Environmental Risks Reduced or Eliminated
Any OIG work that results in, or significantly contributes to, the reduction or elimination of environmental
risks. Measured in terms of the number of individual risks (by type) of exposure, incidence, or immanent
threats, described by number of exposures or people exposed.
Environmental Best Practices Identified
Any OIG work describing a best practice for environmental program implementation, or resolution of an
environmental problem (from any source, State, region, other agency, etc.), measured by the number of
best practices identified and described by potential environmental impact.
Environmental Best Practices Transferred/Implemented
Number of environmental program best practices disseminated (output) through OIG work, and the
number which were implemented (outcome) by Agency program offices, regions, States, other
government agencies, other IGs, or other environmental organizations. Describe each output or outcome
action and its environmental implication.
% Change in Environmental/Health Performance Measure or Indicator (from baseline)
Any OIG work which contributes to a positive change in an EPA or other environmental/health
performance measure, indicator or condition. Described by change in behavior, environmental or health
condition.
Certifications/Verifications/Validations/Information Integrity
Any OIG work which results in the certification, verification, or validation , or contribution of knowledge
or understanding of Agency environmental work/programs upon which decisions and actions depend.
GOAL 2 - IMPROVE EPA's MANAGEMENT, ACCOUNTABILITY AND PROGRAM
OPERATIONS
Questioned Costs (actual numbers, round up, or use decimals)
Report all questioned costs and monetary value of findings and recommendations as COMPLETED
RESULTS on a COMPLETED FORM. Describe nature of costs questioned,
Recommended Efficiencies. Cost Saved or Avoided (monetized results including from advice, research,
training and application of techniques)
As a result of OIG work:
1) the cost of Agency work products or office functions which have been eliminated because they were no
longer of use, or too costly; and,
2) the cost of new or streamlined Agency processes or work products which have been instituted to save
the Agency time and/or money. The monetary benefit, both immediate and for the immanent future
(absence of our involvement), of OIG work to EPA, other Federal/State partners, and stakeholders.
Describe the nature of the savings including time.
Fines. Recoveries. Restitutions. Collections
Dollar value of investigative recoveries, meaning,
1) Recoveries during the course of an investigation (before any criminal or civil prosecution);
2) court-(criminal or civil) ordered fines, penalties, and restitutions; and,
3) out-of-court settlements (including administrative actions resulting in non-court settlements. Describe
nature of amounts and reason.
-------
Criminal/Civil/Administrative Actions
Number of successful prosecutions, indictments, and resulting settlements and convictions affecting EPA
operations and environmental programs. Includes the number of persons or entities that were:
1) indicted, or for which an "information" was filed, 2) were found guilty or pled guilty in a court of law,
or 3) were accepted for pre-trial diversion agreements by the Department of Justice. Described the nature
of the action and the business of environmental implication.
Best Practices Identified
Any OIG work describing a best management or program operation practice, (from any source; State,
region, other agency, etc.) Describe each best practices and its implication for effectiveness & economy.
Best Practices Implemented
Implementation of and best management or program operation practice, identified or described through
OIG work (from any source; State, region, other agency, etc.) Describe each best practices implemented
and its implication for effectiveness & economy (time/money/accuracy/risk reduction/satisfaction).
Examples of Process/Practice/Policv Changes/Decisions/Actions
Any OIG work that influences process, practice, policy regulatory legislative changes, actions or decisions
made by EPA, states, or other agencies. Includes audits, including data integrity or systems audits,
evaluations, investigations, assistance, hotline, research, correspondence, meetings, conferences, contract
audits, advisory briefings, analysis, work groups, etc. Describe action and its implications for
effectiveness & economy (time/money/accuracy/risk reduction/satisfaction).
Recommendations for Management Improvement
Number of management and program operation, regulatory, policy or legislative recommendations,
suggestions or advisory items, from any OIG work including briefings, memos, reports, discussions,
analysis, workgroups etc. Record recommendations on a COMPLETED form as COMPLETED RESULTS
(outputs). Rollup similar recommendations that together address one management improvement or process
change. Describe each recommendation and its implication.
New FMFIA/Management Challenges Identified
Number of new FMFIA program assurance issues and EPA management challenges presented as a result
of any OIG work. Includes issues presented in EPA financial statement audits, and internal OIG reviews.
Corrected Action on Management Challenges/FMFIA Weakness
Number of action taken to resolve management challenges, high risks or material weaknesses within three
years following OIG recommendations, assistance or advice. Describe action and implications.
% EPA External Rating on Implementing GPRA
Outside-influenced scorecard of GPRA ratings for the Agency, which the OIG has influenced for
improvement. This includes ratings on strategic plan, annual plan, and annual performance report, and
specific examples of strengths or improvements influenced by OIG involvement.
Certifications/Verifications/Validations/Information Integrity
Any OIG work which results in the certification, verification, validation, or contributes to the
knowledge/understanding of Agency management work or programs, measures, accounting, and data
integrity upon which decisions and actions depend. Describe the nature of each.
30
-------
GOAL 3 - PRODUCE TIMELY, QUALITY, & COST EFFECTIVE PRODUCTS &
SERVICES THAT MEET CUSTOMERS NEEDS
% Customer Service Value/Quality Rating
Percent based on the results of customer surveys, about the work we do for OIG customers; for/with the
Agency, stakeholders, and other partners in the IG community. This will be coordinated and tabulated by
the Office of Planning. Analysis and Results (PAR) in collaboration with each OIG office.
# of Collaborative Partnerships with Fed/State Agencies
Number of collaborative partners in any OIG product or service that are conducted jointly with other
Federal agencies, States, or entities inside or outside EPA, such as the President's Council on Integrity and
Efficiency (PCIE). List the Partners.
# of Requests to Testify/Info for Hearings/ Presentation Requests
Number of times OIG staff are requested to testify in courts, to Congress, other public venues, or provide
information for presentation in hearings or other judicial or legislative action, regarding audit, investigative
or evaluation findings and recommendations, or other subjects relating to OIG work. Describe the type,
topic and audience for presentation.
Was Assignment/Product/Service Requested or Required
By responding with a Yes or No, we will be able to determine the percent of OIG work requested or
required by number of assignments, and by percent of time expended. A "YES" represents work products
requested from customers, such as various EPA programs, Congress, hotline, allegations, legislation_by
whom the work was requested, or the authority requiring the work. A "NO" means work required only by
internal OIG plans or processes. Describe requirement or requestor.
% of Engagement Letter/Project Plan/Budget Expectations/ Milestones Met
Number of OIG audit, evaluation, project budget and planning commitments met with this assignment, as
a percent of total commitments.
GOAL 4 - ENHANCE DIVERSITY, INNOVATION, TEAMWORK & COMPETENCIES IN OIG
Products Completed By Multi-discipline Teams
All OIG work products accomplished by multi-discipline teams. Record number of products from this
assignment if the products were accomplished by a mix of auditors, investigators, engineers, evaluators, etc.
Most OIG products will be multi-discipline.
Awards for Achievement and Innovation/Teamwork
Number and type of awards for OIG staff (HQ and Regions) per year, or other specified time periods.
Innovative Approaches. Techniques. Processes and Solutions to Management Issues Implemented
Number and types of innovative approaches, techniques, processes and re-engineering successes
implemented within the OIG, to enhance OIG competencies and effectiveness in doing our work, and
enhance staff development activities.
% of Work Produced Electronically
Estimate how much of this assignment was "paperless" , produced using special electronic features, such as
database programs, functions, equations, charts and graphs, formulas, workpapers, files, presentations.
Work Accessible Electronically to customers (Yes/No)
Are OIG products and services easily available and accessible to customers/clients, the general public,
electronically, compared to those only available in hard copy, or e-mail?
31
-------
For questions, comments or to obtain copies of this report, please contact any of the
following:
Eileen Me Mahon, Assistant Inspector General for Congressional and Public Liaison (202) 566-2546
Mcmahon.Eileen@,epa. gov
Elissa Karpf, Assistant Inspector General for Planning Analysis and Results (202) 566-2604
Karpf.Elissa@,epa. gov
Michael Binder, Deputy Assistant Inspector General for Planning Analysis and Results (202) 566-2617
Binder.Michael@,epa. gov
U.S. Environmental Protection Agency
Office of Inspector General
1200 Pennsylvania Ave. NW
Washington, D.C. 20460
This report was prepared by the EPA Office of Inspector General
Office of Planning, Analysis and Results
------- |