I
55
o
%>
DIG
EPA-350-R-02-001
March 2002
OFFICE OF INSPECTOR GENERAL
Annual Performance Report
Fiscal Year 2001
-------
For questions, comments or to obtain copies of this report, please contact any of the following:
Eileen Me Mahon, Congressional and Media Relations (202)260-0401
Mctnahon.Eileen@epa.gov
Elissa Karpf, Assistant Inspector General for Planning Analysis and Results (202)260-8380
Karpf.Elissa@epa.gov
Michael Binder, Deputy Assistant Inspector General for Planning Analysis and Results (202)260-9684
Binder .Mi chael@epa. gov
U.S. Environmental Protection Agency
Office of Inspector General
1200 Pennsylvania Ave. NW
Washington, D.C. 20460
-------
FOREWORD
I am pleased to present the EPA-OIG's first Annual Performance Report. This Report, in
summary format, presents progress toward the attainment of our FY 2001 Performance Targets
and Strategic Goals through FY 2005 as described in our Strategic Plan.
Since our Strategic Plan was designed "Starting with the End in Mind," this report provides
summary performance information beginning with outcomes and outputs from our business line
products and services related to Goal 1 (Contribute to improved environmental quality and human
health), and Goal 2 ( Improve EPA's management and program operations). The Report then
provides performance results and progress supporting internal operations for Goal 3 (Produce
timely, quality and cost effective products and services that meet customer needs), and Goal 4 (
Enhance diversity, innovation, teamwork and competencies). The Report also includes analyses
of financial and staff resource usage, and concludes with a list of the Management Challenges
within the OIG to be addressed during FY 2002 and a revision of FY 2002 Annual Goal Targets,
based on the OIG's progress and lessons learned from FY 2001.
The net result of this report is to present a "Score Board" of performance, progress, results,
activities, and investments that tell a complete story of OIG performance and accountability.
While this is the OIG's first Annual Performance Report based on our Strategic Plan, we are
continuing to research new and better ways to measure, collect, and report data on the results of
our operations. The information in this report will be used to direct and motivate the future
application of resources for even greater results.
Nikki L. Tinsley
-------
TABLE OF CONTENTS
Page
Number
Summary of OIG Results for FY 2001- Scorecard 1
Map of OIG Strategic Vision, Mission, Goals, Objectives, Strategies .... 2
Annual Results Compared to Goal Targets 3
Examples of Results by Goal, Objectives and Measures
Goal 1 5
Goal 2 7
Goal3 9
Goal 4 11
Customer Service Results 13
Financial and Resource Management 16
Timeliness of OIG Products 19
Looking to the Future:
New FY 2002 Annual targets 21
Appendix 1 - Fiscal 2001 Report on Management Controls
(OIG FMFIA Results/Management Challenges) 22
Appendix 2 - Definitions of FY 2001 Performance Measures 27
OIG Products and Services The Work We Do
Financial/
Systems Audits
Systems
Financial Statement
Contract
Assistance Agreements
Computer Security
Fact Finding
Program Audits
and Evaluations
Process
Outcome
Impact
Cost-Benefit
Advisory/Analysis
Legislation & Regulation
Review
GPRA Implementation
Control Assessment
Fraud Prevention
Investigations
Program Integrity
Contract & Procurement
Employee Integrity
Assistance Agreements
Hotline
Computer Forensics
Introduction
This report provides a statistical summary of key OIG performance results and measures against
FY 2001 Annual Performance Targets and progress toward OIG Strategic Goals. In compliance
with the Government Performance and Results Act (GPRA), it describes examples of specific
outputs and outcomes, provides financial statements, Management Challenges, progress on previous
Management Challenges, and revised annual performance targets for FY 2002. It also provides
additional operational performance measures and indicators about OIG product timeliness, customer
survey results, and completeness and quality of data entry into OIG Management Information
Systems. We hope that this information will be used to inform and inspire improved operations.
-------
NOTES TO THE READER
This is the EPA OIG's first comprehensive Annual Performance Report. It
wrings together data from many sources using measures and systems that are
lew to the OIG. This "logic model" approach of aligning output and outcome
Treasures within our Strategic Plan, along with the application of technology
o capture results, is innovative. Like all significant changes, it requires
arganizational adjustment and presents opportunities to learn.
The information presented in this report is primarily for internal use and
iemonstration purposes. The results presented are accumulated from the data
entered into a new OIG Performance Results and Measurement System and
he IGOR (Inspector General Operations Reporting System). Some of the
eported results and timeliness information is incomplete, and not entered
consistently with the system data fields and definitions. The data has not been
comprehensively verified, but represents significant progress in collecting a
lew array of results supporting our four Strategic Goals.
t is from this beginning, that we will identify and apply needed data quality
issurances and accountability controls for reliable management information.
This report is designed to demonstrate progress against the EPA OIG
Strategic Plan, in compliance with the reporting requirements of Government
erformance and Results Act. It is not intended to duplicate descriptions of
significant audits, evaluations, investigations and other organizational activity
Dresented in EPA OIG Semiannual Reports to Congress. We suggest that the
eader also refer to the Semiannual Reports for FY 2001 available
electronically from HTTP://WWW.EPA.GOV/OIGEARTH.
-------
Summary of QIC Results for FY 2001 - Scorecard
Performance
Results
(highlights)
Goal 1. Contribute to Improved Environmental Quality and Human Health
* 59 Agency/delegated entity actions/decisions/changes/improvements influencing
environmental impacts (236% of target)
* 24 Environmental risks reduced or eliminated (160% of target)
> 3 Best practices implemented to improve environment & health (20% of target)
Goal 2. Improve EPA's Management and Program Operations
* $72.4 million in potential costs, savings, fines and recoveries (168% of target)
* 98 Criminal, civil, or administrative actions (196% of target)
* 63 Management practice/policy changes and 4 FMFIA/Mgt Challenges corrected
Goal 3. Produce Timely. Cost Effective Products and Services That Meet
Customer Needs
* 32 Collaborative efforts with other agencies (457% of target)
* 56% Of OIG work is required or requested (186% of target)
Goal 4. Enhance Diversity. Innovation. Teamwork and Competencies in OIG
> 75% of products developed or accessible electronically
* 94% Parity with civilian workforce
Operational
Activity
60 Reports Issued OIG Managed
214 Reports performed by another Federal Agency (e.g. DCAA)
211 Single Audit Act Reviews
172 Reports resolved
137 Audits with no final action by EPA, over 365 days past due
33 Reports for which no management decision was made within 6 months
42 Investigations opened
54 Investigations closed
Customer Service
80.0% Combined OIG Customer Service Rating (104% of target)
- 77.5% OIG Products/Services Rating
> 85.3% OIG Staff Rating
> Highest and lowest rated areas 73% Timeliness
72% Practicality of recommendation/advice
92% Professional & courteous
85% Builds constructive relationships
Use of Resources
$30,665,152 Mgt Expenditures (90% FY01 Appropriation.)
$10,385,655 Superfund Expenditures (91%FY01 Appropriation)
$ 7,334,904 Carryover from FY 00, $11,767,797 Carryover to FY02
252 Mgt FTE Used (91.6% FY01 Budget)
71.3 SF FTE Used (75.8% FY01 Budget)
Cost/FTE: Object class MGT SF Avg. Time: % Direct Elapsed, Staff on Products
> PCB $ 90.7 $102.6 Direct Time Charged of Total Available 42.8%
> Travel $ 5.0 $ 5.9 Average Direct Staff Hours/Investigation 152
> Admin. Exp $ 2.2 $ 2.5 Average Elapsed Months/Investigation 33
> Admin. Cont $ 4.9 $ 6.9 Average Direct Staff Days/Audit 329
> WCF $ 4.5 $ 8.5 Average Elapsed Months/Audit 14.3
EPA Management
Challenges
Reported by OIG
Accountability (including cost accounting)
Automated Info Systems Security
Oversight/Use of Asst Agreements
Backlog of NPDES Permits
Resources for Competencies/Human capital
Quality of Laboratory Data
IRM Management
Working Relations with States/Partners
Results Based Info Technology/Project Mgt
Protecting Critical Infrastructure from Attacks
FMFIA Weakness
in OIG
Human Resources System (Recruitment Strategy)
Career Management (Skills Gaps)
Performance Agreements
Organizational Position Descriptions
Info Tech Long-Range Plan & Purchases
IGOR System
OIG Intranet
Records Management
Business Planning Process
Project Management/Accountability
Cost Accounting
Assignment Follow-up
-------
OIG Vision
We are agents of positive change striving for continuous improvement in
our Agency's management and program operations and in our own office.
We contribute to environmental quality, human health, and good
government through problem prevention and cooperative solutions.
Mission
# Promote economy, effectiveness and efficiency within the agency
# Prevent and detect fraud, waste, and abuse in agency programs and operations
Goals (FY2001 - 2005)
1. Contribute to
improved environmen
quality and human
health
2. Improve EPA's
management and
program operation'
3. Produce timely,
quality and cost
effective products
and services that
meet customer needs
4. Enhance
diversity,
innovation,
teamwork and
competencies
Objectives
O Influence significant
programmatic changes to
legislation, regulations, policy,
processes and practices that
have a positive impact on the
environment and human health
D Identify and recommend
solutions reducing the highest
environmental risks
O Identify best practices in
EPA, states and other Federal
agencies that have directly
contributed to improved
environmental quality and
human health impacts
D Identify opportunities
for improving economy,
efficiency and
accountability in EPA
programs and operations
D Improve the integrity of
EPA programs and
operations by identifying
actual and potential
vulnerabilities for fraud
and risk reduction
O Help EPA resolve its
major management
challenges
n Provide the right
products, at the right time,
to the right customers, at
the right cost
O Build infrastructure,
relationships and
partnerships to leverage
change
D Increase professional
image and demand for
products and services
O Improve organizational
systems and business
processes
O Increase recognition
for creativity, innovation
and teamwork
O Promote continuous
learning and
demonstrated
competencies in EPA
programs, professional
skills, technical skills and
leadership skills
Strategic Areas of Emphasis
1. Air
2. Water
3. Safe Food
4. Waste Management
5. International Issues
6. Environmental Data
7. Scientific Research
8. Enforcement &
Compliance Assistance
1. Financial Management
& Cost Accounting
2. Assistance Agreements
3. Contracts
4. Computer Security
5. Human Capital
6. Systems
1. Customer Focus
2. Partnering
3. Business Planning
(product/process design)
4. Integrated Measurements
(performance linkages)
5. Outcome Orientation
(Begin with the end in mind)
6. Activity Based Costing
(project management)
1. Human Capital (skills &
development)
2. Communications (trust &
understanding)
3. Organizational
Realignment (matrix
management)
4. Integration of High
Performance Culture
5. Integrated Knowledge
Information Systems
-------
FY 2001 EPA OIG Performance Objectives by Goal FY 2001 FY 2001 Results %
Targets Target
Achieved
Goal 1 : Improvements/changes/decisions/actions influencing
environmental impacts or preventing loss
Goal 1 : Environmental risks identified, reduced or
eliminated from solutions and enforcement actions
Goal 1 : Best Practices identified/implemented that can
improve environmental & human health
Goal 1: Other - Recommendations for environmental action
Goal 2: Return 100% of investment in OIG budget from
questioned costs, savings, recoveries and fines
Goal 2: Reduce the risk of loss to resources, and integrity of
programs and operations from enforcement actions
& certifications
Goal 2: Assist EPA improve implementation of GPRA
Goal 2: Other - Recommendations, best practices identified
& implemented, examples of practice changes, new
FMFIA/ Management Challenges identified,
corrected management challenges to improve
management and operations
Goal 3 : Products/services that provide customer satisfaction
Goal 3 : Establish collaborative relationships that leverage
resources and changes with federal and state partners
Goal 3 : Increase image & demand for OIG products/services
Goal 3: Other - Contacts with OMB/Cong, requests to testify
meet milestones/
plans/expectations recommendations cases and
advice are accepted! sustained
Goal 4: Promote use of technology - establish baseline
Goal 4: Enhance recognition for creativity, teamwork and
diversity - establish baseline
Goal 4: Enhance staff competencies - establish baseline
25 Actions &
Impacts
15 Risks
15 Best
Practices
$43 Million
50 civil,
criminal
admin actions
& certifications
46 percentile
77% rating
(2 fed, 5 states)
7 partnerships
30% of work
requested
75% avg
94% parity
1.5 award/emply
18 new IG tech
19 multi teams
% compliance
with prof. stds.
(2 legislation, 3 regulatory,
48 policy/process changes,
6 examples envir. impacts)
59 total action/impacts
(5,894 risks identified
24 risks reduced)
5,918* total risks
(9 identified, 3 transferred)
12 total best practices
34 Recommendations made
$72.4 Million
(98 criminal, civil, admin
actions; 9 certifications.)
107 total actions
25 percentile
148 recommendations made
17 best practices identified
63 process/practice
changes made
4 corrected management
challenges/FMFIA
80% satisfaction rating
32 collaborative efforts
56% of work requested or
required
11 OMB/Congressional
contacts
4 requests to testify
60% milestones met
72% work done electronic.
77% available electronic 'ly
94% parity with civilian
work force
515 awards
18 new techniques in OIG
19 multi-discipline teams
236%
39,293%
80%
168%
214%
121%
104%
457%
186%
: one type of risk was reported to exist at each of over 5600 waste-water treatment facilities
-------
Linking Our Work to Outcomes and Impacts
All of our work is planned based on the
anticipated value toward influencing
resolution of the Agency's major
management challenges, reducing risk,
improving practices, program operations and
saving tax payer dollars, leading to positive
environmental impacts and the attainment of
EPA's Strategic Goals.
Environmental
Impact
tal
OIG Products
& Services
ommendatio
Advic
Agencv Action
Creating a Nexus or Link Between OIG Products/
Services and Environmental Impacts/Goals
'Start With the End In Mind", using a Customer Focused approach.
Linkage of OIG Business Line Measures/Results from Products & Services to Improved EPA Operations &
Impacts: Results compared to resources used, = Return on Investment
Outputs
(Audit, Evaluation, Advisory,
Investigative Products & Services)
Agency Intermediate Outcomes
(Catalysts)
Agency goals/Outcomes
and Impacts
Questioned Costs/Savings
Recommendations/Opinions
Advice/Analysis/Projects
Indictment/Convictions
Civil/Administrative
Fines/Restitutions
Reports/Briefings
Evaluation Conclusions
Legislative Change
Regulatory Change
Policy Change
Practice Change
Enforcement Actions
Industry, Grantee or
State Monitoring
$s Recovered, Offset or Avoided
Improved Efficiencies
Improved Effectiveness
Improved Controls
Increased Compliance
Improved Reporting
Risk Reduction
Improved environmental &
Health Results
Above is an illustration of how
OIG products and services and
accountability feedback for the
or service focus and design.
our work is measured from outputs to outcomes, creating a nexus or linkage between
environmental impacts and goals. Customer Satisfaction surveys provide additional
quality, timeliness and value of our work, as well as information for future product
-------
Goal 1. Contribute to improved environmental quality and human healtl
Objectives
Influence significant programmatic changes to legislation, regulations, policy,
processes and practices that have a positive impact on the environment and
human health. Annual Targets: By 2001, the OIG will recommend 25 improvements
across EPA 's environmental goals. Actual Reported: 59
Identify and recommend solutions for reducing the highest environmental risks.
By 2001, the OIG will identify and recommend solutions and/or enforcement actions
that reduce or eliminate at least 15 risks of loss to health and environmental quality
across EPA 's environmental goals. Actual Reported: 5,918
Identify best practices in EPA, states and other Federal agencies that have directly
contributed to improved environmental quality and human health impacts. By
2001, the OIG will identify at least 15 best and innovative environmental practices and
actions that can be applied and transferred to or within EPA. Actual Reported: 12
Measures of Results and Progress
34 # Recommendations for Environmental Improvement
2 # Legislative Changes/Decisions
3 # Regulatory Changes/Decisions
48 # EPA Policy/Directive/Process Changes/Decision
5,894* # Environmental Risks Identified
24 # Environmental Risks Reduced/ Eliminated
9 # Best Practices Identified
3 # Best Practices Transferred/Implemented
6 # Examples of Environmental Improvements/Impacts/Behavioral Change
-------
Examples of Selected Results Demonstrating Progress on Goal 1.
Objective: Improvements/changes/decisions influencing positive environmental impacts or
preventing loss
* State of Missouri is implementing plans to restore watersheds, and developing objectives
and measures of pollution abatement.
> State of New York took actions to renew focus on combined sewer overflows, and
made changes in its permit prioritization system.
> Audit influenced signing of a multi-agency Memorandum of Understanding (MOU) to
enhance cooperation for water quality standard consultations.
Objective: Risks to environmental quality and health reduced/eliminated from
solutions/enforcement actions
* Asbestos workers found to be unauthorized aliens, and not properly trained in asbestos
removal. Suspended person selling asbestos training certificates to untrained abatement
workers. Actions reduced health risks to building inhabitants and workers.
* Utility sentenced for violation of Clean Water Act for discharging nitrogen directly into a
bay without adequate treatment.
> Regional management agreed to significant changes in Region 6's process for evaluating
Supplemental Environmental Project (SEP) proposals that will ensure that only projects
with significant environmental/human health improvements are accepted.
Objective: Best practices and recommendations identified that can be applied to influence
environmental performance
* New York developed two new Best Management Practices for combined sewer
overflow permitting and improvements.
* Audit recommended that guidance to states should be developed concerning insurance
policies used to establish Resource Conservation and Recovery Act (RCRA) financial
assurance. Ten captive insurance policies were identified in 9 states, which present
environmental risks.
> Recommended changes to the process that requires facilities to report toxic releases into
surface waters. A large number of facilities were still making regular releases.
-------
GOAL 2. Improving EPA s Management and Program Operati,
Objectives
Identify opportunities for improved economy, efficiency, and accountability in
EPA programs and operations. Annual Target: By 2001, the OIG will identify
potential cost savings, recoveries and fines equaling at least one time the annual dollar
investment in the OIG ($43 Million). Actual: $ 72.4 Million
Improve the integrity of EPA programs and operations by identifying and reducing
actual and potential vulnerabilities for fraud and risk reduction. Annual Target: By
2001, the OIG will reduce and prevent the risk of loss from fraud by contributing to at
least 50 cumulative criminal, civil, or administrative actions. Actual: 107
Help EPA resolve its "major management challenges." Annual Target: By 2001, the
OIG will spend at least 5 percent of available time identifying EPA's Major Management
Challenges and advising the Agency on eliminating each management challenge within
three years of its designation as such, and achieving a rating in the top 51 percent of all
agencies for implementation of the Results Act. Actual: 25 percentile GPRA rating
among all Agencies.
Measures of Results and Progre.
S35.1M Questioned Costs
S32.1M Recommended Efficiencies, Costs Saved or Avoided
S5.2M Fines/Recoveries/Restitutions/Collections
98 # Criminal/Civil/Administrative Actions
17 # Best Practices Identified/Implemented
63 # Examples of Process/Practice Changes/Decisions
148 # Recommendations for Management Improvement
2 # New FMFIA/Management Challenges identified
21 % EPA Improvement In GPRA Rating
4 # Corrected Management Challenges/FMFIA Weaknesses
80 % Customer Satisfaction with EPA Service Quality (audit)
9 # Certifications/Verifications/Validations
-------
8
Examples of Selected Results Demonstrating Progress on Goal 2.
Objective: Return on investment from opportunities for improved business processes,
practices, and savings
* Identified two projects that planned to claim $220,000 of unallowable match.
* Funds claimed by the Napoleon, Ohio schools under an EPA asbestos grant and loan
were questionable. EPA should recoup about $2 million.
> About $1.7 million in fines and restitution returned to several environmental trust funds.
Objective: Action reducing/eliminating risk of loss or improvement in operational and
program integrity
* Suspension of company (contractor) and individual.
> Letters of reprimand and warning for three Agency employees.
* Change in wording of 104(e) letter and forms, as a result of an investigation.
Objective: Identification of, and help resolving Management challenges; best practices
promoting management improvements; management recommendations, & EPA
implementation of GPRA
* Recommendations made concerning the need for improving Agency evaluations of
proposed Supplemental Environmental Projects (SEPs), related consent decree
requirements, and documenting SEP completions.
> Two good program practices identified: (1) issuing compliance letters to facilities which
have adequately completed all actions required in enforcement instruments, and (2)
improving file documentation through the use of Technical Review Action sheets.
> Recommendation based on review of Agency electronic payment system resulted in
payment system now operated directly with the Agency's accounting system. This
eliminates faxing of payment forms from the regions to the Las Vegas financial
management center for processing and entry of payment data into two systems, versus
one system; and other data entry improvements.
> Management Accountability Report review for Region 10 suggested corrective actions
that should improve the Regions' award and post-award monitoring procedures for
General Assistance Program grants.
> EPA improved its GPRA Performance Reporting Rating, as reported by the Mercatus
Institute and the Senate Government Affairs Committee, from 11th (45.6 percentile) to 6th
(25 percentile) out of the 24 agencies with Chief Financial Officers, specifically from OIG
recommendations on disclosure of, and reference to, Management Challenges throughout
the Agency's Annual Report.
-------
GOAL 3: Producing Timely, Quality and Cost Effective Products am
Services that Meet Customers Needs
Objectives
Provide the right products, at the right time, to the right customers, at the right cost.
Annual Target: By 2001, the OIG will develop and apply market and business knowledge
to achieve an overall customer satisfaction rating of 77% for its products and services.
Actual: 80% composite customer service rating, 60% of project milestones met
Build infrastructure, relationships, and partnerships to leverage change. Annual Target:
By 2001, the OIG will develop cooperative activities with 2 federal agencies and 5 states
(7 partnerships) Actual: 32
Increase professional image and demand for products and services. Annual Target: By
2001, at least 30% of the OIG work will be customer requested, including testimony and
speeches. Also, OIG staff will achieve an 82% rating for constructive, professional, and
courteous attitude. Actual: 56% of OIG work is required/requested, 85% rating for staff.
Measures of Results and Progress
80 % Customer Satisfaction Ratings (77.5% products/services, 85.3% staff)
60 % of Engagement Letter/Project/Budget Expectations/Milestones Met
4 # of Requests to Testify (Congress or judicial)
32 # of Collaborative Efforts, States/Feds
11 # of Contacts With Congress/OMB Staff
56 % of OIG Work Requested/Required
-------
10
Examples of Selected Results Demonstrating Progress on Goal 3.
Objective: Customer satisfaction with OIG products, services and attitudes
> OIG 2001 External Survey of customer feedback shows a favorable improvement in 13
out of 14 rating questions since the 1998 Survey.
> Customer survey completed at conclusion of Assistance Agreement Awarded to the
Center for Chesapeake Communities report indicated positive results from our product,
services, and attitudes.
> Southern Audit Division team received a letter of commendation from the Agency's
Grants and Debarment Office for balanced reporting and professionalism.
Objective: Build collaborative relationships to leverage change with federal and state
partners
* Joint investigation case worked with Defense Investigative Service, National Aeronautics
and Space Administration OIG, and the Internal Revenue Service's Central Investigation
Division.
* Collaboration with the Agency's Office of Enforcement and Compliance, Office of Water,
regions, and states on a state enforcement audit of Clean Water Act dischargers.
> Investigation on repayment of funds worked with Virginia's Department of
Environmental Quality and Virginia State police.
Objective: Develop professional image, quality and demand for OIG products and services
* State Revolving Fund audits and consulting-type engagements with several regions
(Western Audit Division, Central Audit Division) produced good feedback and demand
for further services.
> Mid-Atlantic Audit Division senior managers pleased with complex audit on Tranguch
Superfund site. Collaboration with State of Pennsylvania, and Pennsylvania State Senate.
* Central Audit Division met engagement letter expectations, project plan, and budget
estimates. Developed good working relationship with region and the Agency's Office of
Water program.
-------
11
Objectives
Improved organization systems and production processes. Annual Target. By 2001,
the OIG will establish a baseline for the use of technology, and use organizational designs
that apply multi-discipline resources for high impact solutions. Actual: 75% work
done/accessible electronically, 19 multi-discipline teams
Increased recognition for diversity, innovation, and teamwork. Annual Target. By
2001, the OIG will establish a baseline for recognizing employee innovation and
teamwork. Actual: 515 awards, 94% parity with civilian workforce, 18 innovations
implemented
Improved continuous learning and demonstrated competencies in EPA programs,
professional skills, technical skills and leadership skills. Annual Target: By 2001, the
OIG will establish a baseline for increasing its skills, abilities, and competencies.
Actual: (baseline data not completed)
Measures of Results and Progress
94.2 % Staff In Parity with Civilian workforce
19 # Assignments By Multi-disciplinary Teams
515 # Awards for Creativity, Innovation & Teamwork
72 % Work Performed Electronically
77 % Products Available Electronically
18 # Innovative Techniques/Processes, Implemented (by OIG)
-------
12
Examples of Selected Results Demonstrating Progress on Goal 4.
Objective: Improve organizational systems and processes through technology and multi-
discipline approach
* OIG engineers participated in several audit teams to provide technical advice.
* State Enforcement of Clean Water Act Dischargers audit incorporated work between
Southern, Central, and Western Audit Division office teams for engineering and technical
writing skills.
* Computer specialist contributed presentation skills for Mid-Atlantic Audit Division
Assistance Agreement Awarded to the Center for Chesapeake Communities audit.
> Central Audit Division team of evaluators, auditors, engineer and scientist collaborated on
an Agency water program audit.
Objective: Increase and recognize organizational and staff achievement, innovation,
teamwork and diversity
* Awards in various categories distributed to OIG staff throughout FY 2001, and
recognized at national training session in Phoenix, AZ.
Objective: Improve staff competencies in EPA programs, technical, leadership, &
professional skills
> Training for technical, leadership and professional skills was taken by all OIG staff during
FY2001.
-------
13
Customer Service Results
Actual OIG External Customer Survey Results Compared to Strategic Targets
(Based on 100% scale)
Year
Questions About:
OIG Products/Services
OIG Staff
TOTAL
1998/1999
Actual
73.5%
77%
74.6%
2000
Actual
74%
79.6%
76%
2001
Actual
77.5%
85.3%
Target
2002
Target
76%*
84%*
79%
2005
GOAL
80%
90%
85%
Customer/Client Attribute Survey Results FY 1999-2001
Year
N Factor
(# of responses)
Question 3
Question 4
Question 5
Question 6
Question 7
Question 8
Question 9
Question 10
Question 11
Question 12
Question 13
Question 14
Question 15
Question 16
Question 17
TOTAL
Attributes
(63% response rate in FY 98/99,
58% response rate in FY 2001)
Respondent knowledge of IG Act/Mission
OIG Products/Services are: Factually accurate
Objective and balanced
Relevant & significant
Useful for decisions & actions
Recommendations or advice are practical
Logical and understandable
Timely
Responsive to needs/requests
Contributes to EPA goals
OIG Staff : Are Professional & courteous
Knowledgeable of programs
Communicate clearly
Seek comments & clarification
Build constructive relationships
1998/99
57
54%
75%
73%
75%
70%
70%
75%
75%
75%
73%
87%
68%
75%
77%
77%
74.6%
2000
26
X
75%
72%
70%
75%
70%
78%
78%
77%
73%
88%
75%
80%
78%
77%
76%
2001
90
50%
78%
77%
82%
78%
72%
78%
73%
77%
77%
92%
78%
85%
82%
85%
80%
-------
14
Narrative Customer Survey Questions and Responses
18. How can we imvrove OIG products/services, processes or results? Number of
Comments
Program knowledge - technical knowledge; more needed by OIG. 11
Recommendations - comments about being/not being too strong, not being
focused enough, not being flexible enough, too many in number 6
Timeliness - thoroughness not adequate in auditing and/or reporting,
assignments started are not getting finished 8
Financial - focus more on financial issues; grants, contracts, agreements &
internal EPA funds for programs 2
Communication - better notification about audits/services, more involvement in
final product 7
State relations - more knowledge of what States may need, be more sensitive to
regional and State relationships. 2
19. With what products or services, and in what yrosram areas can the
OIG best serve EPA?
Root cause analysis - system-wide analysis, weaknesses analysis, program
goals/issues.
Standard - policy and management audits.
State relationship activities - issues, oversight, partnering, permits
assistance/audits.
Progress, followup and knowledge about whether implementation of
recommendations actually took place.
Financial issues - concentrate on financial audits
Communication w/Congress, States, public
Proactive assistance - partner with programs, train programs about
auditing/evaluating/investigating, share investigative skills, more
assist/consultation for programs
Number of
Comments
5
2
3
8
3
20. How do OIG vroducts or services add value? Ifvou do not believe
thev add value, whv not?
Financial, grant, contract, agreement, SRF audits/investigations invaluable to
EPA financial programs
OIG defends/sells Agency positions to Congress, public, other agencies, etc.
Identify EPA program deficiencies and issues, on internal controls, regulations,
policies, allegations, etc.
Provide independent/balanced reviews
Help with State/Region credibility on issues
Help with collaboration/positive working environment with EPA programs,
especially using consulting and assistance
Focus on Issues - OIG helps programs focus on major issues, goals, needs,
financial statements
Number of
Comments
11
5
35
11
9
9
12
-------
15
Other Comments By Customers and Clients
Complaints
OIG has no leverage for change.
We leave Regions in weak position w/States, esp. on enforcement.
Staff did not keep appointments.
Had better relationship with OIG 3 years ago.
Disagreements between program technical staff and OIG technical staff.
Review OIG competencies. Auditees should be able to rank OIG staff.
OIG second-guessed a Regional Administrator's decision.
Would be more productive if we would coordinate w/Programs for common goals.
* Be more flexible with recommendations.
* Don't need criticism that States don't follow Federal guidance.
* Not enough leverage from OIG to ensure changes in program "causes.'
* Faster closure process; re: old reports.
Compliments
Very professional staff. Courteous. Cooperative.
Auditors added value to RCRA program.
Liked ability to review drafts and pre-final discussions, and make changes/flexible.
Willing to discuss recommendations to add value.
Informal management reviews helpful; and assists helpful.
Clarified goals and shared information for better final product.
Coordinated well with program staff during audit.
Open to requests for advice and assistance.
Sought feedback.
Good relationship with programs.
Face-to-face meetings good. Also periodic meetings good.
OIG does thorough job in review.
Well written reports.
Good working relationship with Regional and State staffs.
* Good advance notification about audit.
* Audit team knowledgeable about State programs.
* Did good collaboration with States.
-------
16
FY 2001 FINANCIAL STATEMENTS FOR OIG, (Figures From IFMS - Unaudited)
(Source: EPA Integrated Financial Management System)
Budget Object Class FY2000 Carryover FY2001
Funds Applied in Approp.
FY2001 Expended
before
reprogramming
Total FY 2001 Total FY2001 Cost per
Expenditures Expenditures FTE in OOOs
(carryover +approp) as % of
FY2001
Appropriation
MANAGEMENT
10 PC&B
21 Travel
29 Prog. Expenses
30 Admin. Expenses
32 Prog. Contracts
33 Admin Contracts
35 WC Fund
41 Grants
Reprogrammed PC&B
TOTAL OIG
MANAGEMENT
SUPERFUND
10 PC&B
21 Travel
28 Site Travel
29 Prog. Expenses
30 Admin. Expenses
32 Prog. Contracts
33 Admin Contracts
35 WCFund
41 Grants
Reprogrammed PC&B
TOTAL OIG
SUPERFUND
$ 832,770
19,691
283,771
1,082,157
799,604
1,123,113
0
1,223,239
$ 5,364,345
$ 347,686
21,012
10,330
101,237
284,462
355,644
604,753
0
245,435
$ 1,970,559
$22,853,190
422,042
9,696
279,708
1,304,829
431,342
0
0
$25,300,807
$7,318,194
75,888
602
2,457
74,248
806,207
137,501
0
0
0
$ 8,415,096
$22,853,190
1,254,812
29,387
563,479
2,386,986
1,230,946
1,123,113
$ 0
1,223,239
$30,665,152
$7,318,194
423,574
21,614
12,787
175,485
1,090,669
493,145
604,753
$ 0
245,435
$10,385,655
1
84%
78%
319%
89%
134%
48%
90%
79%
105%
432%
556%
111%
118%
76%
91%
$90.7 |
$ 5.0
$ 2.2
$ 4.9
$ 4.5
$102.6
$ 5.9
$ .3
$ 2.5
$ 6.9
$ 8.5
FY 2001
GRAND TOTAL
$ 7,334,904 $33,715,903 $41,050,807
-------
17
FY 2001 PC&B COST PER FTE BY OFFICE (combined Management & Superfund)
OFFICE
Audit
Human Capital
Program Evaluation
Investigations
Mission Systems
Plann'g Analysis Results
Counsel
Immediate IG Office
TOTAL
TOTAL FTE USED
7.9
249.0
13.2
11.6
16.4
13.1
6.6
5.5
o oo o
JZJ.J
TOTAL PC&B USED
$ 850,384
22,510,595
1,151,784
1,362,972
1,366,012
1,254,601
640,152
796,007
$ 29,932,507
COST Per FTE
$ 107,643
90,404
87,256
117,498
83,293
95,771
97,000
144,729
$ 92,584
FY 2001 FTE USAGE by APPROPRIATION
Approp Account
Management
Superfund
TOTAL
FY 01 FTE Available
275
94
369
FY 01 FTE USED
252.0
71.3
323.3
% FTE Budget USED
91.6%
75.8%
87.6%
FY 2000 APPROPRIATION - FINAL UTILIZATION RATE
Appropriation
Account
Management
Superfund
TOTAL
FY 2000
$ Appropriation
$32,329,700
$11,000,000
$43,329,700
FY 2000
$ Appropriation Used
$32,210,591
$10,861,455
$43,072,046
2000 $s Lapsed
$ 119,108
$ 138,545
$ 257,654
% $s
Used
99.6%
98.7%
99.4%
FY 2001 CARRY-OVER AVAILABLE to FY 2002
Appropriation
Account
Management
Superfund
TOTAL
FY 2001
$ Appropriation
$34,019,000
$11,474,700
$45,493,700
FY 2001
$ Appropriation Used
$25,300,807
$ 8,415,096
$33,715,903
FY 2001
$ Carry-Over to FY 2002
$ 8,718,193
$ 3,059,604
$11,767,797
-------
18
FY 2001 STAFF (FTE) USAGE BY OFFICE
(Source IGOR 2/02)
Office 1 Management 1 Superfund 1 TOTAL
Audit (OA)
Human Capital (HC)
Program Evaluation (PE)
Investigations (OI)
Mission Systems
Planning Analysis & Results (PAR)
IG Counsel
Immediate Office of IG
6.1
189.6
11.2
9.8
13.1
10.5
5.4
4.4
1.9
64.8
1.5
2.1
3.2
2.6
1.3
1.0
8
254.4
12.7
11.9
16.3
13.1
6.7
5.4
TOTAL FTE | 250.1 | 78.4 | 328.5
Distribution of Staff Time: Direct vs Indirect
Total: 328.5
Indirect PE,OA,OI,HC
44.5%
IG Counsel
2.0%
Immd Office
1.6%
Mission Syst
5.0%
PAR
4.0%
Direct Investigative Products
8.2%
Direct Audit Products
34.6%
The OIG used 42.8% of reported available staff time on direct business line products,
exclusive of memo reports, Single Audit Act reviews, audits performed by another
agency, and other briefings and consultative services.
-------
Time Elapsed to Issuance/Completion of Audit Products FY 2001
(Source IGOR 2/02)
This analysis shows the calendar time elapsed between when Audit Products/Services are started until completion/issuance, as an indicator of
performance timeliness. Average elapsed time for all OIG audit products in FY 2001 is 14.3 months, Average staff time charged for all
OIG audits is 328.9 days. Percent of Audit Products/Services with Time Elapse Data* = 55% (33 of 60).
Performance Audits (P) I Elapsed I Staff I Financial Related
I Time in I Time I Audits (1)
I Months I in Days
Elapsed I Staff I Special Audits (S)
Time in I Time I
Months I in Days I
Elapsed I Staff
Time in I Time
Months I in Days
000016 ITAS - EPA Computer
Security Program
000003 CAD -Region 7
Nonpoint Source
000004 CAD - Region 8 RCRA
000001 EAD - Combined Sewer
Overflows
000002 EAD -Children's Health
Risk Initiative Programs
000007 HAD - RCRA Financial
Assurances
000008 MAD - Competitive
Practices for Assistance Awards
000012 MAD - NPDES Program
000017 NAD - Petosky SF Site
000006 NAD - Enforcement
Agreement Compliance
0000 1 1 NAD - SF Interagency
Agreements
000014 SAD - SEP Survey
000005 SAD - Assist.
Agreements to Non-Profit Orgs.
000013 WAD -State
Enforcement Effect
TOTAL 14
AVERAGE TIME
7
21
15
23
10
19
19
16
27
26
13
18
12
19
245
17.5 Mo
449
303
482
524
446
930
917
352
699
865
549
654
123
984
8277
591 days
000074 EAD -E&E 1994
Incurred Cost audit
000101 MAD - Chesapeake
Assistance Agreements
000120 NAD - Chicago Schools
000073 NAD - Napoleon City
Schools
000091 NAD - Earth-Tech
Incurred Cost audit FY 96
000190 NAD - Earth-Tech
Incurred Cost audit FY 97
000192 NAD - Earth-Tech
Incurred Cost audit FY 98
000075 WCD - E.H. Pechan
00003 1 WCD - Guardian Env.
000202 WCD - ICF Floorcheck
000197 WCD - ICF Consulting
Financial Capability
TOTAL 11
AVERAGE TIME
8
12
12
27
4
11
11
11
39
5
12
152
13.8Mos
458
245
131
122
31
49
41
49
91
73
42
1332
121Days
000002 HAD - Joint Review
Mgmt. Of SF Collections
000004 HAD - Region 5 SF
Accounts Receivable
000003 HAD-Reg. 1 Accts.Rec.
000005 HAD-Reg. 6 Accts.
Rec.
000007 HAD - Libby SF site
000010 NAD - Ecorse Creek
0000 1 1 NAD - Disclosure of
EPA Penalty Info to IRS
000001 WAD-South Carolina
SF Recipient
TOTAL 8
AVERAGE TIME
10
10.5
10.5
10.5
9
8
6
9
73.5
9.2 Mos
4
4
4
4
865
210
60
93
1244
155 Days
Elapsed time in this analysis starts from "Notification Date." Only 1 out of 95 assignments completed in FY 2001 had a reported "Start Date.''
-------
Time Elapsed to Issuance/Completion of Investigation Products FY 2001 (Source IGOR 2/02)
This analysis shows the calendar time elapsed & staff time charged from the opening to closing of Investigation cases closed as an indicator of
performance timeliness. For 54 investigation cases closed in FY 2001, average elapsed time is 32.6 months, average staff time is 151.6 hours.
Investigation I Elapsed I Staff
Cases I Time in I Time
by Division \ Months I in Hrs.
Central
000024
000058
000061
000062
000073
000080
000082
000863
000977
001015
65
48
2.5
46
37
36
32
22.5
24
18
29
1284
69
0
20
15
68
69
89
24
Central TOTAL
Cent. AVERAGE
Eastern TOTAL I 196
East. AVERAGE
Eastern
000080
000115
000952
000907
000601
001079
000861
001446
49
35
27
23
20
10
27
5
103
4
123
1305
674
123
126
0
Investigation I Elapsed I Staff
Cases I Time in I Time
by Division \ Months I in Hrs.
HQ
001132
000396
00642
HQ TOTAL
HQ AVERAGE
2
6
5
1
4
0
158
178
Mid-Atlantic
00069
000129
000130
000139
000151
001019
000521
000834
000096
000101
000864
000865
000013
001531
Mid TOTAL
Mid AVERAGE
61
37
45
37
37
31
16
38
36
29
29
15
424
53
13
83
254
439
38
69
135
355
12
115
Investigation I Elapsed I Staff
Cases I Time in I Time
by Division \ Months I in Hrs.
Western
000071
000172
000173
000176
001218
001532
000838
001392
000126
000183
000760
001064
001583
000074
000067
000068
000190
49
40
41
31
28
17
14
11
40
64
23
364
8
78
65
55
37
0
252
178
273
411
220
66
129
118
42
146
34
0
0
0
7
126
Western TOTAL
West. AVERAGE
-------
21
Fiscal 2002 Goal Targets with Map to Supporting Measures
Goal #1. Contribute to improved environmental quality and human health
Target: 50 improvements/changes/decisions/actions
implemented, contributing to or influencing positive
impacts on environmental quality and/or human health
Target: 15 environmental or health risks reduced or
eliminated from solutions, enforcement actions, certifications
Target: 75 recommendations, risks or best practices
identified that can influence environmental/health actions &
Legislative changes/decisions
Regulatory changes/decisions
EPA policy, directive or process change/decision
Examples of environmental improvements/impacts
Best environmental practices implemented
Environmental risks reduced/eliminated
Certifications/validations/verifications
Recommendations for environmental improven
Environmental best practices identified
Environmental risks identified
Goal #2. Improve EPA's management, accountability and program operations
Target: 100%($45.9 Million) return from improved
business practices & potential savings, recoveries & fines
Target: 75 actions reducing or eliminating risk of loss, or
improving efficiency and integrity in management of
resources, operations & programs, resolving Mgt
Target: 150 recommendations, best practices, FMFIA
or management challenges identified to promote action
Questioned costs
Recommended efficiencies, costs saved or avoided
7ines. recoveries, restitutions, collections
Criminal, civil, administrative actions
Certifications/validations/verifications
Examples of process/practice changes/decisions to
reduce risk & improve integrity or efficiency
Corrected Management Challenges/GPRA rating
Best practices implemented
Recommendations for management improvement
New FMFIA/ Management Challenges identified
Best Practices identified
GOAL #3. Produce timely, quality & cost effective products and services that meet customer needs
Targets: 79% Overall OIG Customer Satisfaction
3 New Federal, 5 New State partnerships
35% OIG work requested/required externally
66% Milestones/plans met
Customer survey satisfaction rating
Collaborative partnerships
Requested assignments
Cong. Meetings/requests to testify
Cost/time per assignment, Milestones met
Cases/Recommendations/Advice accepted
GOAL #4. Enhance diversity, innovation teamwork and competencies in the OIG
Targets: 10% Increase in application of technology (82%)
diversity (96% civilian WF), innovation (20 new techniques),
multi-discipline teams (21) & recognition (1.65
Products by multi-discipline teams
Products done and available electronically
Staff in parity with civilian workforce
Innovative techniques/practices in OIG
-------
22
NOVEMBER 1, 2001 Appendix 1
MEMORANDUM
SUBJECT: Fiscal 2001 Report on Management Controls
TO: Christine Todd Whitman
Administrator
I am submitting this annual report from the Office of Inspector General (OIG), in
compliance with the requirements of Office of Management and Budget (OMB) Circular A-123,
Management Accountability and Control, and the Federal Managers' Financial Integrity Act
(FMFIA). This report also complies with the internal control requirements of OMB
Circular A-130, Management of Federal Information Resources.
ASSURANCE STATEMENT
I have taken the necessary measures to assure that we have evaluated our internal controls
in accordance with the guidance provided by the Office of the Comptroller. Based on our
evaluation process and my personal knowledge, I believe that:
The internal controls in effect in the Office of Inspector General on September 30, 2001,
taken as a whole, provide reasonable assurance of compliance with the objectives of FMFIA. The
OIG has, however, identified three broad areas this year as OIG-level weaknesses, two of which
contain related component issues identified as office-level weaknesses in FY 2000.
These areas pertain to:
Human Resources Strategy
Information Technology Strategy
Business Systems
We have provided a brief description of these issues and the actions we have taken or plan
to take under the Office-Level Weakness section of this report. We also plan to develop a more-
detailed Management Action Plan (MAP), with milestone dates and action officials.
MANAGEMENT CONTROL ACCOMPLISHMENTS
The OIG conducted vulnerability assessments in each of its assessable units during Fiscal
Year (FY) 2001. The OIG also conducted Quality Assurance Reviews in two of its investigative
divisional offices and one audit divisional office during FY 2001. We designed the reviews to
evaluate the: (1) extent to which our office is complying with Government Auditing Standards
and/or OIG policies and procedures; and (2) efficiency and effectiveness of OIG management,
programs, and operations. Our assessment steps selectively monitored how leadership and high
-------
23
performance organizational concepts were implemented as a system of interrelated components to
improve and achieve our desired performance results. Our reviews reported recommendations, as
appropriate, to address issues identified during the review work; however, we did not identify any
issues which were significant enough to be considered as material control weaknesses.
In addition to the above, the OIG took the following steps during FY 2001 to improve
management controls:
Implemented a Management Action Plan to improve the OIG's systems, processes, and
office environment. The plan categorized the eight office-level weaknesses identified in
FY 2000; identified goals, action items, accountability, and milestone dates; and
established a process tracking our accomplishments and progress.
Developed and began implementing a Diversity Action Plan to improve workforce
diversity and achieve a staff that is more reflective of the diversity of our national
workforce.
Clarified and reinforced investigative, management, and administrative policies,
procedures, and requirements.
Established a project tracking system within the Office of Counsel to monitor and
identify projects and their status.
Refined the OIG's internal and external outreach strategy, which includes customer
focus, marketing and partnerships.
Reviewed administrative and operational issues to clarify and reinforce policies,
procedures, and management control requirements and established workgroups to
revise policies and procedures, as needed.
Continued organizational-wide training in concepts of personal and organizational
leadership and performance excellence.
Improved OIG budgeting, analysis and reporting requirements under the Government
Performance and Results Act, FMFIA, and the Inspector General Act.
Reorganized to a matrix management structure with regional resource centers for
audits, evaluations and investigations; and established a new Office of Human Capital
to manage, coordinate and provide supervision to the resource centers.
Hired a systems/science advisor to provide expert and independent advice on issues
relating to integration of business systems and those relating to the scientific and
technical problems facing EPA, the results of specific scientific efforts and the impact of
emerging environmental problems.
Provided advice, presentation and instruction for improving planning, measurement and
accountability to OIG, EPA and Government-wide audiences.
Successfully resolved three FY 2000 office-level weaknesses: Freedom of Information
Requests, Internal Review and Tracking System, and Diversity Awareness/Sensitivity.
-------
24
OFFICE-LEVEL WEAKNESSES
The OIG has identified three broad areas as office-level weaknesses for FY 2001. In each
area, we lack clearly articulated plans, policies and procedures to implement our strategy. The
weaknesses and their components are closely interwoven and often mutually independent, each
affecting the other. Details on each of these issues follow.
Human Resources Strategy
The OIG's human resources strategy remains an office-level weakness, particularly with
regard to skill gaps. Specifically, delays in hiring for certain skills and positions continue to limit
the organization's effectiveness. Although we are improving our skill base through targeted
recruitment and developmental training, the OIG still does not have a detailed workforce plan to
clearly define its overall recruitment strategy, including specific special skills and competencies,
diversity goals, and national and local recruitment practices. Since the OIG will be conducting
more complex assignments requiring special skills, managers and staff need to make informed
decisions about how staff can improve their skills and further their careers.
We continue to implement our strategy to improve performance with emphasis on
organizational values and assessment criteria. However, we have not yet implemented assignment-
specific performance expectations agreements to define individual and/or team assignment
expectations. Clearly defined expectations establish accountability and criteria to measure
performance, management controls, and assure quality. To improve our human resources strategy,
we will:
Develop a detailed workforce plan that identifies special skills and competencies needed
and defines diversity goals based on Census 2000 information.
Complete and disseminate our career path framework to serve as criteria for staff
development.
Develop an official feedback procedure that informs the training decision-makers about
training courses to improve the organizational curriculum.
Implement performance expectations agreements OIG-wide.
Work closely with Team Vegas to expedite the development and classification of
organizational position descriptions and to produce vacancy announcements that clearly
describe the skills and competencies needed to fill positions.
Information Technology (IT) Strategy
Our information technology has not expanded at the rate of our ever-changing
organizational needs. While we have a plan to replace aging equipment, we have not clearly
communicated a long-range plan to purchase new, innovative software and hardware technologies.
Moreover, we have operational constraints due to limited remote access;
inconsistently classified, outdated, or improperly "linked" information on our intranet WEB site; an
inconsistent file/records inventory system; and the Inspector General's Operations and Reporting
(IGOR) system's inability to meet current needs.
-------
25
Since IGOR is unable to fulfill our expectations and its benefits have not been fully
recognized by the Organization, its capability and capacity have never been fully adopted. To
address previously identified issues with our IGOR system and maximize its capabilities, we
developed an IGOR User Handbook and OIG-wide training for system users and identified system
enhancements to streamline data entry procedures. Clearly, more needs to be done.
Accountability and responsibility for information do not exist within the system since the
Organization has not established or otherwise acknowledged data ownership. In addition, the OIG
lacks quality control requirements and procedures for ensuring the integrity of data in IGOR. Data
is not being input consistently; IGOR assignment data is often misleading, due in part to the
onerous and unwieldy reporting process and the voluminous choices available to staff when
completing certain IGOR data fields; and there are no controls to ensure that IGOR timecard data
is reviewed and signed in a timely manner. As a result, the assignment status as reported in the
Agency's Management Accountability Tracking System and our IGOR system do not agree.
To resolve our IT Strategy weakness, we will:
Develop a 5-year IT plan that includes immediate needs and new, innovative
technologies.
Implement controls to ensure the thorough, accurate and timely input of assignment
data in IGOR and timely timecard data review, and link this to individual performance
assessments.
Streamline IGOR options for data fields by using broader categories.
Facilitate the use of sub-accounts across office jurisdictions to allow individuals working
on teams outside their individual organizations to charge time to the project.
Finalize new electronic and hard copy records management policies.
Establish an OIG-wide correspondence control system.
Establish a file/records inventory.
Recruit additional staff with the expertise to address the needs of the OIG intranet WEB
system.
Establish a new WEB policy that includes a reporting requirement on WEB
accomplishments.
Identify a system to replace IGOR.
Business Systems
As the OIG transforms itself from a traditional hierarchical organization with strict function
lines of authority to a matrix organization with flexible lines of authority, necessary accountability
business systems to support these changes are not yet established or integrated. These
accountability systems, including assignment management, cost accounting, business planning and
assignment follow-up, must be designed and implemented to be mutually supportive of the OIG's
internal operations and the Agency's efforts to improve its performance.
Business planning is the first step in accountability by defining what an organization does
and how it will do it. As such, our Strategic Plan must be translated into tactical and action plans
supporting the business and corporate strategies, based on current customer, client, and
stakeholder input. Currently OIG tactical planning is fragmented, without a coordinated,
-------
26
recognized process. With changing organizational roles and responsibilities, the established
process for planning annual work priorities and setting directions is not recognized.
The OIG does not have adequate controls to assure accountability and timely completion
of assignments. What gets measured gets done, but there are not yet office-wide systems in place
to ensure milestones, due dates, and objectives are uniformly established and that deviations are
justified. Consequently, assignments are not always completed in a timely manner and objectives
are not always achieved. It is especially important in a matrix management organizational
structure, that project managers make more timely and reliable decisions about assignment
completion.
A fundamental part of an organization's accountability is the knowledge of what products
and services cost, including indirect costs and the overhead. However, the OIG does not have a
cost accounting system to allocate or determine the cost of individual activities, products, services,
or overhead. Without this information, the OIG cannot encourage and adequately recognize the
most efficient use of resources, promote the best return on investment, or serve as an example of
cost-conscious management so severely needed in EPA.
The OIG conducts many types of audits, program evaluations, and investigations to help
identify and recommend needed Agency improvements. Organizationally, we should evaluate
whether the Agency takes the corrective actions with which they agree, and determine whether our
recommendations successfully contribute to resolution of the problem and attainment of EPA's
goals. Currently the OIG does not have a follow-up policy which would hold the Agency
accountable for taking timely appropriate actions and hold the OIG accountable for the outcome of
its work. To address these issues, we will:
Establish a defined planning process and policy on how annual planning will work in the
future.
Establish milestone performance expectations tied to annual goals.
Integrate measurement systems to regularly monitor time and resource use compared to
progress and results.
Link accountability to individual SES performance assessments.
Develop a process for organization-wide cost accounting related to time charges and
results.
Develop a system to report costs for individual projects/assignments and for support
activities.
Establish an OIG-wide correspondence control system.
Develop a policy for OIG follow-up.
Regularly advise the Agency about the status of its agreed-to corrective actions.
Nikki L. Tinsley
-------
27
Appendix 2
DEFINITIONS OF FY 2001 PERFORMANCE MEASURES
EPA OIG Performance Measurement Results
System Database
GOAL 1 - CONTRIBUTE TO IMPROVED ENVIRONMENTAL QUALITY AND
HUMAN HEALTH
Number of Recommendations for Environmental Improvement
Number of environmentally-related (not management process) recommendations, suggestions or advisory items
from any OIG work including briefings, memos, reports, discussions, etc. (Can be compared to
recommendations not acted upon, or opportunities/costs lost.) Record these recommendations on a
COMPLETED form as COMPLETED RESULTS (outputs).
Rollup similar recommendations that together address one environmental improvement or process change.
Legislative Change/Decision (means to impacts)
Any OIG work which through recommendations, discussions, observations, conclusions, or risk identification
results in, or significantly contributes to, prospective or actual legislative changes to improve an environmental
program. Measured in number of changes and potential impact on environmental programs and results, such
as impact time, measures of environmental improvement, numbers of people affected, health and behavioral
changes (with narrative description in box). Would also be identified in other measures listed below, within
each goal.
Regulatory Change/Decision
Any OIG work which results in, or significantly contributes to, prospective or actual regulatory changes to
improve environmental program implementation. Measured in number of changes and potential impact on
environmental programs and results, such as time, measures of environmental improvement, numbers of people
affected, health and behavioral changes (with narrative description in box). Would also be identified in other
measures listed below, within each goal.
EPA Policy, Directive or Process Change/Decision
Any OIG work which results in, or significantly contributes to, specific changes in definitions, purposes,
objectives, processes, simplifying requirements, or reducing reporting in the formulation of environmental
policies or directives. Measured in number of changes and potential impact on environmental programs and
results, such as time and measures of environmental improvement, numbers of people affected, health and
behavioral changes (with narrative description in box). Would also be identified in other measures listed below,
within each goal.
Examples of Environmental Improvements/Impacts
Any OIG work that results in, or significantly contributes to, an identifiable environmental improvement,
measured in the number of improvements, and the prospective or actual impacts on physical characteristics or
behavioral changes (narrative description in box).
Environmental Risks Identified (including noncompliance)
Any OIG work which identifies actual or potential environmental risks, or health risks, for future plans, action,
reduction or elimination.
-------
28
Environmental Risks Reduced or Eliminated
Any OIG work that results in, or significantly contributes to, the reduction or elimination of environmental
risks. Measured in terms of the number of individual risks of exposure, incidence, or imminent threats. (Can
be compared to risks not acted upon, or opportunities/costs lost.)
Environmental Best Practices Identified
Any OIG work describing a best practice for environmental program implementation, or resolution of an
environmental problem (from any source: State, region, other agency, etc.), measured in terms of the number of
best practices identified and potential environmental impact.
Environmental Best Practices Transferred
Number of environmental program best practices disseminated (output) through OIG work, and the number
which were implemented (outcome) by Agency program offices, regions, States, other government agencies,
other IGs, or other environmental organizations(provide narrative describing each output or outcome action).
(Can be compared to best practices not acted upon, or opportunities/costs lost.)
% Change in Environmental/Health Performance Measure or Indicator (from baseline)
Any OIG work which contributes to a change in an EPA or other environmental or human health performance
measure or indicator (from baseline).
Certifications/Verifications/Validations/Information Integrity
Any OIG work which results in the certification, verification, or validation of Agency environmental work or
programs.
GOAL 2 - IMPROVE EPA's MANAGEMENT. ACCOUNTABILITY AND
PROGRAM OPERATIONS
Questioned Costs (in thousands, round up, or use decimals)
Report all questioned costs as COMPLETED RESULTS on a COMPLETED FORM. Sustained costs will not
be a consideration for this database item. (Collections and Recoveries are covered under the third measure
below.)
Recommended Efficiencies, Cost Saved or Avoided (monetized results)
As a result of OIG work;
1) the cost of Agency work products or office functions which have been eliminated because they were no
longer of use, or too costly,
2) the cost of new or streamlined Agency processes or work products which have been instituted to save the
Agency time and/or money. The monetary benefit, both immediate and for the imminent future (absence of our
involvement), of OIG work to EPA, other Federal/State partners, and stakeholders.
Fines, Recoveries, Restitutions, Collections
Dollar value of investigative recoveries, meaning:
1) Recoveries during the course of an investigation (before any criminal or civil prosecution),
2) court-(criminal or civil) ordered fines, penalties, and restitutions,
3) out-of-court settlements (including administrative actions resulting in non-court settlements).
Criminal/Civil/Administrative Actions
Number of successful prosecutions, indictments, and resulting settlements and convictions affecting EPA
operations and environmental programs. Includes the number of persons or entities that were: 1) indicted, or
for which an "information" was filed, 2) were found guilty or pled guilty in a court of law, or 3) were accepted
for pre-trial diversion agreements by the Department of Justice.
-------
29
Time Saved (cycle time in months)
Months saved, actual and projected, from changes in doing Agency business. Any product or service the OIG
provides which saves the Agency time, but does not compromise quality of end product.
Best Practices Identified or Implemented
Any OIG work describing a best management or program operation practice, or implementation of a best
management or program practice (from any source; State, region, other agency, etc.). (Can be compared to best
practices not acted upon, or opportunities/costs lost.)
Examples of Process/Practice Changes/Decisions
Any OIG work that influences results in process/practice changes or decisions made for EPA, states, or other
agencies. Includes audits, including data integrity or systems audits, evaluations, investigations, assistance,
hotline research, correspondence, meetings, conferences, contract audits, etc.
Recommendations for Management Improvement
Number of management and program operation recommendations, suggestions or advisory items, completed by
the Agency in the Management Audit Tracking System (MATS) reports, or verifiable from another source, or a
closeout memo, from any OIG work including briefings, memos, reports, discussions, etc. (Can be compared to
improvements not acted upon, or opportunities/costs lost.) Record these recommendations on a COMPLETED
form as COMPLETED RESULTS (outputs). Rollup similar recommendations that together address one
management improvement or process change.
New FMFIA/Management Challenges Identified
Number of new FMFIA program assurance issues and Agency management challenges presented as a result of
any OIG work. Includes issues presented in Agency financial statement audits, and internal OIG reviews.
Corrected Management Challenges/FMFIA Weaknesses
Number of management challenges, high risks or material weaknesses resolved or corrected within three years
following OIG recommendations, assistance or advice.
% Customer Satisfaction with EPA Service Quality
Results of customer surveys from within or outside the Agency, about satisfaction with the work of the Agency
for areas we recommend improvements. (OIG may not have outside survey knowledge at this time. Disregard
unless you can cite specific knowledge of an Agency product survey.)
% EPA Rating (by OMB. GAP) on Implementing GPRA
Outside-influenced scorecard of GPRA ratings for the Agency, which the OIG has influenced for improvement.
This includes ratings on strategic plan, annual plan, and annual performance report, and specific examples of
strengths or improvements influenced by OIG involvement.
Certifications/Verifications/Validations
Any OIG work which results in the certification, verification, or validation of Agency management work or
programs, measures, accounting, and data integrity. (Can be compared to data, information, and measures
being at risk.)
GOAL 3 - PRODUCE TIMELY, QUALITY, & COST EFFECTIVE PRODUCTS &
SERVICES THAT MEET CUSTOMERS NEEDS
-------
30
% Customer Service Satisfaction Rating*
Percent based on the results of customer surveys, about the work we do for OIG customers; for/with the
Agency, stakeholders, and other partners in the IG community. This will be coordinated and tabulated by PAR
in collaboration with each office.
# of Collaborative Partnerships with Federal /State Agencies
Number of audit, investigation, evaluation and assistance assignments or projects that are conducted as joint
work with other Federal agencies, States, or entities inside or outside EPA, such as the President's Council on
Integrity and Efficiency (PCIE).
# of Requests to Testify/Information for Hearings
Number of times OIG staff are requested to testify in courts, to Congress, other public venues, or provide
information for presentation in hearings or other judicial or legislative action, regarding audit, investigative or
evaluation findings and recommendations, or other subjects relating to OIG work.
Was Assignment/Product/Service Requested or Required (demand for products and services)
By responding with a Yes or No, we will be able to determine the percent of OIG work requested or required by
number of assignments, and by percent of time expended. A 'YES" represents work products requested from
customers, such as various EPA programs, Congress, hotline, allegations, legislation (by whom the work was
requested, or the authority requiring the work). A "NO" means work required only by internal OIG plans or
processes.
% of Engagement Letter/Project Plan/Budget Expectations/ Milestones Met
Number of OIG audit, evaluation, budget and planning commitments met with this assignment, as a percent of
total commitments.
GOAL 4 - ENHANCE DIVERSITY, INNOVATION, TEAMWORK AND
COMPETENCIES IN THE OIG
Products Completed By Multi-discipline Teams
All OIG work products accomplished by multi-discipline teams. Record number of products from this
assignment if the products were accomplished by a mix of auditors, investigators, engineers, evaluators, etc.
Most OIG products will be multi-discipline.
Awards for Achievement and Innovation/Teamwork
Number and type of awards for OIG staff (HQ and Regions) per year, or other specified time periods.
Innovative Approaches, Techniques, Processes and Solutions to Management Issues Implemented
Number and types of innovative approaches, techniques, processes and re-engineering successes implemented
within the OIG, to enhance OIG competencies and effectiveness in doing our work, and enhance staff
development activities.
% of Work Produced Electronically
Estimate how much of this assignment was "paperless," produced using special electronic features, such as
database programs, functions, equations, charts and graphs, formulas, word processing, workpapers, files,
presentations etc.
% Work Accessible Electronically to Customers (Yes/No)
Are OIG products and services easily available and accessible to customers/clients, the general public,
electronically, compared to those only available in hardcopy, or e-mail? Answer "NO' if product is restricted
or limited from public, Congress, Agency, or general distribution.
------- |