U.S. ENVIRONMENTAL PROTECTION AGENCY (EPA) & MAJOR PARTNERS'
LESSONS LEARNED FROM IMPLEMENTING EPA's PORTION OF THE
AMERICAN RECOVERY AND REINVESTMENT ACT:
FACTORS AFFECTING IMPLEMENTATION AND PROGRAM SUCCESS
SUMMARY OF Six SPECIFIC REPORTS
SEPTEMBER 2013
EPA-100-K-13-005
PREPARED FOR
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF THE CHIEF FINANCIAL OFFICER
WASHINGTON, DC
-------
-------
ACKNOWLEDGEMENTS
This study could not have been possible without the help and cooperation of the many
U.S. Environmental Protection Agency (EPA) employees at Headquarters and Regional
offices who agreed to be interviewed, state staff and funding recipients who
participated in lively focus group sessions, and the many other EPA and state staff who
graciously provided answers to follow-up questions after the interviews and focus
groups were completed. The Science Applications International Corporation (SAIC) Team
appreciates the time given to share experiences beyond all the other audits and
questions. The recollections of those 'working in the trenches' during the intense period
of American Recovery and Reinvestment Act (ARRA) implementation were invaluable in
this study.
September 2013
Page i
-------
This page intentionally blank.
September 2013 Page ii
-------
TABLE OF CONTENTS
EXECUTIVE SUMMARY 1
PURPOSE 1
APPROACH 1
MAJOR FINDINGS 2
SECTION 1. INTRODUCTION 3
1.1 BACKGROUND-PURPOSE OF THE STUDY 3
1.2 SUMMARY OF EPA FUNDS DISTRIBUTION, FUNDS MANAGEMENT AND REPORTING FLOW 4
1.3 Six EPA PROGRAM AREAS 6
SECTION 2. OVERALL METHODOLOGY 7
2.1 STRUCTURE OF SAICARRA STUDY 7
2.2 STUDY QUESTIONS 10
2.3 DATA COLLECTION METHODS 11
2.3.1 Data Collection 13
2.4 DATA COLLECTION CHALLENGES 15
2.5 PARTICIPATION ACROSS THE COUNTRY 17
SECTIONS. FINDINGS 20
SUCCESSES OF ARRA IMPLEMENTATION 20
LESSONS LEARNED ABOUT ARRA IMPLEMENTATION 21
UNINTENDED CONSEQUENCES 21
RECOMMENDATIONS FOR EPA 22
TABLE 1. OFFICE AND PROGRAM AREAS EVALUATED BY TASK 10
TABLE 2. EVALUATION/STUDY QUESTIONS FOR EACH TASK AREA 10
TABLES. DATA COLLECTION METHODS USED FOR EACH TASK 12
TABLE4. STATES PARTICIPATING IN FOCUSGROUP 14
September 2013
Page iii
-------
FIGURES
FIGURE 1. SUMMARY OF EPA FUNDS DISTRIBUTION, FUNDS MANAGEMENT AND REPORTING FLOW
CONCEPT CHART 5
FIGURE 2. STRUCTURE OF SAICARRA STUDY 9
FIGURE 3. LOCATIONS OF CASE STUDIES FOR THE INNOVATIVE TECHNOLOGIES AND LEVERAGING TASKS AND
STATES PARTICIPATING IN THE FUNDS MANAGEMENT AND GPR/COST ESTIMATING Focus GROUPS.IS
FIGURE 4. PARTICIPANTS IN ARRA STUDIES - EPA HQ, EPA REGIONS, STATES, AND FUNDING RECIPIENTS 19
KSHffilBH^
APPENDIX 1: COMPILED EXECUTIVE SUMMARIES FROM THE Six TASK REPORTS APPENDIX 1-1
COST ESTIMATING
FUNDS MANAGEMENT
INFORMATION SYSTEMS DEVELOPMENT AND ENHANCEMENT
GREEN PROJECT RESERVE
INNOVATIVE TECHNOLOGIES
ECONOMIC IMPACTS OF LEVERAGED PROJECTS
APPENDIX 2: TOPICS COVERED IN REVIEW OF EXISTING STUDIES APPENDIX 2-1
September 2013
Page iv
-------
EXECUTIVE SUMMARY
PURPOSE
On behalf of the U.S. Environmental Protection Agency (EPA), Science Applications International
Corporation (SAIC) studied EPA's and its major partners' use and management of the $7.2 billion Congress
appropriated to EPA in the American Recovery and Reinvestment Act (ARRA). SAIC studied how EPA and
its partners used the $7.2 billion to attain EPA's environmental mission and accomplish ARRA's broader
economic goals, while meeting ARRA's additional requirements and tight deadlines. EPA sought to
understand major ARRA outcomes and lessons learned as well as states' and funding recipients'
perspectives to inform efforts to improve EPA operations and inform EPA and partners' management of
other large-scale events such as Gulf Coast Oil Spill or Hurricane Sandy.
Unlike previous reports, this study did not measure compliance with ARRA requirements, nor attempt to
develop mid-course adjustments. The study aimed to add to existing reviews by capturing how EPA and its
major partners worked to attain ARRA's goals. It aimed to capture, consolidate and analyze perspectives
and information from states, funding recipients and vendors, who performed the construction or
remediation work on the ground.
APPROACH
SAIC and its subcontractor, Toeroek Associates, Inc., reviewed previous studies, analyzed EPA and
stakeholder data and interviewed or held focus groups involving over 330 EPA, state and funding recipient
officials who worked on EPA ARRA implementation. SAIC and Toeroek aimed to capture EPA, states' and
funding recipients' lessons learned, successful approaches and their recommendations for future large
efforts with new requirements and significant levels of funding. Since Paperwork Reduction Act
regulations limited the number of non-federal employees who could be interviewed, the study used focus
groups to encourage informed dialogue involving local officials from different specialties.
The study produced six reports; three on management topics and three on results topics:
Management:
Cost Estimating processes
Funds Management processes
Data Systems Development and/or Enhancement
Results:
Green Project Reserve (GPR) benefits
Technological Innovation supported by ARRA
Leveraging Funds in Local Economies
1 ARRA used the term 'prime recipients' when referring to states and tribes and 'sub-recipients' when
referring to their awardees (e.g., local municipality receiving an award). This report uses the term 'states'
when discussing states and 'funding recipients' when referring to 'sub-recipients.'
September 2013 Page 1
-------
MAJOR FINDINGS
Challenging Workload
The study confirmed that it took a tremendous amount of extra effort from all stakeholders to attain
EPA's ARRA goals. Participants shared that the large number of projects, substantial amounts of money,
additional requirements, tight deadlines and sometimes evolving guidance posed significant challenges.
EPA and state participants noted that the effort required diligent dedication from their most experienced
and most knowledgeable staff. Tight deadlines required top-notch staff who knew how to get things
accomplished in the most streamlined manner possible, how to adjust to major changes efficiently, and
whom to call when they needed answers quickly. All stakeholder groups remarked that the pace and
intensity of ARRA implementation took a toll on their over-committed staff members and required trade-
offs against other goals. Nevertheless, stakeholders recognize and value the environmental and economic
benefits made possible through the many ARRA-funded projects.
Partnerships Critical
The challenges posed by ARRA forged changes in how stakeholders communicated and worked with
each other. More effective communication among all stakeholders - EPA HQ, Regions, states, and local
entities - rather than one-way communication from EPA through to funding recipients was an important
development. This strengthened partnerships and allowed stakeholders to meet the challenges they faced
during ARRA implementation. Participants in the study noted that early, organized and frequent
collaborative efforts involving all major stakeholders were crucial in implementing ARRA. This enhanced
teamwork between EPA Regions and states allowed for streamlining of existing processes and programs,
thus avoiding delays from developing wholly new processes and programs.
Benefits Achieved
Local officials endeavored to maximize projects' environmental, health, economic and other benefits by
involving their other local stakeholders in project planning. In addition, ARRA's Green Project Reserve
requirement and EPA's goals to encourage new, innovative technologies led local drinking water and
wastewater utilities to expand their programs to consider green and innovative technologies.
Conflicting Priorities
Sometimes ARRA priorities conflicted with EPA and state priorities. State officials noted that some of the
Green Project Reserve provisions and the timeline required for getting most innovative technologies on-
line conflicted with federal goals. Furthermore, the push to obligate and spend funds quickly was
hindered by additional ARRA mandates, such as the Buy American and Davis-Bacon provisions. Study
participants noted the need to collectively agree on priorities and adopt policies and procedures that
reflect those priorities.
Improved Approaches Adopted
ARRA implementation helped EPA and partners identify successful approaches and improve processes,
grant procedures, communication strategies and management controls. EPA and many states have
permanently incorporated many of these improvements into their programs.
September 2013 Page 2
-------
SECTION 1. INTRODUCTION
1.1 BACKGROUND - PURPOSE OF THE STUDY
In February of 2009, Congress passed ARRA, aimed primarily at making new jobs and saving old ones,
stimulating economic activity and long-term growth, and fostering accountability and transparency in
government spending. Of the $787 billion authorized in the Recovery Act, EPA was given $7.2 billion. EPA
distributed the majority of its ARRA funds to states in grants and contracts to support clean water and
drinking water projects, diesel emissions reductions, leaking underground storage tank cleanups,
Brownfields development and Superfund cleanups. This was a massive undertaking for EPA. The
administration of the funds, which were to be injected into the economy at an unprecedented pace,
required that EPA develop or revise policies, processes and automated information systems. In the fall of
2011, EPA tasked SAIC and its subcontractor, Toeroek Associates, Inc., to design and conduct a study to
capture, verify and analyze the critical lessons learned and successful approaches, policies and procedures
of EPA's implementation of ARRA.
Management Topics - EPA asked the SAIC Team to review three specific management activities:
Cost estimating processes,
Funds management processes and
Systems enhancement and development.
Results Topics - EPA also asked the Team to look at three topics geared more towards outcomes than
management processes. These include the:
Green Project Reserve initiative,
Use of ARRA funds to spur innovative technologies and
Use of ARRA funds to leverage local economic benefits.
The study's results were intended to help EPA and its partners continue to improve the operations of its
programs and help manage any new large projects or programs with additional requirements. EPA has
already incorporated many improved processes and procedures from ARRA in its regular programs and
has used the study's preliminary results to help manage both the Gulf Coast Oil Spill and Hurricane Sandy
response and recovery efforts.
The study resulted in a series of six reports, each covering one of the six topics noted above, plus this
overarching summary report, which contains overall findings of the study, highlights of each of the six
reports and a description of the goals and methodology for the entire study. For example, the study
looked at questions such as the following. What changes did states made to their grants management
processes and why? What processes helped states obligate funds in time to meet the tight deadlines
imposed by the ARRA statute? Through several years of implementing ARRA-funded projects, what did
EPA and states learn? What would EPA and states do differently if given the opportunity again? What can
states recommend to EPA? Did 'green' projects generate benefits beyond primary environmental
benefits?
The study was based on the data captured in EPA, state and funding recipient tracking systems; existing
analyses; and discussions with EPA, state and funding recipients. The study was designed to yield results
that could help future decision-makers. It is critical to note that the study was not an audit and not
designed to check whether rules were followed. This report reflects EPA's intent to share lessons learned
September 2013 Page 3
-------
with EPA, state, and tribal program offices to spur positive changes in program/project implementation in
the future. Unlike other ARRA assessment reports prepared by the Office of the Inspector General or
other EPA program offices, this report includes information directly from the perspectives of funding
recipients, including vendors.
Several of the study efforts involved interviews with specific technology vendors. It should be noted that
the mention of trade names, specific vendors or products does not represent an actual or presumed
endorsement, preference or acceptance by EPA or the Federal Government. Stated results, conclusions,
usage or practices do not necessarily represent the views or policies of the EPA.
The remainder of this report describes Background relevant to ARRA implementation, the Methodology
used (including charts, study questions and data collection steps) and Findings. The Findings include
perspectives from interviewees and focus group participants as well as SAIC analyses and
recommendations. Appendix 1 contains a compilation of the six Executive Summaries from the six task
studies.
1.2 SUMMARY OF EPA FUNDS DISTRIBUTION, FUNDS MANAGEMENT AND
REPORTING FLOW
The funding made available to EPA through the Recovery Act was of historic proportions. It is important to
understand the flow of funds from Congress to EPA to recipients and the subsequent flow of project and
spending information from recipients to EPA to Congress. Figure 1 illustrates the responsibilities of EPA,
states, and funding recipients in the movement and management of ARRA funds, and subsequent
reporting on ARRA project status.
The diagram illustrates three major processes - money flow, money and project management and
reporting - that EPA, states and funding recipients managed during ARRA implementation. The diagram
highlights specific items that SAIC studied to identify lessons learned. On the chart, the three major
processes are color-coded:
, the Funds Distribution Boxes trace the movement of ARRA funds from Congress to EPA and then
out to the six program areas - Drinking Water State Revolving Fund (DWSRF), Clean Water State Revolving
Fund (CWSRF), Superfund (SF), Diesel Emission Reduction Act (DERA), Leaking Underground Storage Tank
(LUST), and Brownfields.
': ;;: : : the Funds Management boxes show funds management responsibilities for EPA, states and
funding recipients.
In the Reporting and Verification Boxes depict how reporting and verification information flows back
to EPA, and eventually to Congress and the White House.
September 2013 Page 4
-------
FIGURE 1. SUMMARY OF EPA FUNDS DISTRIBUTION, FUNDS MANAGEMENT AND REPORTING FLOW CONCEPT CHART
Funds Distribution
Recovery Act
Transparency
Board (RATB)
Programs (DWSRF,
CWSRF, SF, DERA,
LUST, Brownfields)
EPA Funds
Distribution,
Funds
Management
and Reporting
States and
Tribes
Funding
Recipients
Funds Management
EPA HQ &
Regions develop
and implement
project
requirements,
performance
measures,
oversee States'
implementation
EPA or States
oversee funding
recipients'
implementation
and compliance
with EPA ARRA
required policies
and procedures
Funding
Recipients
oversee
project
activities
Labor hours,
materials, and
contractors for
protection and
restoration
projects
Reporting and Verification
Monitoring by EPA
Program Offices
Regions
September 2013
PageS
-------
1.3 SIX EPA PROGRAM AREAS
SAIC coordinated with EPA to prioritize activities on the six tasks across the six environmental program
areas receiving ARRA funding:
Clean Water Act State Revolving Fund (CWSRF) - $4 billion,
Safe Drinking Water Act State Revolving Fund (DWSRF) - $2 billion,
Superfund (SF) - $600 million,
Brownfields-$100 million,
Leaking Underground Storage Tanks (LUST) - $200 million, and
Diesel Emissions Retrofit Act (DERA) - $300 million.
Given funding levels, the SAIC study focused primarily on State Revolving Fund (SRF) projects funded
through ARRA, with some lessons learned from the other four programs as well.
September 2013 Page 6
-------
SECTION 2. OVERALL METHODOLOGY
This section describes the EPA ARRA implementation process, explains the structure of the study,
identifies the general study questions and presents the general data collection methodology. Each of the
six reports provides a more detailed description of the methodology used for that particular study. SAIC
worked with EPA and used a variety of tools to understand how the EPA ARRA program was intended to
work to achieve its aims. EPA outlined the major steps performed in order to plan for, implement and
successfully manage the six programs receiving ARRA funds, and thus helped to highlight specific stages in
the ARRA implementation process to examine during the study.
SAIC prepared detailed study methodology documents specific to the six evaluation tasks. The
methodology, or scoping, documents identified the detailed study questions, presented SAIC's intended
approach to access the information sources and analyze the data, and described SAIC's proposed strategy
for reporting results. Final reports from each of the six studies are included as separate components of
the overall final report and include the methodology for each.
2.1 STRUCTURE OF SAIC ARRA STUDY
The graphic presented below depicts the structure of SAIC's study and the offices and program areas
reviewed.
Figure 2 depicts the basic study process, the data sources that SAIC used, specific task topics and products
that resulted from the study. The six task topics are grouped in two categories: management and results,
based on the objectives of the individual tasks and the associated study questions. In general, the tasks
within the Management category focused on EPA funds distribution, reporting and verification processes,
and implementation activities. The tasks within the Results category focused on the results of program
implementation. The general study objective of each task is described below.
Management Topics - Funds Distribution, Reporting and Verification, and Implementation
Cost Estimating - capture best practices information and understand the factors that caused variations in
estimated costs; identify factors that might be most directly related to better cost estimation.
Funds Management - capture, verify and analyze the critical lessons learned, successful approaches and
recommendations related to ARRA funds management, particularly relative to the:
Timely obligation and expenditure of ARRA funds;
Approaches used to comply with additional requirements imposed by the Recovery Act Buy
American provisions, Davis-Bacon Act, mandatory reporting, and Green Project Reserve; and
Grants management process.
Development and/or Enhancement of Systems - capture the lessons learned from system development
and enhancements to provide EPA with crucial lessons learned in how best to manage its continuing ARRA
system efforts as well as other special efforts such as hurricane response or large environmental disasters.
September 2013 Page 7
-------
Results Topics - External Effects of EPA ARRA Projects
Green Project Reserve - capture information related to successes, strategies and lessons learned in the
ARRA green initiative (20 percent set aside of ARRA SRF funds for green projects); where possible, identify
secondary and unanticipated outcomes of GPR projects receiving funding under ARRA.
Technological Innovation - gather and report on information regarding any new or expanded industries
or markets fostered as a result of spending under ARRA.
Leveraging ARRA Funds in Local Economies - conduct an economic impact evaluation to characterize the
types of economic impacts on local communities from projects receiving EPA ARRA funds that leveraged
other resources, including long-term benefits of improved environmental quality associated with each
project as well as short-term economic benefits in the local region.
September 2013 Page 8
-------
FIGURE 2. STRUCTURE OF SAIC ARRA STUDY
Inputs
Information Sources
Data
- Recipient reporting
- Project files
-GrantlGMS
- Program databases
(CBR, DWSRF-PBR, DERA
DRIVER, Superfund, LUST,
Brownfields ACRES)
Previous/Ongoing
Analyses
- OW project verification
- OW/OPAA Green projects
- OCFO Policy verification
-DIG Reports
- Other agency reviews/reports
Interviews and
Discussions
- Agency leadership
- Subject matter experts
- Select recipients
- EPA staff
- Focus groups
Analyses
Study Topics
Major Analytical Goals (grouped by topic)
Management- Funds Distribution,
Management, Reporting and Verification
Cost estimating - Best practices
Funds management and reporting with focus
on:
- Timely funds obligation and expenditures
- Grants management and reporting
- Approaches to comply with additional
mandates (Buy American, Davis-Bacon
Section 1512 Reporting )
Systems Development and/or Enhancement
Results - External Impacts of EPA projects
Green Initiatives (Green Project Reserve)
Technological Innovation
Strategies for Leveraging ARRA funds - Local
Economic Impacts
Notes:
1. NOT an audit - focus is on results, not rules.
2. Analyses follow all Confidential Business Information
and Information Collection Requirements.
3 Analysis and data collection effort overlap-
Outputs
Lessons Learned/Cutouts
Written Analyses
- Specific analyses, etc.
- Summary Evaluation Reports
i
i
Data Compilation and
Analyses
- Completed data sets
- Synthesized summaries, etc.
^^^^^^^j
F
Visuals
-Charts, graphs, presentation
materials, etc. illustrating
lessons learned
Acronym Legend
ACRES - Assessment, Cleanup and Redevelopment Exchange System
CBR - CWSRF Benefits Reporting
DERA DRIVER - Diesel Emissions Reduction Act Database for
Reporting Innovative Vehicle Emission Reductions
PBR - DWSRF Project and Benefits Reporting IGMS - Integrated Grants
Management System
LUST - Leaking Underground Storage Tank
OCFO - Office of the Chief Financial Officer
DIG - Office of the Inspector General
OW - Office of Water
OW/OPPA - Office of Water/Office of Policy and Planning Analysis
PBR- DWSRF Project and Benefits Reporting
September 2013
Page 9
-------
Table 1 shows the EPA offices involved in each of the six tasks.
TABLE 1. OFFICE AND PROGRAM AREAS EVALUATED BY TASK
Cost Estimating Best
Practices
Funds Management
Systems Development and
Enhancement
Green Project Reserve
Innovative Technologies
Leveraging Funds in the
Local Economy
Office of Water Off ice of Solid Waste And °ff ice °f Air and
Emergency Response Radiation
CWSRF
X
X
X
X
X
X
DWSRF
X
X
X
X
X
X
Superfund
X
X
X
-
-
-
Brownfields
-
X
X
-
X
X
LUST
-
X
X
-
-
-
DERA
-
X
X
-
-
-
2.2 STUDY QUESTIONS
Table 2 provides the study questions associated with each task area.
TABLE 2. EVALUATION/STUDY QUESTIONS FOR EACH TASK AREA
Study Areas Focus Areas
Cost Estimating
Funds
Management
Systems
Enhancement and
Development
Cost Estimating
Approaches
Cost Estimate Timing
Cost Variance
Factors
Project Scope
Changes
Results/Impacts of
Cost Changes
Lessons Learned
Policies, Processes
and Procedures
Factors
Challenges
Factors for Success
System
Development/
Modification
Challenges
What cost estimating approaches do funding recipients use for developing project cost
estimates, and why?
What effect does the timing of the cost estimate have on the project outcome?
What cost factors influence differences between initial cost estimates and final costs?
What changes were made to the contract after award but prior to project completion
as the result of scope changes such as field change orders?
What are the results and impacts of changes between initial cost estimates and final
costs?
What are the lessons learned in cost estimating practices?
What funds management policies, processes and procedures helped/hindered ARRA
implementation efforts, and why?
What factors helped and/or hindered funds management policies, processes and
procedures implementation, and why?
How were challenges to implementation of policies, processes and procedures
overcome?
What factors or changes contributed to developing or modifying systems that met the
needed data requirements of ARRA?
What were the challenges to developing or modifying EPA's information management
systems to meet Recovery Act requirements? What factors created challenges to
developing or modifying systems?
September 2013
Page 10
-------
Study Areas
Green Project
Reserve
Innovative
Technologies
Leveraging ARRA
Funds in Local
Economies
Focus Areas Study Questions
Leveraging for
Future Systems
Development
Environmental
Outcomes
Outcomes Extent
Influencing Factors
Lessons Learned
Products and
Services
Suppliers, Industries
and Markets
Impacts
Subsidy Levels
Leveraging Levels
How well did systems development and modification efforts succeed in meeting their
ARRA objectives? Which ARRA information management process changes have been
and should be leveraged for future use? Are any system development changes
applicable to multiple EPA program systems?
What secondary environmental outcomes (positive and negative) are associated with
GPR projects?
What was the magnitude of those outcomes?
What factors lead to projects that show secondary environmental outcomes?
What information or documentation is available to assist municipalities to better
assess and predict all environmental outcomes (primary and secondary) associated
with potential projects?
What types of innovative products or services did the ARRA recipients implement, and
why?
How did ARRA funding affect suppliers of innovative technologies and the industries or
markets for the innovative products and services identified above?
What impact did the selected projects have on the local economies?
How do subsidy levels affect the extent of local impact?
How do leveraging levels affect the extent of local impact?
2.3 DATA COLLECTION METHODS
The data collection methods for the six task areas of the study share several common steps. The SAIC
Team generally implemented the study in the following five steps. Table 3 indicates how they were used
for each specific task. Not all the steps were used in each task; the individual task reports explain the
specific data collection steps followed for each task.
1. Reviewed and analyzed information and documents from EPA files, web sites and databases;
state web sites and files; and funding recipient files.
2. Interviewed a select number of EPA personnel.
3. Conducted focus groups of selected states.
4. Conducted discussions with selected funding recipients (as needed).
5. Reviewed EPA or state project files.
September 2013
Page 11
-------
TABLE 3. DATA COLLECTION METHODS USED FOR EACH TASK
Analyzed
Held Discussions
Interviewed Conducted State with Selected
Funding
Recipients
Project
: Estimating
Best Practices
Funds
Management
Systems
Development
and
Enhancement
Green Project
Reserve
Innovative
Technology
Leveraging
Funds In Local
Economies
* Interviewed EPA Staff only on Superfund component of Cost Estimating (not SRF or Green Project Reserve staff)
** Conducted file reviews in North Carolina and Montana for Cost Estimating; did not examine files in other states
because file review did not provide detailed information.
September 2013
Page 12
-------
2.3.1 DATA COLLECTION
Not all tasks used all steps in the same way. The more detailed methodology descriptions in each task
report describe how the study was conducted.
STEP 1: REVIEWED ANALYZED INFORMATION DOCUMENTS
All study tasks included the review of existing information and documents prior to additional data
collection efforts. The SAIC Team reviewed and analyzed information and data from the following sources:
Existing studies: The SAIC Team reviewed a number of reports from different sources to extract
information on improvements made or needed, lessons learned and successful approaches; SAIC
summarized and compiled this information in Appendix 2. Topics Addressed by Existing Studies
and Evaluations. SAIC used this background information to develop each task's methodology and
study questions. This included reviews of EPA's ARRA implementation activities by EPA's program
offices, the Office of the Inspector General (DIG) as well as outside agencies such as the U.S.
Government Accountability Office (GAO).
EPA and state web sites: The SAIC Team searched and reviewed documents posted on EPA and
state web sites that address ARRA implementation strategies or approaches, or lessons learned.
EPA databases (e.g., federalreporting.gov, CWSRF Benefits Reporting System (CBR), DWSRF
Project and Benefits Reporting System (PBR), Integrated Grants Management System (IGMS)):
The SAIC Team analyzed data from EPA databases to identify patterns/differences among EPA
Regions and states for selection criteria used to identify states for focus groups.
EPA and state files: Based on recommendations from interviews with EPA HQ and EPA Regional
personnel (Step 2), the SAIC Team reviewed selected EPA and/or state files containing
information relevant to the evaluation questions for each task.
\ STEP 2: INTERVIEWED EPA PERSONNEL
Information resides with EPA personnel who implemented the changes and have knowledge of what
worked, where problems occurred and what improvements were seen. The SAIC Team:
Identified key EPA personnel to interview. The SAIC Team interviewed EPA Recovery Act senior
leaders, Subject Matter Experts (SMEs) and other EPA staff. The SAIC Team worked with EPA to
develop a list of specific individuals at EPA HQ and EPA Regional offices able to give their
perspective on major lessons learned as well likely to fill information gaps and/or clarify existing
information due to their positions or roles and responsibilities within the grants programs.
Developed an interview protocol for interviews with EPA staff, and scheduled and conducted
pilot interviews with a select number of EPA HQ staff and/or EPA Regional staff. The Interview
Protocol Checklist outlined specific steps for the interviewer to ensure that information was
obtained and recorded consistently.
Conducted interviews (approximately one hour in length). At the beginning of the interview, the
interviewer informed the interviewee that individual responses would not be distributed to EPA,
September 2013 Page 13
-------
nor appear in the report in a way that allows specific responses to be associated with specific
respondents.
Analyzed interview responses and used preliminary findings to help define additional EPA
interviews at the EPA Regional level and to confirm and identify appropriate state focus group
participants and funding recipients.
STEP 3: CONDUCTED STATE FOCUS GROUPS
The SAIC Team used a focus group approach with states to obtain information, understand recipients'
perspectives and better understand program context not available otherwise (from EPA interviews or
existing data). Focus group sessions are facilitated discussions that allow participants to engage in topics
of common interest, without having to adhere to a specific set of survey questions. The format
encouraged participants to volunteer information about their experiences with ARRA implementation.
Identified Potential States for Focus Group Discussions. The SAIC Team convened state focus
groups to meet the needs of three studies based on selection criteria specific to the tasks.
Selection criteria included: timing of award, award amount, project size, project location or
project completion. States were selected based on these criteria and through discussions with
EPA and state staff. Table 4 lists the states included in Focus Group Discussions for each of the
three studies - Cost Estimating, Funds Management and Green Project Reserve. SAIC staff
determined that it made sense to combine the Cost Estimating and Green Project Reserve Focus
Groups, because the same state and recipient personnel could contribute to discussions of both
topics. Information gathered during the Cost Estimating/Green Project Reserve focus groups was
also used in the Funds Management report, as participants volunteered significant information
relevant to funds management processes.
TABLE 4. STATES PARTICIPATING IN FOCUS GROUP
STATES INCLUDED IN FOCUS
Funds
Management
Colorado (Pilot)
Missouri
New Hampshire
New York
Texas
Virginia
Washington
Cost Estimating
Green Project Reserve
Iowa
Louisiana
Montana
New York
North Carolina
Oklahoma
September 2013
Page 14
-------
Developed a Focus Group Protocol and Conducted Pilot Focus Group with State Staff. The SAIC
Team conducted a pilot Focus Group with State of Colorado ARRA personnel for the Funds
Management Task. SAIC invited state staff who were involved with ARRA implementation (e.g.,
contract management, project management, technical oversight) to participate. The pilot focus
group provided the SAIC Team the ability to test the process and discussion topics. SAIC gathered
suggestions from state staff on possible projects files the Team could review or funding
recipients SAIC could contact for further data collection efforts. The initial pilot focus group with
Colorado provided very valuable information on organizational structures and administrative
processes. The outcomes from this pilot were applied to the rest of the Funds Management focus
groups as well as to the Cost Estimating and Green Project Reserve focus groups.
4: CONDUCTED WITH SELECTED (VENDORS,
\ MUNICIPALITIES, CONTRACTORS)
The ARRA-funded projects were implemented at the local level; data collection for some study tasks
required direct contact with funding recipients. As indicated in Table 3, this level of data collection was
appropriate for all tasks except Systems Development and Enhancement (a study based on data collection
from EPA Headquarters staff and records only).
For the Innovative Technologies and Leveraging studies, the SAIC Team spoke with select funding
recipients to gain their particular insights regarding their ARRA projects. For Funds Management, Cost
Estimating, and Green Project Reserve, states participating in focus groups invited funding recipients to
attend the focus groups. While the number of local representatives attending focus groups was limited,
they provided a valuable local perspective on the challenges they faced, the strategies they developed,
the EPA activities that helped them with regard to cost estimation, and approaches for compliance with
the additional mandates (Buy American, Davis-Bacon and reporting requirements).
STEP 5: REVIEWED EPA OR STATE PROJECT FILES
If warranted after interviews with EPA staff or focus groups with state contacts, the SAIC Team reviewed
specific project files to research specific fund recipient information that provided additional details to
answer the study questions (e.g., cost estimating). Examples of the types of information included
differences observed between initial cost estimates and actual project costs.
2.4 DATA COLLECTION CHALLENGES
Across the ARRA study effort, there were several data collection challenges. The detailed data collection
methodologies for each task took these challenges into account in the design of the specific approaches
for each task. In general, the challenges and basic approach have been summarized in the following
section.
Much of the information gathered in the study effort was qualitative in nature rather than quantitative.
The fidelity of the accumulated information to actual events during ARRA implementation depends on the
accuracy of recollections and reporting from an assortment of individuals, all of whom brought their
specific, and likely selective, memories to the table. Different individuals placed different emphasis on the
myriad of factors affecting ARRA program implementation. SAIC recognizes these challenges but also
September 2013 Page 15
-------
understands the importance of and insight that can be gained from documenting these
recollections/memories. Therefore, the SAIC Team conducted a large number of interviews and focus
groups from a broad base of sources and used a systematic data review process to aggregate similar
responses and identify patterns and themes. While the analysis does not provide a representative view of
all EPA, state and funding recipient experiences, it identifies trends that highlight especially valuable
lessons learned and successful approaches adopted to improve program implementation.
The lack of an Information Collection Request (ICR) specific to this study limited the methods SAIC could
use to collect information from non-federal employees. It did not limit the number of EPA staff that SAIC
could contact to discuss the ARRA implementation process; this was particularly relevant for the Systems
Development and Enhancement task and the Funds Management task. For tasks requiring input from
state and funding recipient contacts, SAIC's approach to use focus groups at the state level (described in
detail in Step 3 above) and general discussions with the funding recipients (described in detail in Step 4
above) maximized the amount of state and funding recipient input. Where specific conversations/data
collection efforts needed to be conducted with state or funding recipient representatives, SAIC limited the
data collection efforts to nine or fewer representatives for any set of questions or topic.
Per discussions with EPA, SAIC understands that focus groups that solicit participant discussion around
broad topics rather than responses to specific questions do not require an ICR. Similarly, SAIC understands
that general discussions with funding recipients for the purpose of clarifying information already provided
as part of ARRA reporting requirements are covered under existing ICRs.
Staff turnover at EPA, state and funding recipient offices since the ARRA program was implemented
may have limited the number of individual contacts available (and therefore the amount of information)
that the SAIC Team could gather. Where staff turnover had occurred, the SAIC Team made attempts to
contact the staff who had responsibility for ARRA implementation (but had moved to a new position
within the agency) or located alternative staff who could help in piecing together the picture of program
set-up and implementation. EPA staff members were helpful in directing SAIC Team members to others in
the Agency who could answer questions about ARRA topics.
The EPA ARRA programs were implemented alongside ongoing work funded through annual
appropriations. This was an advantage to EPA in implementing the programs since EPA could use and
build on an existing set of rules and expertise, but it created a challenge in trying to differentiate the
impact of the ARRA portion. Without a clear comparison to show what would have happened in the
absence of ARRA funding, this study did not attribute outcomes and impacts directly to ARRA but
identified how ARRA influenced or contributed to outcomes and impacts. The studies do not try to show
causality per se. Instead, the case study and interview approach used in this study is intended to generate
analyses that captured the lessons learned and successful approaches developed.
Many individuals and offices, when originally contacted about possible participation in the study, were
reluctant to participate. Some were concerned that the study was a compliance assessment (especially in
for those involved in Cost Estimating). Some responded that they were still catching up from the workload
of ARRA. Some people preferred focusing on the future and not reviewing the past, particularly since they
believed it unlikely that EPA would get another Recovery Act anytime soon. Some saw it as yet another
OIG or GAO audit. And some were concerned that results of such a study might reflect poorly on their
programs, state or office. The SAIC Team devoted considerable effort to engaging state and local
participation.
September 2013 Page 16
-------
2.5 PARTICIPATION ACROSS THE COUNTRY
Five of the task topics - Innovative Technologies, Leveraging Funds in Local Economies, Green Project
Reserve, Funds Management and Cost Estimating - involved gathering information from many states and
EPA Regions. The map below in Figure 3 shows locations of the case studies for the Innovative
Technologies and Leveraging Funds tasks and the states participating in the Funds Management and
Green Project Reserve/Cost Estimating focus groups.
September 2013 Page 17
-------
FIGURE 3. LOCATIONS OF CASE STUDIES FOR THE INNOVATIVE TECHNOLOGIES AND LEVERAGING TASKS AND
STATES PARTICIPATING IN THE FUNDS MANAGEMENT AND GREEN PROJECT RESERVE/COST ESTIMATING FOCUS GROUPS.
Economic Impact of Leveraged Projects
A Brownfields
Clean Water State Revolving Fund Projects
Drinking Water State Revolving Fund Projects
Innovative Technology
+ Innovative Technology Projects
State Participation
V/\ Funds Management State Focus Groups
Cost Estimating and GPR State Focus Groups
Amsterdam
lager stow n
Cape Charles
lolly Springs
ily County
EPA Regional Interviews
Region 1 Region 6
Region 2 Region 7
Region 3 Region 8
Region 4 Region 9
Region 5 Region 10
Puerto Rico
September 2013
Page 18
-------
Figure 4 illustrates the number of individuals providing input to the six ARRA studies. These include EPA
staff from HQ and the Regions, state program staff and several funding recipients (e.g., vendors for
innovative technologies). Over 330 individuals participated in the effort.
FIGURE 4. PARTICIPANTS IN ARRA STUDIES - EPA HQ, EPA REGIONS, STATES, AND
FUNDING RECIPIENTS
'f.
IEPAHQ
Region 1
Region 2
Region 3
Region 4
Region 5
Region 6
Region 7
Region 8
Region 9
Region 10
334 Participants from EPA HQ, EPA Regions, States, and Funding Recipients
September 2013
Page 19
-------
SECTION 3. FINDINGS
In the course of this study, SAIC interviewed or included in focus groups more than 330 individuals who
had worked on EPA ARRA programs either at EPA or with states or funding recipients. While many relayed
project-specific or state-specific incidences, some common sentiments rose to the top, including
successes (as identified by study participants), lessons learned, some unintended consequences of ARRA
processes or projects and recommendations for the future.
SUCCESSES OF ARRA IMPLEMENTATION
Early, frequent collaborative efforts were key. Planning meetings before ARRA was even signed
helped EPA and states hit the ground running. States were already compiling their lists of
projects to be funded when the statute was signed. Communication throughout ARRA
implementation, especially when conducted through electronic means to all stakeholders,
enabled everyone to have access to the same information at the same time.
Building on existing programs, processes and systems saved time. There was no need (or time)
to reinvent the wheel. Adjustments made to existing programs, processes and systems were
easier to absorb.
'Live' meetings, webinars and training sessions contributed to the success of ARRA
implementation. Face-to-face communication bolstered the sense of a team united to address
common challenges together.
Strong partnerships among EPA, states and localities were essential to ARRA implementation.
EPA's communications underscored the understanding that EPA's environmental goals can be
met only through coordination and cooperation with its partners.
Implementation inspired the development of forms and templates to standardize and
accelerate processes. These, and other streamlining initiatives, have been permanently adopted
by some programs.
ARRA funding supported the development and sale of innovative technologies. Water and
wastewater utilities, which are normally risk-averse, were motivated to iuse new technologies.
Green Project Reserve's requirement to use at least 20 percent of ARRA-SRF funds for green
projects served as a catalyst to seek out energy efficient projects and incorporate this criterion
into the states' existing programs.
The Superfund program's ability to use existing contract vehicles at existing sites meant the
program was able to readily meet ARRA deadlines and requirements. The program maintained
a large stock of large shovel-ready projects and did not pose significant burdens to staff because
remediation contracts were already in place.
ARRA projects stimulated local economies during the recession. For every ARRA dollar spent,
the regional impact ranged from one and a half to three dollars across nine case study projects.
Thus communities experienced economic and environmental benefits from ARRA projects.
ARRA provided needed funding to disadvantaged and rural communities. State and funding
recipients noted that ARRA funding contributed significantly to their rural and disadvantaged
communities, citing that the majority of these communities are unable to afford an SRF loan
September 2013 Page 20
-------
and/or to coordinate larger projects. Communities appreciated ARRA's subsidy/loan forgiveness
provision, but states noted that this created extra administrative work to track different
percentages of loan forgiveness across different communities.
LESSONS LEARNED ABOUT ARRA IMPLEMENTATION
The workload was intense and EPA and partners used several strategies to address it. EPA HQ,
Regions and states all agreed that the workload from the bounty of funding, and the
accompanying requirements, was daunting. Some programs hired new staff; most shifted
responsibilities to ensure that experienced staff handled the ARRA work while junior staff
managed the base programs. States were grateful for the funding, but many reported they would
think twice if offered the same opportunity again.
Early guidance was helpful, but changes over time created substantial re-work and uncertainty.
States appreciated the guidance issued by EPA but commented that changing the guidance mid-
stream was confusing and contributed to the overall workload.
ARRA implementation required the most knowledgeable, dedicated people. The complexity
and scale of the effort required people with extensive knowledge of how the existing programs
worked and who they needed to contact when the rules changed - as they had under ARRA.
The level of oversight did not always match the need. States and Regions commented that in
some cases the amount of time devoted to monitoring expenditures and progress seemed
unnecessary. Participants did not see clear targeting of oversight issues based on risk.
Participants noted that it appeared to them that few people used the information gathered
through recipient reporting.
The mandate to spend the funds quickly sometimes conflicted with state priority lists for
projects. States recounted instances in which they had to pass over priority projects with greater
environmental benefits in favor of shovel-ready projects with fewer benefits to the public.
Tight deadlines for obligating and spending funds was inconsistent with the timeline for most
innovative technologies, which require concept design, fabrication, pilot testing and full-scale
commercialization. The SRF's deadline for having projects under contract within one year made it
difficult for water projects to take advantage of green infrastructure or innovative technology
options, especially if project blueprints had to be revised.
The realities of the financial times changed cost estimates for projects, which were lower than
expected due to the fierce competition between companies trying to stay afloat. Profits
became less important than retaining skilled employees. States related that businesses definitely
would have failed if ARRA funding had not come to the rescue.
UNINTENDED CONSEQUENCES
During the course of communications with stakeholders, SAIC unearthed several unintended
consequences of ARRA implementation. Some are positive consequences; some are less so. It is difficult
to plan for the unexpected; if you know it is going to happen, you can plan for it. Nevertheless, it may be
instructive to keep these lessons in mind when considering the impacts of distributing funding to
accomplish environmental and economic goals.
September 2013 Page 21
-------
Mixed blessings. Small communities receiving sizeable grants for ambitious projects were
grateful for the improvements in their towns. With little or no town staff or expertise though,
they were overwhelmed and needed to turn to their state or consultants for project
management support. State staff, pressed into double-duty as program implementers and
project managers, carried significant loads. The same can be said for not-so-small towns who had
never received SRF funds before; they also required considerable attention from already over-
burdened state staff.
You get (only) what you pay for. One of the easiest project types to fund quickly was metering.
In some cases, metering could be counted as a 'green' project and thus also helped states
achieve their 20 percent requirement for the Green Project Reserve. Towns with inadequate
metering reported that they were losing public revenues due to losses of produced water
(potable water treated and distributed through the public water supply) to unmetered users or
users with old, inaccurate meters. After new meters were installed, users accustomed to very
low water bills were shocked at the true cost of their water usage. Instead of viewing these
public improvements as a positive change for their communities, some citizens viewed the new
meters as technologies that increased their water bills.
Just the (wo)man for the job. Some states used ARRA-funded administrative dollars to make
new hires to handle the increased workload. In some cases the new hires, which were advertised
as temporary, performed so well that the states elected to make the hires permanent. Thus the
stimulus bill did accomplish the goal of creating good, meaningful jobs. Employees who thought
they were getting only a temporary job landed a permanent one.
Double bonus points. It is intuitive that money invested in a community will prompt other
investments. What might not be intuitive is that this concept could apply to sewage treatment
plants or road paving projects. States cited several instances in which an ARRA-funded
community project sparked additional investments from the community members either from
increased civic pride or created economic opportunities. Thus, not only did the community reap
the benefits of new public spending, they multiplied the benefits by investing their own funds in
the community.
Innovation turns green. Water and wastewater utility managers reported that the adoption of
innovative technologies funded through ARRA provided more than an efficient means of
achieving compliance with regulatory standards. Participants confirmed that the green
requirements helped encourage recipients to consider additional benefits including energy
savings, lower maintenance costs, reduced water waste and less chemical use.
RECOMMENDATIONS FOR EPA
Based on participants' input and data analyses, SAIC and Toeroek Associates, Inc. discerned several
recommendations.
Use multi-disciplinary, multi-organizational teams to draw input from all stakeholders. All
perspectives are needed to identify the best processes for implementation, discern a common
lexicon that all stakeholders will understand, and clarify confusing terms that connote different
meanings between different agencies and organizations (e.g., 'shovel-ready', 'green'). In
September 2013 Page 22
-------
particular, submit guidance documents and forms to a thorough vetting process such that all
stakeholders can provide their perspectives before the guidance is officially issued.
Think strategically about information needed to assess success, whether for environmental,
economic or other goals - not only what will be needed now but in five years. The SAIC Team
was surprised to find that states accurately tracked the loans that they provided to fund projects
but did not keep detailed records of total project costs. By only tracking the funds they provided,
they had no information on funds received from non-EPA sources or the grantee itself. As a
result, the estimated cost of the project could not be compared to the final project cost. (New
York was an exception in that program managers monitored all project costs regardless of the
source of the funds.) Likewise, when the Team attempted to document primary and secondary
environmental benefits of green projects, states responded that they were not required to
collect such data and indeed did not. Knowing data needs ahead of implementation will shape
systems development, reporting requirements and oversight processes and schedules.
Identify conflicting goals and objectives as early as possible and attempt to prioritize. As noted
above, states had difficulties in finding shovel-ready projects that also met important
environmental priorities. Sometimes states reluctantly passed over higher ranking projects just
because they were not classified as 'green' or were not shovel-ready. If quick obligation of funds
is paramount, do not attach new complex requirements that slow the process. In addition, many
perspectives are critical to recognizing what might be the major management, economic,
technological challenges and trade-offs as early as possible. Recognizing and planning for these
is critical to successfully implementing programs with new goals and requirements.
Develop a process to facilitate peer learning throughout the implementation process. States
and localities noted that they could have benefited from learning about how their peers were
handling similar situations. Particularly for small communities or for areas that previously had not
participated in EPA grant programs, funding recipients needed additional guidance on managing
grant funds, reporting and managing project selection and construction. This training must
happen early in the process.
Preliminary Recommendations Used for Emergency Response and Recovery Efforts
EPA already adopted some preliminary recommendations from this study to guide decisions and
management for the Gulf Coast Oil Spill and Hurricane Sandy response and recovery efforts. With regard
to the Gulf Coast Oil Spill (i.e., Deepwater Horizons), EPA accepted the role of chair of a multi-agency task
force to coordinate government actions and develop effective strategies that addressed a wide range of
challenges. Internally, EPA assembled an interdisciplinary team to develop policy initiatives and an
administrative group to coordinate funds management for the Gulf. Many of these funds had special legal
provisions and limitations.
For Hurricane Sandy, EPA received $600 million primarily for State Revolving Funds. Using the lessons
learned from this ARRA study, EPA organized a multi-disciplinary management team, reached out to state
officials immediately to understand their management goals and helped organize the President's Sandy
Task Force. Specifically, EPA developed an Internal Control Plan based on input from all stakeholders that
focuses efforts on commonly accepted risks and establishes agreed upon priorities that EPA and the states
will monitor. This effort helped to align EPA and state priorities.
September 2013 Page 23
-------
For both recovery efforts, EPA coordinated with other federal agencies that had the same challenge of
distributing large sums of money quickly to targeted recovery efforts. Also, EPA's Office of Water has
continued efforts to improve the capabilities of both the PBR and CBR data systems to make them more
user-friendly, with project details that are more useful to EPA and state users.
September 2013 Page 24
-------
APPENDIX 1: COMPILED EXECUTIVE SUMMARIES FROM THE SIX TASK REPORTS
-------
This page intentionally blank.
-------
EXECUTIVE SUMMARY - COST ESTIMATING
PURPOSE
This study seeks to examine the methods used for estimating costs of Clean Water and Drinking Water
State Revolving Fund (CWSRF and DWSRF) and Superfund projects funded by the American Recovery and
Reinvestment Act (ARRA). This study compares estimated costs to final project costs, with the objectives
of capturing successful approaches, determining the factors that contributed to variations between
estimated costs and final project costs, and identifying lessons learned.
METHODOLOGY
EPA contracted with Science Applications International Corporation (SAIC), and its subcontractor Toeroek
Associates, Inc., to review both estimated and final costs of CWSRF, DWSRF and Superfund projects
funded by ARRA. SAIC used two different approaches to review cost estimates and compare them to final
costs, one for the CWSRF and DWSRF programs, and one for Superfund. One major reason for the
different approaches was that the SRFs are primarily state-managed with cost estimates performed by
localities and utilities using the grant funds for clean and waste water projects, while the Superfund sites
were primarily federally managed.
For the SRF programs, SAIC performed the following activities: reviewed existing studies and information
on cost estimates; conducted state focus groups and reviewed state files with project-specific data;
analyzed nine individual projects' field change orders to assess their impacts on costs and scope; and
analyzed the data obtained from the focus groups and file reviews.
For the Superfund program, SAIC reviewed existing studies and information on Superfund cost estimating;
collected additional information on completed Superfund projects; interviewed EPA Superfund staff; and
reviewed and analyzed collected information and interview results.
FINDINGS
Upon completion of this study, SAIC prepared several observations and noted lessons learned from the
cost estimating process for EPA's consideration for both SRF and Superfund projects.
CLEAN WATER AND DRINKING WATER SRF PROGRAMS
The ARRA funding program favored projects that replaced existing infrastructure. These
relatively routine projects minimized the likelihood of costs increasing beyond original estimates
and typically did not require significant environmental review. ARRA funds enabled much-
needed maintenance of wastewater and water infrastructure, which often are the largest capital
assets of many communities. An ample supply of qualified contractors and laborers for this type
of work led to an increase in the number of competitive bids, which lowered costs.
Flexible scopes (e.g., replace aging pipelines in a generally defined area) allowed the quantity
of work to expand or shrink to match the estimated cost. The length of pipeline replaced or the
number of meters installed could be increased or decreased to fully utilize the funding available
without exceeding the funding amount.
September 2013 Appendix 1-3
-------
Cost estimates for a project evolve over time and each successive cost estimate adds its own
value to the process. Accepted engineering practices are used to prepare cost estimates. These
practices are uniform from state to state. Estimates (become more accurate with each iteration,
which provides useful information to the SRF programs. There is not some 'better' approach that
could be used to prepare cost estimates. The tight time constraints created by the oneyear
deadline to be under contract often favored projects with more developed cost estimates.
States generally identified prevailing economic conditions as the most significant factor
contributing to variances from costs estimates. In the early part of the ARRA program
commercial construction dropped off significantly because of difficulty in obtaining loans for
commercial and residential construction. This increased competition for municipal projects.
Grantees received many more bids for ARRA-funded projects than they had previously received
for similar projects. This increased competition reduced bid costs below earlier estimates.
States developed and implemented procedures to ensure ARRA loans closely matched
estimated project needs. The procedures implemented included: working with the grantees
during project development to enable a better understanding of project goals and contribute
their engineering experience in a collaborative manner; basing loan offers on project bid costs;
providing contingency amounts for construction to account for unforeseen circumstances; using
non-ARRA SRF funds for contingency amounts (money not used could be returned to the non-
ARRA SRF account); encouraging grantees with excess available funding to utilize the excess
money to improve the project (e.g., purchasing spare parts) while staying within allowable
expenditures; and prohibiting the use of ARRA funds to cover cost over-runs.
SUPERFUND:
The program has one standard method for cost estimating but the accuracy of the bids is
dependent on variables such as the quality of data known about the site. Factors that improve
cost estimating include a better defined site with fewer unknowns, timing of the estimates
(prepared closer to commencing the project) and understanding the extent of contamination.
Less complex sites lead to more accurate cost estimates. Smaller and less complex sites or
activities (e.g., road building) are easier to cost and lead to more accurate cost estimate.
Advanced sites with existing activities lead to more accurate cost estimates. Even complex
sites with remediation underway lead to better cost estimates then those initially presented in
the Record of Decision (ROD). The extent of contamination is better understood by this point and
the contractors are experienced with the site.
Superfund adapted more easily to ARRA deadlines and influx of funds. TheSuperfund program
adapted more easily to the influx of ARRA funding because it had ongoing and shovel-ready
projects or projects that had been through the ROD process. This allowed for more accurate and
timely cost estimating.
Existing contracting vehicles simplify award without changing scopes. The process to award
tasks was simplified as each Region had pre-existing remediation contracts/contractors in place,
ready and experienced with many of the sites or types of projects.
September 2013 Appendix 1-4
-------
Provide forum for successful approaches. States in the focus group discussions described several
techniques used to minimize differences between cost estimates and final project costs. EPA could
provide a forum for sharing these techniques among states.
September 2013 Appendix 1-5
-------
This page intentionally blank.
September 2013 Appendix 1-6
-------
EXECUTIVE SUMMARY- FUNDS MANAGEMENT
PURPOSE
This study seeks to capture the lessons from EPA, states, tribes, localities and other fund recipients'
management efforts related to the $7.2 billion ARRA funds appropriated to EPA. The scale, scope and new
requirements posed daunting management challenges to all of these stakeholders requiring new
guidance, processes and systems. Although there is no proposal before Congress for another major
investment similar to ARRA, EPA and its partners periodically face challenges managing substantial
funding for many projects with new terms, requirements and conditions, such as for Hurricane Sandy, and
have already used some of these lessons.
METHODOLOGY
ARRA, enacted in 2009, bolstered the economy, in part, by funding six EPA programs. These programs
included drinking water and clean water infrastructure projects through the State Revolving Funds (SRFs)
and environmental initiatives through Superfund, Diesel Emissions Reduction Act (DERA), Leaking
Underground Storage Tank (LUST) and Brownfields programs. EPA contracted with SAIC, and their
subcontractor Toeroek Associates, Inc., to review the activities and process for funds distribution,
management and reporting for each of the six programs that received ARRA funding. The objective of the
review was to capture, verify and analyze the critical lessons learned and successful approaches related to
ARRA funds management. To achieve this objective, the SAIC Team gathered information on three specific
aspects: timely obligation and expenditure of funds, additional mandates of ARRA (Section 1605 Buy
American mandate, Section 1606 Davis-Bacon Prevailing Wage mandate, Section 1512 reporting
requirements, and Green Project Reserve mandate), and grants management processes for the
previously mentioned programs.
The data collection method used interviews with EPA staff and state focus groups, combined with
additional information from literature, websites and databases. SAIC and Toeroek gathered the majority
of the information from 47 EPA staff interviews and 12 state focus groups, which included 108 state
personnel and 9 funding recipients or funding recipient consultants/contractors.
EPA distributed approximately 86 percent of its $7.2 billon ARRA funding to individual states through
capitalization grants based on an allocation calculation. This was approximately twice EPA's historical
annual grants per fiscal year for the SRF drinking and clean water programs. Each state met the
accelerated, first ever imposed deadline of its kind, by surmounting difficult barriers and addressing
unforeseen challenges. All funds were obligated without any re-obligation within the one-year timeframe;
more than 90 percent of the funds were expended within the three-year or specific programmatic
deadlines.
September 2013 Appendix 1-7
-------
FINDINGS
In this study, the SAIC Team discerned several important aspects of ARRA funds management processes.
Following the Executive Summary, the full report presents the findings in three major categories:
Challenges, Successful Approaches and Recommendations.
The report addresses:
Changes in policies, processes and procedures, and evolution and response to those changes.
Factors that helped or hindered, as well as the challenges affecting different outcomes among
the programs, Regions, states and projects.
Successful approaches to addressing conflicts and implementing the ARRA-funded programs.
CHALLENGES AND SUCCESSFUL APPROACHES
Based on the perspectives from all stakeholders, below are summaries of the major challenges described
and some successful approaches used.
Scale - Overall, the additional workload from the sheer magnitude of the ARRA funding amounts,
coupled with the work associated with the four mandates, produced serious demands on EPA
and state staff. Everyone from EPA to states to local vendors worked above and beyond usual
schedules to meet ARRA goals. EPA Regions and states who overcame their initial reluctance to
hire new staff with ARRA Management and Oversight funds fared better in handling the
additional workload. Early and frequent communication among stakeholders, with regularly-
scheduled meetings and webinars that began before ARRA was even passed helped to form solid
working relationships. The enhanced communication and collaboration between EPA, states and
funding recipients were critical factors in getting through this intense period. Innovative,
streamlined management approaches were developed out of necessity, but remain today as
permanent program improvements. The ARRA 1512 reporting requirements helped states
prepare for requirements of the Federal Funding Accountability and Transparency Act.
Regional and State Variation - During ARRA implementation, it became clear that there were 51
distinct SRF programs (50 states plus Puerto Rico); any changes made at the national level
affected these 51 programs differently. Existing, established processes and procedures served as
a good foundation on which to incorporate ARRA requirements. Modifying existing processes
made the changes easier to implement. The fast-paced nature of ARRA implementation
encouraged federal, state, and local organizations to 'flatten out their management process' by
eliminating unnecessary reviews at multiple levels, allowing for simultaneous rather than
sequential reviews, and providing information and updates to everyone at the same time, ideally
through web postings, to eliminate the time required for information to trickle down from one
level to the next. Some improvements have been incorporated; some remain as future goals. For
example, some states have shortened the period from application submission to contract award.
Additional Requirements - The four additional mandates that came with ARRA funding - Davis-
Bacon labor requirements, Buy American stipulations for iron, steel and manufactured goods,
Section 1512 reporting on expenditures, and the designation of 20 percent of ARRA funds for
'green' projects - created extensive challenges for EPA, states and funding recipients. The
September 2013 Appendix 1-8
-------
provisions were complex, guidance was often changing and the workload associated with the
mandates alone was significant. Some states opted to designate specialists to remain up to date
on the mandates and process the necessary paperwork. EPA HQ and Regions endeavored to
provide training and guidance to work through complex issues.
Inexperienced Recipients - Many states widened their customary range of grant recipients to
include new communities and new businesses, in an effort to 'spread the wealth.' These new
funding recipients, unfamiliar with EPA program requirements, had difficulties in handling the
accelerated schedules and the sizes of the projects awarded. Communities without sufficient
management experience, especially small towns with little or no town staff, turned to their state
program offices for assistance. The towns were very appreciative of the new activity in their
communities; in some instances, ARRA funding kept businesses from going bankrupt or laying off
personnel. State offices admitted though that their personnel were overextended in managing
projects for entities without sufficient staff or experience.
Conflicting Goals - The fundamental ARRA goal to inject money quickly into the economy
sometimes conflicted with the EPA goal to fund projects that would yield the greatest
environmental benefit. The need to find 'shovel-ready' projects meant that sometimes higher
priority projects were passed over in favor of other projects that were ready to implement.
Despite this conflict, ARRA funds enabled the completion of beneficial projects that would never
have been done otherwise. ARRA funded projects that yielded environmental and public health
benefits. The requirement to include 'green' projects encouraged engineers to add water and
energy efficiency technologies into their drinking water and clean water projects.
RECOMMENDATIONS
The numerous discussions with stakeholders in the funds management processes generated
recommendations for going forward. Recommendations in the following summary are not listed in any
priority order and are presented as guideposts for EPA and states' existing programs and any new
initiatives for the existing programs.
Think strategically about information that will be needed in the future. Involve all stakeholders
(multi-disciplinary and multi-organizational) in assessing what information will be needed to
measure outcomes or results. Identify and agree on specific monitoring data needs prior to
implementation; collect the necessary data to track incremental milestones that measure the
progress or status of activities.
Work as a team to achieve process efficiencies. Collaborate with all stakeholders through
working groups to develop plans, policies and guidance prior to and throughout implementation.
Attempt full transparency in communication; share information with everyone simultaneously to
avoid unnecessary delays as information slowly spreads. Communicate with one voice to avoid
conflicting messages from different federal agencies. Create short-term working groups focused
on specific issues or processes.
Follow a strategy with clearly defined goals; eliminate conflicting goals. Clarify primary and
secondary goals so that states do not have to wrestle with tough choices. Consider incorporating
new requirements as incentives or goals instead of mandatory requirements where possible.
Provide for flexibility at the state and local level to allow for adjustments to the local situation.
September 2013 Appendix 1-9
-------
Delegate the decisions to the local level as much as possible because the local situation will
dictate what or how to apply requirements. Simplify requirements to enable quick
implementation and better compliance. Target oversight to ensure that states with historical
program or financial management issues receive attention early in the process.
Use effective tools and processes. Develop and maintain a website to provide one central place
for updated policies and guidance; remove obsolete guidance to avoid confusion. Use the
website to disseminate successful approaches developed by stakeholders. Provide reasonable
deadlines and establish internal indicators to alert managers of potential problems/delays.
Provide sufficient resources to handle the increased workload. Dedicate experienced staff to the
effort; reassign staff from other program areas. Make temporary hires and use contractors.
Some of the lessons learned from ARRA implementation have already been applied by EPA managers in
their regular programs and in response to funding for Hurricane Sandy recovery. Experience with ARRA
provided managers insight on what questions to ask. The lessons about early and frequent
communications among all stakeholders led EPA to create a communications network that enables
questions to be posed and discussed. The Hurricane Sandy Task Force crafted guidance and policy to
make guidance clearer initially and reduce the need for re-work. This included specifying clear financial
deadlines and clarifying crucial definitions.
September 2013 Appendix 1-10
-------
EXECUTIVE SUMMARY - INFORMATION SYSTEMS DEVELOPMENT AND
ENHANCEMENT
PURPOSE
ARRA, enacted in 2009, required EPA to rapidly obligate its ARRA funds to many hundreds of projects in
six environmental programs across all 50 states, tribes and territories and to satisfy the special reporting
requirements for oversight, accountability and transparency. EPA contracted with SAIC to assess EPA's
information systems development and enhancement efforts to implement ARRA.
METHODOLOGY
SAIC used interviews of EPA senior staff combined with additional information from literature and
databases to capture, verify and analyze the critical lessons learned and successful approaches related to
EPA's system development and enhancement efforts.
During the interviews, EPA respondents identified a number of challenges, successful approaches,
recommendations/lessons learned and systems and process changes that were initiated due to ARRA, but
have had more expansive impacts to EPA's programs beyond ARRA.
FINDINGS
More than half of the respondents identified the following two challenges: 1) unclear and evolving Office
of Management and Budget (OMB) and Recovery and Transparency Board (RAT Board) guidance, and 2)
the time constraints for ARRA implementation. The majority of the respondents identified successful
approaches that helped them meet the deadlines for ARRA implementation: 1) use existing funding or a
simple process for obtaining funding for systems modification, 2) use existing staff, 3) modify existing
systems, and 4) use existing contracts. Using existing personnel and infrastructure made it possible for
EPA to meet the aggressive ARRA implementation schedule.
Respondents described the process and systems changes they made as a result of the ARRA program and
plan to implement and maintain on a permanent basis. One EPA program established a systematic
process for future system updates. Other programs expanded system capabilities to improve data
management for the entire program, not just ARRA-related functions (e.g., providing states direct-entry,
expanding the use of funding recommendation templates, improving accounting consistency between
multiple funds management systems, and adding data elements to enable storage of estimated and actual
data.) In addition to these process and system changes which EPA offices made permanent, three
respondents noted that ARRA brought an increased focus on geospatial elements of data reporting.
RECOMMENDATIONS
Two of the recommendations proposed by the EPA interviewees were directly related to challenges that
they faced. They might be possible to implement during future large-scale funding efforts.
Respondents recommended providing additional lead time to allow for strategic planning related
to systems changes. The short implementation schedule did not allow time for strategic planning.
September 2013 Appendix 1-11
-------
One EPA respondent strongly recommended that the responsible entities from all levels -
highest management level to the lowest level (individual EPA offices) define the systems
requirements before starting the effort (and not change them when implementing the
requirements). (During ARRA, defining the system requirements was not possible due to the
ongoing changes to the OMB and RAT Board guidance and the changes made to the data
requirements only a few weeks before the first reporting period.)
September 2013 Appendix 1-12
-------
EXECUTIVE SUMMARY - GREEN PROJECT RESERVE
PURPOSE
This study seeks to capture the benefits and outcomes related to the Green Project Reserve (GPR)
requirements of ARRA funds allocated to the EPA Clean Water and Drinking Water State Revolving Fund
(CWSRF and DWSRF) programs. EPA and states had to target almost $1.2 billion for green projects - one
of the largest single goals that EPA ever had to meet.
METHODOLOGY
EPA contracted with SAIC, and their subcontractor Toeroek Associates, Inc., to review the benefits and
outcomes of the green projects undertaken to fulfill the GPR requirements for the CWSRF and DWSRF
programs under ARRA. 'Green' projects include those that deal with green infrastructure, water
efficiency, energy efficiency or those that are environmentally innovative. The objective of the review was
to gather and report on information related to both primary and secondary outcomes of State Revolving
Fund (SRF) green projects. To achieve this objective, the SAIC Team reviewed existing documents and
information related to green projects; reviewed existing EPA databases such as the CWSRF Benefits
Reporting system (CBR) and the DWSRF Project Benefits Reporting system (PBR); categorized clean water
and drinking water projects; conducted focus group discussions and file reviews in six states; and
documented the results of these analyses.
Several challenges became evident as SAIC began collection and analysis of data for this study. ARRA did
not mandate that states collect data that would allow for measurement or documentation of primary or
secondary environmental benefits of completed green projects. In addition, the downloaded data that
SAIC received from the CBR and PBR databases represented only a snapshot of ARRA projects. Finally,
finding published data to enable SAIC to identify environmental outcomes was particularly difficult, as
completed projects had only been in operation for a few years at most.
FINDINGS
SAIC primarily based its analyses on perspectives from six state focus groups. SAIC was largely unable to
find existing studies that included quantitative analyses of environmental outcomes of green ARRA
projects. The one exception is a draft partial analysis of anticipated environmental benefits of ARRA-
funded GPR projects conducted by Industrial Economics, Inc. (lEc). The authors of this study were only
able to find data for about a third of the GPR projects. Similarly, a review of EPA's CBR and PBR databases
produced limited findings; data was not entered consistently, so SAIC was unable to compare the
amounts spent on different types of green projects.
The six state focus groups offered largely qualitative information on primary and secondary
environmental benefits of GPR projects. Participants noted that there was no documentation of
environmental benefits from ARRA-funded GPR projects nor was it requested or required by EPA, but all
of the state participants were able to identify some secondary outcomes during discussion sessions.
September 2013 Appendix 1-13
-------
BENEFITS AND LESSONS LEARNED
SAIC found that the majority of GPR projects undertaken by the CWSRF and DWSRF programs across the
nation involved the installation or replacement of water meters (113 projects), the rehabilitation or
replacement of leaking pipes (41 projects), the construction of wastewater treatment plants (183
projects), and stormwater management (194 projects). SAIC's review of these projects and those
discussed during the state focus groups yield several anecdotal observations of primary and secondary
environmental benefits, including:
The overall improvement of environmental awareness of project area residents.
Increased community pride, enhanced property values and overall neighborhood beautification.
Increased efficiency in water meter reading operations, which also results in less vehicular
emissions, better use of water utility staff, improved customer service and increased funding for
utilities to use in other areas of water line maintenance.
Large cost savings to municipalities due to reduction in energy consumption by water treatment
and distribution activities, allowing for investment in other community improvement projects.
An overall increase in the innovative attitude of municipalities and engineers, and a desire to
pursue future green projects.
In addition to these environmental benefits, SAIC observed several notable lessons learned, including:
ARRA projects that were 'categorically' green did not require a business case to document
expected environmental benefits, because the primary benefits were assumed.
Existing project priority scoring mechanisms in Intended Use Plans were not designed to capture
green project benefits that would address specific green priorities.
The short time frame available to EPA to develop ARRA guidance for the state SRFs may have
resulted in less than optimal guidance in some areas.
RECOMMENDATIONS
Upon completion of this study, SAIC formulated several recommendations for EPA's consideration. EPA
could consider:
Requiring business case documentation as well as quantifying primary and secondary
environmental benefits for all completed projects, which would be useful in quantifying total
environmental benefits.
Tracking and evaluating costs and secondary benefits more closely.
Developing guidance on assessing secondary benefits of green projects.
Encouraging cooperation between states and outside organizations to leverage the ability to
document green project benefits.
September 2013 Appendix 1-14
-------
EXECUTIVE SUMMARY - INNOVATIVE TECHNOLOGIES
PURPOSE
ARRA, enacted in 2009, bolstered the economy, in part, by encouraging the adoption of innovative
technologies, particularly for drinking water and wastewater treatment. EPA contracted with SAIC to
review examples of innovative technologies adopted through ARRA-funded projects. The objective of the
review was to capture examples and successful approaches from technological innovation made possible
by various EPA programs that distributed ARRA funding. To achieve this objective, SAIC gathered
information regarding nine innovative technologies in the Drinking Water State Revolving Fund (DWSRF)
and Clean Water State Revolving Fund (CWSRF) programs that received support through ARRA funds. The
findings are summarized below.
METHODOLOGY
For the qualitative analyses, SAIC interviewed local experts familiar with the drinking water and
wastewater projects as well as company representatives from the major suppliers. SAIC also reviewed
studies of technological innovation and other materials provided by interviewees.
FINDINGS
ARRA funding supported the adoption of innovative technologies. Despite the challenging
circumstances, ARRA funding supported the adoption of innovative technologies. The scale and scope of
the adoption varied in expected ways (e.g., widespread adoption of advanced metering technologies and
regional adoption of tank mixers). These are low-cost or low-risk technologies in that technology failure
does not result in noncompliance.
ARRA funding motivated water utilities to consider innovative technologies and assume some
additional risk. Normally risk-averse water and wastewater utilities adopted innovative technologies that
were potentially cost-effective and/or satisfied additional operating constraints provided their
performance claims could be realized. The utilities interviewed evaluated both conventional and
innovative technologies before selecting the innovative technologies. It is possible that favorable ARRA
funding conditions such as principal forgiveness encouraged utilities to accept a little more risk than they
might normally accept. It is clear that the ARRA funding enabled several projects - large and small - to
proceed.
ARRA funding supported adoption of innovative technologies that improved environmental protection.
All recipients interviewed were satisfied with the performance of the selected innovative technologies.
Not only did the technologies help them achieve their main compliance objective, but they also provided
green benefits, such as energy savings, less infrastructure, less sludge production, less water use and loss,
and less chemical use.
ARRA funding positively affected innovative technology sales during the economic recession. Funding
made available specifically for innovative technology projects motivated utility managers interviewed to
try new technologies. Funding conditions, such as the Buy American provision that increased the burden
on funding recipients, also led some businesses to prefer U.S. manufacturing sources. They learned that
onshore manufacturing increased their control over product quality and lowered shipping costs.
September 2013 Appendix 1-15
-------
Successful demonstration projects sparked follow-on business for technology vendors. In each instance
of a first-time installation for vendors interviewed, the vendor has experienced or anticipates follow-on
work because of demonstrated performance and positive word-of-mouth. This suggests the need for
demonstration projects that establish the capabilities of new technologies and also identify whether they
are ultimately more cost-effective than conventional alternatives.
RECOMMENDATION
Provide a longer timeframe for innovative technology projects. Innovative technologies require time for
concept development, design, pilot demonstration, and commercialization phases. Consider extending
obligation and expenditure deadlines to encourage innovative technology development.
September 2013 Appendix 1-16
-------
EXECUTIVE SUMMARY - ECONOMIC IMPACTS OF LEVERAGED PROJECTS
PURPOSE
To help EPA better understand how ARRA funding was used to successfully leverage local resources to
achieve short-term and long-term economic benefits, SAIC studied the impacts of several ARRA-funded
projects. A crucial goal for ARRA, enacted in 2009, was for local communities to leverage funds in their
local economies to stimulate economic activity during the recession. To understand how particular
programs leverage resources and expand local economic activity, EPA sought to capture some of the
successful examples of ARRA programs and funding recipients leveraging resources and strengthening
local economic activity.
EPA distributed the vast majority of its ARRA funding through programs designed to assist communities
making investments in infrastructure such as water treatment plant upgrades or pipeline replacements or
industrial site cleanups. These ARRA-funded investments potentially had two types of economic impact.
First, the infrastructure expenditures increased the demand for locally produced goods and services. This,
in turn, increased the demand for 'upstream' goods and services that produce the goods and services
needed by the infrastructure project. Thus, a dollar of infrastructure spending led to more than one dollar
of regional economic output. Second, the infrastructure investment may result in long-term economic
benefits by achieving environmental and/or development goals such as reducing health risks or
supporting local growth objectives.
Infrastructure investments such as water treatment plant upgrades to meet regulatory standards for
water quality can be expensive. For some municipalities, these kinds of infrastructure investments pose a
fiscal challenge when they have to raise fees and taxes to repay the capital construction loans or bonds.
ARRA funding provided an opportunity for these recipients to leverage local resources using federal
funding to implement such investments.
The study objectives are to quantitatively estimate the ratio of total regional economic growth relative to
the original project investment, called an "impact ratio," and to qualitatively address the long-term
benefits of the investment. To achieve these objectives, SAIC gathered information on nine ARRA-funded
projects in the Drinking Water State Revolving Fund (DWSRF), the Clean Water State Revolving Fund
(CWSRF), and the Brownfields program.
METHODOLOGY
For the qualitative analysis, SAIC used two information sources. SAIC interviewed local experts familiar
with the infrastructure projects and reviewed studies of economic benefits of environmental regulations
for projects that were part of a regulatory compliance plan.
For the quantitative analysis of regional economic impacts, SAIC collected detailed project expenditures
data and used the Regional Input-Output Modeling System (RIMS II) to estimate local economic impacts.
The RIMS II model was developed by the U.S. Bureau of Economic Analysis (BEA) to estimate the effect of
direct expenditures on indirect expenditures and induced expenditures in a region. Direct expenditures are
those paid to implement the project (e.g., laying a new pipeline), while indirect expenditures represent
the additional economic impact of increases in the demand for 'upstream' goods and services (e.g., piping
manufacturers or excavation companies), and induced expenditures represent the additional economic
September 2013 Appendix 1-17
-------
impact of increased demand of consumer goods and services attributable to 'upstream' labor earnings.
The longer each dollar of direct expenditure can remain within a local community - going from vendor to
vendor in the form of new revenue - the higher its regional impact will be. This is the multiplier effect that
the RIMS II estimates. The multiplier effect is limited by the tendency for money to flow out of a region to
pay for 'imported' goods and services, which is often called leakage. These are not imports in the sense
they are goods or services produced outside the United States; any good or service that originates outside
the local region is considered an import in RIMS II.
FINDINGS
Based on SAIC's interviews of individuals associated with nine ARRA projects and analysis of data, the
major case study findings regarding economic impacts are as follows.
The projects examined will provide the affected communities with a variety of medium- and long-term
environmental and economic benefits. SAIC's qualitative analysis shows that the environmental benefits
stem from meeting various regulatory compliance requirements. The benefits include human health risk
reductions and improvements in surface water quality from reduced nutrients, sediments and toxics in
wastewater discharges. The DWSRF projects will also reduce water use and/or energy production costs.
Both DWSRF and CWSRF projects will have some additional tangible financial benefits in the form of cost
savings for the utility and for customers. Two of the projects will also facilitate community economic
development objectives by increasing utility capacity to support residential and commercial growth. A
third project supports economic growth through the renewal and sale of urban land to industrial and
commercial businesses.
The case study project expenditures unambiguously achieved the objective of stimulating local
economies during the recession. The regional economic impact per dollar of project expenditure ranges
from $1.58 to $2.96 across the nine case study projects. These per-dollar estimates represent the
quantifiable direct, indirect and induced expenditures in the regional economies that can be attributed to
the infrastructure projects. These values are based on the impact ratios that SAIC estimated using RIMS II.
The regional economic impacts were higher for projects that could rely primarily on local sources of
goods and services. The projects that retained the highest proportion of direct expenditures in the local
community generally have higher impact ratios because the RIMS II multipliers applied to a majority of
total project expenditures. Projects that required imports of expensive materials tend to have lower
impact ratios. Because case study projects with treatment plant upgrades were more likely than other
project types to have expensive treatment equipment imports, these projects had lower regional
economic impacts.
These findings are subject to constraints that can lead to potential errors, uncertainties and biases. These
constraints arise from factors such as a having limited number of case studies, which restricts the extent
to which regional economic impact results can be generalized, and having a project mix specific to a short
timeframe (2009 to 2011).
September 2013 Appendix 1-18
-------
APPENDIX 2: TOPICS COVERED IN REVIEW OF EXISTING STUDIES
-------
This page intentionally blank.
-------
TOPICS COVERED IN REVIEW OF EXISTING STUDIES
__^^HH^^^H__
Improvements needed to OMB ARRA Implementation
Guidance
Awarding assistance agreements and contracts to
recipients with open audit recommendations
Contractor performance evaluations
Contracts and grants workforce and workload
EPA Management Integrity guidance for conducting
internal control reviews
Financial activity reporting
Financial monitoring reviews
Grant accrual methodology
Grant activity documentation errors
Grant project delays
Grant recipient reporting and data quality review
Grants terms and conditions deficiencies
Green reserve projects
Impact of ARRA funds on economically disadvantaged
communities
Independent government cost estimates
Interagency agreements
Management of Superfund special accounts
OIG oversight efforts
Process improvements to ensure projects meet ARRA
deadline
Promoting grants competition
Open audit recommendations on grants, contracts and
interagency agreements
Recovery of Superfund removal costs for projects not on
the National Priority List (NPL)
State oversight of subrecipients
PROGRAM(S)
All
All
All
All
All
All
Superfund
All
DERA
DERA
All
LUST
CWSRF, DWSRF
All
Superfund
Superfund, DWSRF
Superfund
All
DWSRF
DERA
All
Superfund
CWSRF, DWSRF
ffft
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
2
1
1
1
1
2
OIG March
2009a
OIG July 2009
OIG April 2010
OIG October
2010
OIG April 2009c
OIG June 2010
OIG January
2011
OIG August
2009
OIG March
2011c
March 2011b
OIG October
2009
OIG September
2010
OIG November
2010b
OIG February
2010b
OIG April 2011
OIG February
2010a
OIG November
2010a
OIG March
2009b
OIG March
2010a
OIG March
2011a
OIG December
2009
OIG March
2010b
OIG April
2009b
OIG April
2009a
OIG September
2008
OIG August
2011
September 2013
Appendix 2-1
-------
This page intentionally blank.
September 2013 Appendix 2-2
------- |