c/EPA
November 2011
EPA-100-F-11-029
Evaluation of Region 7
Tribal Grants
Programmatic Processes
Promoting Environmental Results
i 1
Through Evaluation
-------
ACKNOWLEDGEMENTS
This report, Evaluation of the Region 7 Tribal Grants Programmatic Processes, was developed for the
United States Environmental Protection Agency's (EPA) Office of Policy under Contract EP-W-10-002
between EPA and Industrial Economics, Inc. (lEc) of Cambridge, Massachusetts. The evaluation team
(also referred to as the "evaluators" throughout this report) consisted of Andrew Schwarz, Victoria Diaz-
Bonilla, Jane Zhou, and Tom Santoro.
Ashley Betts and Heather Hamilton of EPA's Region 7 Office of Policy and Management and Office of
Tribal Affairs, respectively, provided critical assistance and background information on the Tribal Grants
Program throughout the course of this study. Britta Johnson of EPA's Office of Policy Evaluation Support
Division provided technical support and advice for the evaluation.
This report was developed under the Program Evaluation Competition, sponsored annually by EPA's
Office of Policy. Program Evaluation is one of the performance management tools EPA uses to assure
itself, the public, and other interested stakeholders that EPA programs are protecting human health and
the environment effectively and efficiently. The information obtained through program evaluations can
shed light on whether EPA programs are meeting their goals and objectives, provide the evidence and
road map needed to replicate successes, and identify those aspects of EPA programs needing
improvement. To access copies of this or other EPA program evaluations, please visit EPA's Evaluation
Support Division's website at http://www.epa.gov/evaluate.
-------
ACRONYMS
BMP: Best Management Practice
CAA: Clean Air Act
CERCLA: Comprehensive Environmental Response, Compensation, and Liability Act
CWA: Clean Water Act
EPIC: Environmental Protection in Indian Country
GAP: General Assistance Program.
GMO: Grants Management Office
IGMS: Integrated Grants Management System
NOFA: Notice of Funding Availability
PO: Project Officer
QAPP: Quality Assurance Performance Plan
RTOC: Regional Tribal Operations Committee
TA: Technical Advisor
-------
TABLE OF CONTENTS
EXECUTIVE SUMMARY
1. INTRODUCTION
Scope 2
2. METHODS
Information Collection and Analysis 3
Identifying Evaluation Questions 3
Review of Existing Materials 6
Interviews 6
Data Analysis 6
3. FINDINGS
Overview 8
Efficiency, Consistency, and Transparency 9
Program Processes 9
Program Office Organization 11
Communication with the Tribes; Issues Affecting Workplan Negotiation n
Impact of EPA and Tribal Staff Turnover 14
Need for More Direct Communication 14
Need for More Training 14
Collaboration Among Region 7 Programs 15
4. RECOMMENDATIONS
Overview 16
Implement Grant Programs Consistently as Practicable, Adhering to Best
Management Practices 16
Identify Organization Design Features that Facilitate of Hinder Efficient Operation
of the Grant Program and Collaboration Among Programs 17
Work to Improve Communication with Tribes 18
Better Equip EPA Staff to Work Effectively with Tribal Staff 18
Formalize the Process to Review Grants 19
in
-------
APPENDICES:
Appendix A: List of Tribes in EPA Region 7
Appendix B: Grant Descriptions
Appendix C: Evaluation Methodology
Appendix D: Literature Review
Appendix E: List of Interviewees
Appendix F: Sample Interview Guide
Appendix G: Process Maps
Appendix H: Best Management Practices
IV
-------
EXECUTIVE SUMMARY
The U.S. Environmental Protection Agency (EPA) is responsible for administering Federal environmental
statutes on all U.S. land. Tribal National governments play an important role in the design and
administration of tribal programs to ensure clean water, air quality, proper waste management, and
safe management of pesticides and other toxic chemicals. To help Tribes build capacity and administer
environmental programs, EPA provides guidance and technical support, and issues grants, to federally
recognized Tribal governments and intertribal consortia.
In EPA Region 7, some tribal representatives have expressed concerns about the programmatic grants
processes. These points, albeit anecdotal in nature, encompass a variety of issues, including that the
grants negotiation process is too lengthy, that feedback related to grant products is not timely or
productive, and that tribal priorities are undervalued in workplan activities. In order to better
understand these concerns and to ensure that the programmatic grant processes are as efficient,
consistent, and transparent as possible, the Region 7 Tribal program and the Regional Tribal Operations
Committee (RTOC) have formed a subgroup charged with reviewing and improving communications
around tribal grants.
In support of this review, EPA performed an evaluation of the tribal grant program to assess how grants
are currently handled and to make recommendations to improve Region 7 processes. EPA contracted
with Industrial Economics, Inc. to assist in conducting a process evaluation, focusing on four of the grant
programs: the Clean Air Act 103 (CAA) program, the Clean Water Act 106 (CWA) program, the
Brownfields 128(a) program, and the General Assistance Program (GAP).
METHODS
Working closely with representatives from EPA headquarters and Region 7, the evaluator developed a
program logic model that outlined the basic structure and design of the tribal grant process and
identified the expected outcomes associated with the grant process. Using the logic model and a set of
questions originally developed by Region 7, the evaluation team refined these into a series of key
questions that provided the framework for the remainder of the evaluation.
1. How can Region 7 make its tribal grant processes more efficient, consistent, and transparent
across Regional program areas as well as tribes?
2. How can Region 7 improve its communication and coordination with tribes throughout the
entire grants process, from negotiation of grant applications to the management of activities
conducted under the grants?
ES-1
-------
3. Are there ways for the different Region 7 programs to coordinate and collaborate for improved
(e.g., consistent) grants management and tribal relations, while still effectively utilizing their
specific programmatic expertise?
To answer these questions, the evaluator relied on two major approaches: (1) a review and analysis of
existing materials from EPA, and (2) interviews with key technical and administrative personnel (e.g.,
POs and grant administrators) in Region 7 and other EPA regions, as well as with tribes in Region 7.
The information collected and reviewed by the evaluator included, but was not limited to:
• National Program Guidance documents, GAP Guidance documents, and regional guidance
documents;
• Tribal assistance protocols;
• The tribal grants best management practices developed by the regional and tribal
communication workgroup;
• Process maps and related information developed by the program grants office to describe the
steps they take to solicit and award grants; and
• Notes from the RTOC meetings.
Interviews with EPA Region 7 personnel, tribal staff, and staff from other EPA Regions served as the
primary source of information collected for this evaluation and as the basis for many of the report's
findings.
DATA ANALYSIS
The evaluator's analysis and findings are based primarily on the results of the interviews, supplemented
by material gleaned from the background information. The evaluator first took the information provided
from the interviews and organized the results around the key evaluation questions. To better
characterize the comments obtained, the evaluator noted common themes and distinguished between
ideas raised by certain groups
FINDINGS
The findings are summarized below:
EFFICIENCY, CONSISTENCY, AND TRANSPARENCY
We examined the steps that each program office took in administering the grant process and found that,
for the most part, the programs follow similar steps. Some variation does exist and results from the
differences in underlying requirements or elements of the programs themselves. The important
distinctions and issues follow not from these programmatic differences, but rather from the discrepancy
in how the programs implement the steps of the process. The relationship between the program
officials and the tribal representatives and the speed and effectiveness of the communication between
them has the greatest impact on the efficiency and consistency of grant implementation. In addition, we
found that to some degree, differences in the way that personnel in regional programmatic offices are
ES-2
-------
organized impacts the relationships with the tribes and the efficiency of the grant process. While no
single organizational structure is necessarily superior or should be adopted across the board, the
differing approaches can impact both internal coordination within the Region and the interaction
between the Region and the tribes.
COMMUNICATION AND COORDINATION WITH TRIBES
When analyzing the grant solicitation, review, and award process, the biggest "sticking point" occurs
around workplan negotiation. Many of these problems had a common underlying theme:
communication issues and personality clashes between EPA and tribal personnel. Perceived lack of
responsiveness to tribal concerns and delays in responding to inquiries were among the issues raised by
tribes in discussing communication issues that affected the workplan negotiations. On several occasions,
the evaluator heard from the tribes that the relationship with the EPA program officer was the most
important factor in an efficiently run grant. Turnover of staff, both at EPA and within the tribes, can have
a detrimental impact on this relationship and, therefore, the efficient administration of grants. The
need for more frequent face-to-face communication was a common viewpoint expressed by the tribes.
COORDINATION AND COLLABORATION WITHIN REGION 7
With regard to coordination between the Region 7 tribal grant programs, some regional respondents
indicated that they thought the GAP program POs or the Grants Management Office was responsible for
reviewing grants and identifying issues of concern, including duplication of effort or grants submitted to
the inappropriate program. While this type of review on the programmatic side (e.g., determining the
appropriate grant vehicle) did occur on an informal and somewhat ad hoc basis, there is no formal
mechanism for reviewing the grants across programs. The Grants Management Office does in fact
review each grant but with a focus only on the administrative components and not on the programmatic
issues.
RECOMMENDATIONS
Based on the key issues that the Region wanted addressed in the evaluation and an analysis of the
findings, the evaluator has identified the following set of recommendations for EPA to consider:
• Implement grant programs as consistently as practicable across all grant programs, following the
Best Management Practices developed by the Tribal/EPA Grants and Communications
Workgroup;
• Identify organizational structures that facilitate or hinder efficient operation of the grant
program and collaboration among programs;
• Work to improve communications with the Tribes, especially with regard to workplan
negotiations;
• Better equip EPA staff to work effectively with Tribal staff, who are often new or inexperienced;
and
ES-3
-------
Formalize the internal process to review grants among programs to ensure that activities are
funded under the right grants and to limit duplication of effort.
ES-4
-------
INTRODUCTION
The U.S. Environmental Protection Agency (EPA) is responsible for administering Federal environmental
statutes on all U.S. land. Tribal National governments play an important role in the design and
administration of tribal programs to ensure clean water, air quality, proper waste management, and
safe management of pesticides and other toxic chemicals. To help Tribes build capacity and administer
environmental programs, EPA provides guidance and technical support, and issues grants, to federally
recognized Tribal governments and intertribal consortia.
The states of Nebraska, Kansas, Missouri, and Iowa comprise EPA Region 7 and are home to nine tribal
nations (see Appendix A for a full list of the tribes). EPA currently awards cooperative agreements
(hereafter referred to as grants) to seven of the nine tribes. Tribes use these grants as one source of
funding to acquire the skills and capacity to develop, manage and implement environmental programs.
The grants include the General Assistance Program (GAP), designed to enhance tribal capacity as well as
to help with implementation of media-specific tribal grants. Media-specific grants include the Clean Air
Act (CAA) 103, the Clean Water Act (CWA) 106, and the Brownfield Response Program under the
Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA 128(a)), funding to
implement the pesticide circuit rider, CWA 319, and Performance Partnership programs.
Within the region, technical and administrative project officers (POs) manage tribal grants and work
with the regional Grants Management Office (GMO) to ensure that grant administration requirements
are met and that grant activities achieve desired results.
POs oversee the entire grant process from the initial notification of tribes that funding is available
through the award and closeout. In the grant community, POs are generally classified as either
administrative or technical and their responsibilities differ accordingly; however, many of the individuals
who handle tribal grants in Region 7 fulfill both functions1. Administrative duties include reviewing the
application, budget, and workplan for completeness and reasonableness; creating funding
recommendations and commitment notices for input into EPA's electronic grants award system; and
reviewing quarterly reports to ensure the workplan activities are commensurate with the funding
drawdowns. Technical duties involve reviewing the workplan and reports for technical merit and
comparing planned activities with program guidance, regulations, and statutes, and providing guidance
on technical activities during the grant period and meets with technical tribal personnel as needed.
EVALUATION SCOPE
In the past some tribal representatives have expressed concerns about the programmatic grants
processes. While often anecdotal in nature, tribes have raised concerns, including that the EPA grants
negotiation process is too lengthy, that feedback related to grant products (work plans, reports, and
Only within the water program are the administrative PO and technical PO separate positions.
-------
Quality Assurance Performance Plans (QAPP)) is not timely or productive, and that tribal priorities may
be undervalued in work plan activities. Moreover, tribes have noted that poor communication often
impedes effective grant management. In a survey that EPA Region 7 conducted in 2006, over 40 percent
of tribal environmental staff indicated that they felt there was a communication gap with Region 7 staff
on programmatic and administrative matters. EPA officials also want to improve the efficiency of the
grant process, noting that while tribal grants represent only a small percentage of all the grants issued
by the Region, they take up a relatively large proportion of EPA's grants staff time.
In order to better understand these concerns and to ensure that the programmatic grant processes are
as clear and efficient as possible, the Region 7 tribal program and the Regional Tribal Operations
Committee (RTOC) have formed a subgroup charged with reviewing and improving communications
around the tribal grants, specifically focusing on increasing transparency to improve grant
implementation.
In support of this review, EPA initiated this evaluation of the tribal grants program to assess how grants
are currently handled and to make recommendations to ensure that grants are addressed efficiently,
consistently, and transparently. In addition, EPA wanted to identify recommendations that would
encourage effective communication between Region 7 staff and tribes as well as collaboration among
Region 7 programs. This evaluation focuses on the Agency's tribal grant allocation, approval, and
management practices, rather than on grant products or activities conducted under the grants by tribes.
The evaluation centers around four Region 7 grants: CAA 103, CWA 106, CERCLA 128(a), and GAP
(descriptions of these four grants are included as Appendix B)., with the understanding that examining
this set of grants would address the issues and concerns that had been identified. The evaluation was
conducted with two key audiences in mind: the Region 7 personnel involved in the administration of the
grant programs, including their supervisors, and the tribes who receive grants from EPA.
EPA contracted with Industrial Economics, Inc. (hereafter referred to as the evaluator) to help conduct
the evaluation. The evaluator worked closely with representatives from EPA in designing and
conducting the evaluation, though the findings and recommendations reported herein are the
evaluator's. Together, Industrial Economics, Inc. and the EPA representatives comprised the evaluation
team.
-------
METHODS
INFORMATION COLLECTION AND ANALYSIS APPROACH
This chapter describes the evaluator's approach for assessing the processes underlying the tribal grant
program. A detailed methodology, developed at the beginning of the evaluation, is included in
Appendix C. The analysis sought to identify issues affecting the efficient, consistent, and transparent
administration of the tribal grant process and to develop recommendations to address those issues,
while at the same time identifying opportunities for both internal and external coordination and
collaboration throughout the process.
IDENTIFYING EVALUATION QUESTIONS
In support of this evaluation, EPA Region 7 submitted a proposal to the Program Evaluation
Competition, conducted by EPA's Environmental Services Division (ESD). As part of its submission, the
Region included a series of draft evaluation questions that were then modified by the evaluation team.
The logic model, shown in Figure 1, is a graphical representation of the steps involved in awarding and
managing the grants, as well as the outcomes achieved by tribes following implementation of specific
grant activities. While post-award grant activity outcomes are shown in the logic model for the sake of
completeness, they were beyond the scope of this evaluation. Also, instead of surveying how effectively
tribes use grant funds, this evaluation revolves around the process behind grant awards. Furthermore,
the evaluation focuses on the steps leading up to the time the grant application is entered into the
Integrated Grants Management System (IGMS) (the top two lines of logic within the "Activities" and
"Outputs" columns).
As can be seen on Figure 1, the process outcomes (the main focus of this evaluation) are
• Fair and transparent grants processes across tribes and program areas;
• Effective communication and coordination between tribes and EPA;
• Valuation of tribal priorities and environmental needs; and
• EPA and tribal satisfaction with the process.
-------
Figure 1.
Resources/
Inputs
Logic Model for EPA R7 Tribal Grant Processes'
Activities
Outputs
Process
Outcomes
Grant Activity
Outcomes
• Funding for
EPA Region 7
program and
grant offices
• EPA Region
7 technical
and
administrative
personnel:
- Project
officers
- Grant
administrators
- Regional
Quality
Assurance
Manager
• EPA IT
systems:
- Integrated
Grants
Management
System (IGMS)
• Financial
Data
Warehouse
• Grant application
solicitation (if necessary)
and support
• Grant application
negotiation, review, and
decision (incl. workplan
and budget):
- Threshold criteria
- Legal, regulatory,
and programmatic
compliance
• Obtaining concurrences
and additional approvals
(if necessary)
• Grant management.
progress tracking, and
support
• Quality assurance
project plan (QAPP)
review
• Postings on EPA website.
Grants.gov, or in Federal Register:
• Requests for proposals
• Notices of funding availability
• Grant application templates
• Inputs to IGMS
• Notifications of application
receipt
• Application feedback and
technical advice
• Negotiation milestones
• Funding recommendations
• Commitment notices
• Grant approvals and signatures
• Congressional notification
• Notification and mailing of grant
awards to Tribes, or notification of
grant ineligibility/ denial
• Updates to IGMS Post-Award
Database and grant files
• Feedback on quarterly progress
and final performance reports
(incl, requests for revisions, if
necessary)
• Phone calls and in-person visits to
Tribes for technical support
• QAPP approval (or requests for
revisions, if necessary)
1
• Fair, simple.
consistent, time-
efficient, and
transparent grants
processes across
Tribes and
program areas
• Effective
communication
and coordination
between EPA and
Tribes
• Valuation of
Tribal priorities
and environmental
needs
• EPA and Tribal
satisfaction
• Efficient and
effective use of
grant funding by
Tribes
• Increased Tribal
capacity to
administer
environmental
programs and
tackle
environmental
challenges
• Tribal activities
to monitor and
improve
environmental
conditions
• Enhanced
environment and
human health
Note: Grant activity outcomes are shown in the logic model for the sake of completeness, but will were not be addressed by the evaluators during its their evaluation study. This evaluation was a process evaluation only,
focusing on the Agency's tribal grant allocation, approval, and management practices, rather than on grant products or activities conducted under the grants by tribes.
-------
Using the process outcomes identified in the logic model, the questions were developed to address the
degree to which these outcomes were being met and why leading to the following three major
questions and related sub-questions:
1. How can Region 7 make its tribal grant processes more efficient, consistent, and transparent
across Regional program areas as well as tribes?
a. How are Region 7 programmatic grant processes currently structured?
b. What causes the grants negotiation and performance reporting processes to extend longer
than they should, according to established guidelines? At which stages do the processes get
held up?
c. What are the differences in work plan and quarterly report requirements for the different
types of tribal grants (i.e., GAP grants and media-specific grants)?
d. How well are tribal priorities and environmental needs incorporated into the grant
processes?
2. How can Region 7 improve its communication and coordination with tribes throughout the
entire grants process, from negotiation of grant applications to the management of activities
conducted under the grants?
a. Do Region 7 project officers and tribes communicate on a consistent basis, or only when
issues arise?
b. Is there ever confusion about roles and responsibilities?
3. Are there ways for the different Region 7 programs to coordinate and collaborate for improved
(e.g., more consistent) grants management and tribal relations, while still effectively utilizing
their specific programmatic expertise?
a. Which grant steps are mandated by EPA Headquarters or statutes/regulations? Are there
different grants steps in the other EPA Regions?
b. How does Region 7 coordinate across programs? When a program issues a grant to a
particular tribe, does it always know which other grants the tribe may be receiving, and does
it coordinate its resources with these other grant programs?
c. Are there programmatic or environmental justifications for having some differences in the
requirements for the different types of tribal grants?
The evaluation questions, coupled with the process evaluation intent, necessitated a qualitative analysis
based on stakeholder input, informed by available information on the tribal grants process. The general
steps taken to collect and analyze the information, as well as the types of information used, are
described below. The approach consisted of two major components: a review and analysis of existing
materials from EPA and interviews with key EPA technical and administrative personnel (e.g., POs and
grant administrators) as well as with tribal environmental staff.
-------
REVIEW OF EXISTING MATERIALS
EPA staff provided the evaluator with a great deal of background material on the programmatic and
administrative functioning of the tribal grant program, which was supplemented by an independent
literature search. The information collected and reviewed by the evaluator included, but was not limited
to:
• National Program Guidance documents, GAP Guidance documents, and regional guidance
documents;
• Tribal assistance protocols;
• The tribal grants best management practices developed by the regional and tribal
communication workgroup;
• Process maps and related information developed by the program grants office to describe the
steps they take to solicit and award grants; and
• Notes from the RTOC meetings.
A full list of the materials reviewed by the evaluator is included in Appendix D.
INTERVIEWS
Interviews with EPA personnel and tribal staff served as the primary source of information collected for
this evaluation and form the basis for many of the report's findings. The interviews were designed to
gain perspectives from a variety of individuals involved in the tribal grants process. The evaluators
conducted interviews with Region 7 staff and their supervisors from the grants management and tribal
affairs offices as well as with staff and management from the four programmatic offices evaluated. We
also spoke with representatives from six tribes as well as held a scoping call with members of the RTOC,
who worked on grants. Additionally, interviews were conducted with representatives from other EPA
regions to gain perspective of alternate grants management processes. A complete list of interviewees is
included as Appendix E.
Prior to the interviews, the evaluator prepared interview guides, tailored to the different interviewees,
based on the overall evaluation questions. Appendix F contains an example interview guide. The
evaluator provided the specific guide to interviewees prior to the interview, which was then conducted
around these questions.
DATA ANALYSIS
As noted above, the evaluation was based on a review and analysis of existing materials and interviews
conducted with tribal and EPA personnel. The background material helped inform the design of the key
evaluation questions and the interview guides. The evaluator's analysis and findings are based primarily
on the results of the interviews, supplemented by material gleaned from the background information.
For example, in developing the programmatic process maps described in Chapter 3, the evaluators relied
both on information provided during the interviews and the accompanying background information.
-------
The evaluator first took the information provided from the interviews and organized the results around
the key evaluation questions identified earlier The evaluator looked for common themes or concerns
and also noted where ideas were raised by specific groups, such as those who worked on various grants.
Given the relatively small sample of interviewees, the evaluator also took note of issues raised by only
one or two interviewees, in addition to ideas or themes raised by several commenters. We then took
the interview results and identified "answers" to the key evaluation questions, which are organized as
the Findings in Chapter 3.
In addition, based on the interview results and related background materials, the evaluators developed
graphical representations of the steps followed by EPA in administering each of the grants programs (up
to the point where the grant application is submitted to IGMS) and of the ways that the different
programmatic grant programs are organized. The evaluator then confirmed the accuracy of these
graphics with program staff.
-------
FINDINGS
OVERVIEW
This chapter presents our findings, organized by the three overarching evaluation questions identified in
Chapter 2. To begin our analysis, we examined the steps that each program office took in administering
the grant process and found that for the most part, the programs follow similar steps. Some variation
results from the differences in underlying requirements or elements of the programs themselves. (For a
full description of each grant program, see Appendix B.) The important distinctions and issues covered
in this chapter follow not from these differences, but rather from the discrepancies in how individual
programs implement the grant administration steps. The relationship between the program officials
and the tribal representatives and the speed and effectiveness of the communication between them has
the greatest impact on the efficiency and consistency of how the grants are implemented. In addition,
we found that, to some degree, differences in the way that the regional programmatic offices are
organized can impact the relationships with the tribes and the efficiency of the grant process. While no
single organizational structure is necessarily superior or should be adopted across the board, the
differing approaches can impact both internal coordination within the Region and the interaction
between the Region and the tribes.
When analyzing the grant solicitation, review, and award process, the biggest "sticking point" occurs
around work plan negotiation. Many of these problems with workplan negotiation have a common
underlying theme: communication issues and personality clashes between EPA and tribal personnel.
Perceived lack of responsiveness to tribal concerns and delays in responding to inquiries were among
the issues raised by tribes discussing communication issues that affected the workplan negotiations. The
evaluator heard on several occasions from the tribes that the relationship with the EPA program officer,
and how that relationship affected the development and acceptance of the workplan, was the most
important factor in an efficiently run grant. To compound this, frequent turnover of staff, both at EPA
and within the tribes, can have a detrimental impact on this relationship. Finally, the need for more
frequent face-to-face communication was a common viewpoint expressed by the tribes.
With regard to internal coordination among the Region 7 tribal grant programs, some regional
respondents indicated that they thought the GAP POs or the Grants Management Office (GMO) was
serving to review grants and identify issues of concern, including possible duplication of effort or that a
grant might be more appropriately placed with a different program. The GMO does in fact review each
grant but focuses on administrative components more than programmatic issues while the GAP Pos
review the programmatic side of some grants on an informal and somewhat ad hoc basis, there is no
formal mechanism for reviewing the grants across programs.
-------
EFFICIENCY, CONSISTENCY, AND TRANSPARENCY
PROGRAM PROCESSES
To begin the analysis, process maps were developed based on interviews and document reviews. A
process map is a graphical representation of the sequence of steps necessary to complete a process or
program and serves as a tool both to review those steps and to identify areas of possible improvement.
Tribal and EPA interviewees confirmed that, for the most part, the GAP, CAA 103, CWA 106, and CERCLA
128(a) programs generally follow the same steps in administering the tribal grants. Working with
information provided during our interviews as well as material provided by EPA, we developed process
maps for each of the four programs (see Appendix F). Process maps were shared with EPA staff in each
program to ensure their reflections of the grants process were accurate. In addition to the maps we
produced for each program, we also put together a general process map that outlines the common
steps taken across the programs (See Figure 2):
Figure 2. REGION 7 TRIBAL GRANTS PROCESS MAP
Tribe is alerted either formally (e.g., issuance of NOFA) or informally (e.g., scoping discussions) of
funding availability. At this point, the regional program office may or may not know the total
available funding amount.
The Tribe submits an initial proposal, possibly with a preliminary budget and workplan or list of
proposed activities. Tribes may or may not be asked to follow a budget cap.
A decision on funding is made.
Tribe submits draft workplan to Region 7 PO. Negotiation to finalize the budget, activities, and
workplan language occurs.
Tribe submits full award application . Region 7 PO prepares funding recommendation attaching
final workplan.
award approval process follows the same process as other program aw;
-------
Broadly speaking, there are 5 steps that comprise the workplan negotiation process for tribal grants.
1. Some communication (either formal or informal) with the tribes begins the process, followed by
submission of an initial proposal from the tribe outlining what it would like to accomplish with
the grant. The tribes and regional staff may discuss programmatic needs and what might be
covered in the grant during RTOC meetings or over the phone.
2. Following interaction between Regional (and for the Brownfields tribal response program,
Headquarters staff), a decision on funding availability is made and communicated to the tribe.
3. The tribe submits a draft workplan based on the available funding, and negotiations begin. This
process can involve several iterations.
4. Once agreement is reached on the scope and specifics of the workplan, the tribe submits the full
application.
5. Once the application enters IGMS, the steps within EPA are the same across all grants and
beyond the scope of this evaluation.
While each grant program generally follows similar steps up to the point at which the application enters
IGMS, important distinctions exist among the different programs. Also, at the request of the tribes, the
GAP grants are staggered throughout the year, unlike those for the other programs, which follow a more
consistent schedule.
Initially, there are differences in the amount of detail related to funding availability that is provided to
the tribes at the beginning of the process. Prior to the beginning of the formal grants process, all the
programs engage in informal communications with the tribe to discuss tribal priorities and the possible
range of activities that might be included in the grant. For three of the programs evaluated, the formal
process begins with the issuance of a Notice of Funding Availability (NOFA). In contrast, with the CWA
106 grants, the tribes initiate the process by submitting their initial proposal.
The presence or absences of a NOFA does not directly affect the future success of the workplan
negotiation process, although some tribal representatives noted that they liked the specificity and
reference to regional priorities in the CAA 103 program's NOFA. The overarching problem that can
occur at this stage is the absence of specific information on the likely future funding amount that will be
available through the grant. This is exacerbated by the fact that the final funding decisions are often not
made until late in the grant cycle. The disconnect between initial expectations on the part of the tribes
and the final allocation, which can be much lower, can lead to concern among tribes that their priorities
are not being seriously considered throughout the process. In response to this problem, the Region 4
water program no longer solicits an initial submission from the tribes prior to knowing the grant amount
to be allocated. Representatives from Region 4 note that this change has been received positively by the
tribes and regional staff.
Another notable difference in the process occurs regarding flexibility in determining the size of the grant
that will be given to a specific tribe. Unlike the other three programs reviewed, specific tribal funding
for the Brownfields program is prescribed by the national program office, rather than being at the
discretion of the regional staff. This inconsistency between individual grants, although not a problem
that can be addressed by the Region itself, contributes to a sense of inconsistency in the overall tribal
grants program.
10
-------
PROGRAM OFFICE ORGANIZATION
In addition to the processes that each program undertakes in administering the grants, the
organizational structure of each office varies. Figure 3 on the following page shows the organizational
structures for the four program offices we reviewed.
While a formal organizational analysis to identify specific elements contributing to efficient operations
was beyond the scope of this evaluation, differences in the organization can impact interactions
internally or with the tribes. Moreover, different program responsibilities as well as budget and staffing
realities have a direct impact on the way programs organize. For example, in the CERCLA 128(a)
program, there is an identified grant coordinator who acts as an intermediary between the project
officers and the regional grants office in the grant awarding process. The grant coordinator handles
most of the administrative functions, leaving the POs to focus on the technical aspects of the grants.
This split of functions and the introduction of a layer between the POs and the GMO can possibly lead to
miscommunication within the office. For example, we learned of an instance where a PO got conflicting
advice on a workplan and budget dispute with the tribe from the grant coordinator and the GMO.
Another example is the decision of the water office to shift the position of the tribal coordinator to the
division director's office. This change was designed to give the program a higher profile and facilitate
the elevation of issues to the division director. Another change in the water program was transferring
much of the programmatic work to technical advisers (TAs) familiar with the specifics of the different
water programs and splitting the tribal PO's administrative duties from one person to three. Direct
contact between the tribes and TAs is warranted by the technical nature of many of the water programs,
but at least one of the tribes expressed some frustration that the single point of contact with the PO no
longer existed. The addition of the two other administrative POs was made in recognition of the need
for more direct communication with the tribes, the overwhelming demands of job for one person, and
efforts to improve tribal PO succession planning.
11
-------
Figure 3. CERCLA, CWA, GAP, and CAA Organization Charts
CERCLA
CWA 106
Division
Director
GAP
Tribes
Project
Officers
Programmatic grant
management. Prepares
Region 7 Tribal Program
Coordinator
Director for OTA
legion 7 Tribal
t Programs
field questions fr<
rram officers.
nts Office
CAA 103
Chief of Community Partnership Section
Chief of Air Planning and Development Branch
Director
Final approval and signature of awards.
Region 7
n. i . •', •
12
-------
Communication with the tribes; issues affecting workplan negotiation
As noted above, the most contentious stage of the grant process is the negotiation around the
workplan, and the common theme expressed is the need for more effective communication between
the region and the tribes. Primarily based on our interviews with regional and tribal staff we have
identified a number of factors that relate to this problem, as illustrated in Figure 4 and discussed on the
following pages.
FIGURE 4. Issues Cited in Interviews, as a Percentage of Responses
Frequency of Issues Cited In Interviews
40%
.-SI
oh
X Z
—
-------
Region. Tribes in particular noted that there
was too much reliance on e-mail and direct
communication was too infrequent. Some
"If you are on the third round of reviews of a work
plan or QAPP, pick up the phone or go visit the tribe
to work the issues out."
EPA staff indicated that they rely on email
i 4. 4-u 4. 4-u u -4.4. ~EPA Program Manager
alone to ensure that they have a written
record of communications. On some
occasions there was a lot of "back and forth" between tribes and the Region on work plans or other
documents, all being conducted by e-mail. More direct discussions might have addressed issues in a
more efficient and less frustrating manner. Some tribal representatives noted that conflicts (usually
attributable to specific individuals' personalities) were not rapidly or consistently addressed by the
Region, nor was the process for addressing them was well understood. Several tribal members pointed
out that it took several years to get EPA to move a PO from a program, despite numerous complaints
from the tribes.
NEED FOR MORE TRAINING
Interviewees cited the need for more training for both EPA and the tribes. For EPA staff, a need exists
for more training directed at dealing effectively with the tribes, recognizing their status as independent
Nations and the fact that many of the tribal staff do not have the experience or expertise that other
grantees might possess. Moreover, it is important for EPA staff to recognize that each tribe is different
and the need to be flexible in dealing with the
,.,, ij., ^ , , ,. ..,, "The grant process is an art, not a science and
different tribes; more so than when dealing with
there is wiggle room to make judgments. POs are
other grantees such as states. There is an online
trained in the mechanics to award a grant but
training currently available on how to work
they don t receive training on applying
effectively with the tribes, but many EPA staff ...... . ^ . . . , „
discretion...their training could be improved.
expressed a need for more classroom training,
with modules presenting specific regional GMO staff
examples, and the opportunity to interact with
people who are more experienced in dealing with the tribes. EPA staff also identified the need to assist
tribal staff in learning more about how to deal with the grants process and the specifics of different
program requirements, but recognized that the high turnover rate complicates the delivery of effective
training.
COLLABORATION AMONG REGION 7 PROGRAMS
The final question the evaluation addressed was whether there was sufficient collaboration among the
grant programs to ensure that tribal activities were conducted under the right grant programs and to
minimize any duplication of effort. During our interviews with EPA staff, some media program POs
indicated that they rely on the GAP POs to review grants and identify any potential duplication of effort
or resources being requested and to verify if an activity was being conducted under the correct grant
program. While project officers did, on some occasions, raise concerns on about a particular grant to
the GAP POs for review and comment, this occurred on an ad hoc basis and not in a systematic manner.
The GMO also reviews each grant, but focuses on administrative components more than programmatic
issues.
14
-------
The determination of whether an activity is being conducted under the right grant program is
complicated by confusion over what activities are permissible under the GAP program. This issue is not
unique to Region 7 and the EPA Headquarters American Indian Environmental Office is working on
guidance to clarify what activities are acceptable and which are unallowable under the capacity
development provisions of GAP.
15
-------
RECOMMENDATIONS
OVERVIEW
Based on the key issues that the Region wanted addressed in the evaluation and an analysis of the
findings, we have identified the following set of recommendations for EPA to consider, and which we
discuss in detail below:
• Implement grant programs as consistently as practicable across all grant programs, using the
Best Management Practices (BMPs) developed by the Tribal/EPA Grants and Communications
Workgroup (Addresses issues raised about inconsistencies in the amount of information provided
at the beginning of the grant cycle and the need for more consistent adherence to deadlines and
milestones);
• Identify internal organizational structures that facilitate or hinder efficient operation of the
grant program and collaboration among programs (Identifies best practices and possible
approaches that other programs might want to adopt);
• Increase face-to-face communications with the Tribes, especially with regard to workplan
negotiations;
• Better equip EPA staff to work effectively with Tribal staff, who are often new or inexperienced
(Ensures that EPA understand how Tribes differ from other grantees and how to work with
them); and
• Formalize the internal process to review grants among programs to ensure that activities are
funded under the right grants and to limit duplication of effort.
IMPLEMENT GRANT PROGRAMS AS CONSISTENTLY AS PRACTICABLE, ADHERING TO BEST
MANAGEMENT PRACTICES
As discussed above, most issues of concern arise from the way that the process steps are implemented,
not from the steps themselves. The Tribal/EPA Grants and Communications Workgroup developed a
Tribal Best Management Practices Guide (see Appendix G) that contains a comprehensive description of
appropriate actions to ensure that the grant process is managed effectively. We recommend that
programs implement the process in a manner consistent with the BMPs. The guide is designed to set
forth tested methods for effective management of tribal grants. We recommend that all programs
adopt all these practices, but the most important to address concerns identified in this evaluation are:
• When announcing funding opportunities for tribes, EPA needs to be as clear and precise as
possible regarding how much funding is projected to be available in order to help the tribes
develop an appropriate project in the workplan based on the funding that will be received.
While Region 7 may not know the exact funding amount, providing as much guidance as possible
would minimize the potential disconnects between the early submissions by the tribes and the
amount of money that is actually available in the grant.
16
-------
• To prevent long negotiation times for workplans, all POs, TAs, and tribal contacts will be
accountable to review and provide comment to the workplan as soon as possible but within no
more than 30 calendar days from receipt of draft workplan submission. If this timeline cannot
be met, it is important to communicate this as soon as possible.
• Workplans should not take more than two drafts to reach the third and final document. If
consensus is not reached, a new approach is necessary (meet face-to-face in order to create a
third draft). Moving to more direct communication instead of continuing to trade emails or
marked-up drafts is more likely to ensure that interests and concerns are accurately conveyed.
• Workplan negotiations should begin with the EPA and tribal counterparts agreeing to
negotiation milestones and discussing workplan ideas up front. The milestones should include a
face-to-face kickoff meeting including all program staff and managers, or representatives of
managers (i.e., Section Chiefs) who have activities in the workplan, as resources allow.
Identifying and keeping to well defined milestones will ensure that neither party believes the
other is delaying the process.
In addition, the evaluator recommends that the program offices share the process maps with the tribes.
In our interview with the Regional Tribal Operating Committee (RTOC), members of the group indicated
they had not seen any clear description of the Region's processes and seeing this would be useful for
them.
IDENTIFY ORGANIZATIONAL DESIGN FEATURES THAT FACILITATE OR HINDER EFFICIENT OPERATION
OF THE GRANT PROGRAM AND COLLABORATION AMONG PROGRAMS
We recommend that the Region undertake a formal or informal organizational analysis to determine
what organizational set ups are most conducive to effective grants management. In this analysis, the
Region should ask why was the current decision made and how has the program benefited or been
hindered by the organizational set-up. Among the questions that should be addressed are the following:
• Do Tribal grantees have a main point of contact, and/or do they have regular direct access to
appropriate representatives for programmatic, administrative, and technical questions?
o Are there separate technical points of contact for tribes? Does the nature of the tribal
grant program lend itself to this set-up?
• Do tribal project officers work exclusively with tribal grants, or do they also work with other
types of grantees?
• Is there a single point of contact within the media grant program that interacts with the GMO,
or does each PO contact the GMO for their projects?
• Who contacts Headquarters or Region 7 Office of Tribal Affairs when there are programmatic
questions - the project officer, or a single liaison for the program?
• What is the process for handling disputes between tribes and POs, and does it differ from
program to program? Does the organizational structure facilitate or impede the prompt
handling of personality disputes between tribal and regional personnel?
17
-------
This recommendation does not suggest that there is one best structure for all grants programs.
Differences in program requirements and the number of tribes with different grants argue against the
notion that one structure would work best for all programs. Rather, we recommend that staff get
together to discuss what works well and what structures do not work as well and see what best
practices emerge that might be shared among program offices.
As with the programmatic process maps, we recommend that the program offices share their
organizational charts with the tribes including providing contact information for the individuals involved.
WORK TO IMPROVE COMMUNICATIONS WITH THE TRIBES
We recommend that, to the extent possible, the Region work to increase direct and face-to-face
communications with the tribes. The evaluator recognizes that time and resource constraints can make
face-to-face meetings difficult or impossible. When these meetings cannot be scheduled or held, we
recommend that the Region make more effective use of webinars or other electronic tools to encourage
more direct communication.
In addition, we recommend that the Region consider collaboration software, such as SharePoint, to
allow regional and tribal staff to share information in a more timely way, track the status of documents,
and view calendars. The Region could allow differential status to different users, allowing some to edit
documents and post information while allowing others only to view documents that had been posted.
Where these tools are not available, the evaluator recommends the use of phone calls, with follow-up
emails if the substance of a meeting needs to be recorded. Over-reliance on emails often leads to
miscommunication; it is very hard to read "tone" in an email.
BETTER EQUIP EPA STAFF TO WORK EFFECTIVELY WITH TRIBAL STAFF
We recommend that the Region provide staff with additional training focusing on 1) understanding the
differences between working with the tribes and other grantees; and 2) adhering to best management
practices mentioned above. Part of this training should focus on the specific problems that arise from
dealing with relatively inexperienced tribal representatives (due in part to the high turnover among
tribal staff) and ways to assist these tribes in navigating through the grants process. We understand that
the Region is already planning additional EPIC (Environmental Protection in Indian Country) training for
POs and other tribal staff and we applaud this move.
In addition, we recommend that the Region consider ways to enhance the status of the tribal PO
position in order to attract and retain qualified people. One way to do this would be to focus more
attention on the importance of the position and include an assessment of the PO's and other staff
members' effectiveness in working with the tribes as part of each staff members' performance review.
As noted in the region's best management practices, employees do what is measured and increasing
attention to the role of the PO through performance measurement can improve performance and
elevate the status of the position. In addition, the evaluator suggests that senior management at Region
7 review the current job description and classification and discuss whether they need to be modified to
reflect the importance that the Region places on this position.
FORMALIZE THE PROCESS TO REVIEW GRANTS
Finally, we recognize that only a limited number of grants need to be reviewed to ensure that activities
are being conducted under the most appropriate program and there is no duplication of effort or
18
-------
resources among grants. As noted above, there is some internal review that occurs for these purposes
but it does not occur in a standardized or routinized manner. We recommend that the program offices
work together to identify and institute a more formal process for this review. This process should
identify criteria for identifying those grants that should be reviewed and establish policies to ensure that
those programmatic grants that needed to be reviewed, in fact are run through the process.
19
-------
APPENDICES
APPENDIX A:
FEDERALLY RECOGNIZED INDIAN TRIBES IN REGION 7
• Meskwaki Nation (Sac & Fox Tribe of the Mississippi in Iowa), Jama, Iowa
Known as the Mesquakie or Meskwaki Indian Settlement, the approximately 8,000 total acres
(BIA '05) of Sac & Fox land are located in the south central region of Tama County, Iowa. The
Mesquakie Indian Settlement lies approximately 130 miles from the Mississippi River, the state's
eastern border. As a member, of the Algonquian confederacy, the Mesquakie (commonly called
the Sac & Fox Tribe) originally lived along the eastern seaboard. Pressure from White settlers
and encroachment from other tribes caused them to migrate to an area around Lake Michigan.
After the Blackhawk wars in 1842, the Mesquakie were forcibly removed to a reservation in
Kansas. To prevent a forced resettlement to Indian Territory in Oklahoma, the tribe returned to
the Mississippi River Valley in Iowa and purchased its own land. This property, originally 84
acres, was placed in trust with the Governor of Iowa. In 1896, the Bureau of Indian Affairs
assumed jurisdiction over the tribe and now the land is held in trust by the United States
Government. On the Web: http://www.meskwaki.org/
• Iowa Tribe of Kansas and Nebraska, White Cloud, Kansas
The Iowa Tribe's Reservation straddles two states. The approximately 12,200 total acre (BIA '05)
reservation is located in northeastern Kansas in Brown and Doniplan counties and in the
southeastern corner of Nebraska in Richardson County. The original Iowa Reservation was
established by Treaty of 1836, only to be reduced by the treaties of May 17, 1854 and March 6,
1861. Tribal headquarters are located three miles east of Morton, Kansas, on State Highway 36.
On the Web: http://iowav.nativeweb.org/iowayksne.htm
• Kickapoo Tribe in Kansas, Morton, Kansas
The Kickapoo Reservation is located in Brown County, Kansas, five miles West of Morton, Kansas.
The Reservation is six miles long and five-miles wide with approximately 19,800 total acres (BIA
'05). The Kickapoo Tribe lived in Wisconsin and Illinois in the days prior to diplomatic relations
with the United States government. The Kickapoo Tribe was originally designated reservation
lands in an 1819 treaty in Missouri and was later moved to Kansas with lands recognized in a
treaty with the United States signed on March 16, 1854 and amended in 1864 to further reduce
land holdings. The Tribal Office is located six miles west of Morton on Highway 20, Yi mile North,
% mile North, % mile West at Senior Citizens Complex.
On the Web: http://www.ktik-nsn.gov/government.htm
• Prairie Band Potawatomi Nation, Mayetta, Kansas
The Potawatomi reservation consists of approximately 78,760 total acres (BIA '05) in roughly an
11-mile square. The Potawatomi are originally from the Great Lakes area of present day
Michigan and Wisconsin. The Prairie Band were forcibly relocated to present day Kansas by the
Indian Removal Act of 1830. The tribal headquarters are located seven miles west and one mile
south of Mayetta. On the Web: http://www.pbpindiantribe.com/
20
-------
• Sac & Fox Tribe of Missouri in Kansas & Nebraska, Reserve, Kansas
The Sac & Fox reservation straddles two states. The Sac & Fox reservation lands consist of
approximately 14,500 total acres (BIA '05) within Brown County, Kansas and Richardson County,
Nebraska. Their reservation borders the Iowa reservation on the west. The original homeland
of the Sac & fox was in the Great Lakes region, where the Sacs in habited the upper Michigan
peninsula and the Foxes, the south shore of Lake Superior. The tribal headquarters is located in
Reserve, Kansas.
On the Web: http://www.mnisose.org/profiles/sacfox.htm
• Omaha Tribe of Nebraska, Macy, Nebraska
The Omaha Tribal homelands are located in the northeast corner of Nebraska, overlapping into
a small portion of western Iowa. The area is comprised of the Omaha Tribal Reservation and
adjacent counties with approximately 198,550 total acres (BIA '05). The Nebraska counties are:
Thurston, Burt, Cuming, Wayne, and the Iowa county Monona. The Omaha Tribe lived near the
Missouri River in present day Nebraska in the days prior to diplomatic relations with the United
States government. The Omaha Tribe was originally designated reservation lands along the
Missouri River recognized in a treaty with the United States signed on March 16, 1854. Macy,
Nebraska is the location of the headquarters for the Omaha Tribe of Nebraska. On the Web:
http://www.omahatribeofnebraska.com/
• Ponca Tribe of Nebraska, Niobrara, Nebraska
The Ponca Tribal homelands are located in portions of three noncontiguous counties located in
the eastern third of the state of Nebraska. The counties are Knox and Madison, situated in the
northeastern section of the state, Douglas and Lancaster, located in southeastern Nebraska and
Charles Mix in south central South Dakota. The service areas cover approximately 1,800 square
miles. The Tribal Headquarters is located in Niobrara, Nebraska. There are four field offices
located within the service area in Lincoln, Norfolk, Omaha, and Sioux City, Nebraska. The Ponca
Tribe lived near the Missouri River in present day Nebraska in the days prior to diplomatic
relations with the United States government. The Ponca Tribe signed several treaties in 1817,
1825, and 1858 and was originally designated reservation lands along the Missouri River
recognized in a treaty with the United States signed in 1865. On the Web:
http://www.poncatribe-ne.org/
• Santee Sioux Nation, Santee, Nebraska
The Santee Sioux Nation in Nebraska reservation is within Knox County located in the north
central part of Nebraska. The service area includes the entire county, totaling 1,105 square
miles with approximately 117,461 total reservation acres (BIA '05). The Santee Sioux Tribe
consists of the members of the Isanti and lhanktowan divisions of the Great Sioux Nation. The
Tribe was relocated to the reservation after Little Crow's War in Minnesota. It was originally
designated reservation lands along the Missouri River recognized in a treaty with the United
States signed in 1863. The Santee Sioux Nation was further defined and the boundaries
expanded by the act of March 2, 1889. The Santee Sioux Nation tribal headquarters are located
in Santee, Nebraska.
On the Web: http://www.santeedakota.org/
• Winnebago Tribe of Nebraska, Winnebago, Nebraska
21
-------
The Winnebago Reservation is located in the northern half of Thurston County in the
northeastern corner of Nebraska, plus 1,800 acres in Woodbury County, IA; 26 miles southeast
of Sioux City, Iowa and seventy miles north of Omaha, Nebraska on state highways 75 and 77.
The Winnebago reservation has a total area of approximately 113,038 acres (BIA '05). The
Winnebago Tribe lived near the Missouri River in present day Nebraska in the days prior to
diplomatic relations with the United States government. The Winnebago Tribe was originally
designated reservation lands along the Missouri River recognized in treaties with the United
States signed on March 8, 1865 and June 22, 1874. On the Web:
http://www.winnebagotribe.com/
22
-------
APPENDIX B:
BRIEF DESCRIPTIONS OF GRANTS ADDRESSED IN REPORT
CAA 103: Section 103 of the Clean Air Act permits EPA to provide funding to tribes for the purpose of
building tribal knowledge and increasing their capacity to manage air quality issues on reservations and
tribal trust land. Air quality management activities eligible for funding under this grant include: setting
goals and standards to protect public health and the environment; assessing air quality through
emissions inventories and monitoring; and determining necessary reductions in pollution.3
CWA 106: EPA is authorized under Section 106 of the Clean Water Act to award federal grants to assist
tribes in administering water quality programs for the prevention, reduction, and elimination of water
pollution. Eligible water pollution control activities include: water quality planning, assessments, and
studies; ambient monitoring; community outreach and education activities; and data management and
reporting.4
CERCLA 128(a): Under Section 128(a) of the Comprehensive Environmental Response, Compensation,
and Liability Act (CERCLA), EPA is able to provide funding to tribes for the establishment of response
programs and enhancement of their capacity. These response programs typically address the
assessment, cleanup, and redevelopment of brownfields sites as well as other contaminated sites. A few
of the activities eligible for funding are the development of legislation, regulations, procedures,
ordinances, guidance, etc. that would establish or enhance the administrative and legal structure of the
tribal response program.5
GAP: The Indian Environmental General Assistance Program Act allows EPA to provide General
Assistance Program (GAP) grants to federally recognized tribes and tribal consortia for planning,
developing, and establishing environmental protection programs in Indian Country. In addition, GAP
funding supports the development and implementation of tribal solid and hazardous waste programs.
The program aims to assist tribes in managing and developing the capacity of their environmental
protection programs, and to develop and implement solid and hazardous waste programs that are
consistent with federal laws and regulations and which address tribal needs.6
3 "The Tribal Air Grants Framework: A Menu of Options for Developing Tribal Air Grant Work Plans and Managing
Grants for Environmental Results," U.S. EPA, revised October 2007.
4 "Final Guidance on Awards of Grants to Indian Tribes under Section 106 of the Clean Water Act," U.S. EPA.
5 "Funding Guidance for State and Tribal Response Programs Fiscal Year 2010," U.S. EPA, OBLR, November 2009.
6 "Indian General Assistance Program 2006 Grant Administration Guidance - Effective February 24, 2006," U.S.
EPA.
23
-------
APPENDIX C:
METHODOLOGY FOR EPA REGION 7 TRIBAL GRANTS EVALUATION
INTRODUCTION
The U.S. Environmental Protection Agency (EPA) Region 7 awards grants to Tribal nations across its
various media programs. Tribes use these grants as one source of funding to develop the skills and
capacity to develop and manage environmental programs, as well as to help implement them. Technical
and administrative project officers in each Region 7 media program manage Tribal grants and work with
the regional Grants Office to ensure that grant administration requirements are met and that results are
achieved.
Region 7's cross-cutting grants approach helps Tribes develop and implement multi-media
environmental programs. However, Tribes have expressed concerns about the programmatic grants
processes, noting that requirements may vary among the different media programs and may be
conflicting or inefficient. In addition, some Tribes believe that their priorities have not been sufficiently
recognized and addressed through programmatic grant processes. For example, in a Tribal
communications survey that EPA Region 7 conducted in 2006, over 40% of Tribal environmental staff
indicated that they felt there was a communication gap with Region 7 representatives on programmatic
and administrative matters.
In order to better understand these concerns and to ensure that the programmatic grant processes are
as clear and efficient as possible, the Region 7 Tribal program and the Regional Tribal Operations
Committee (RTOC) have formed a subgroup charged with reviewing and improving communications
around the Tribal grants, specifically focusing on improving transparency to improve grant
implementation. The group developed maps of existing Tribal grant processes for each of the different
media programs and drafted a list of best practices for managing grants for both the Agency and Tribes.
To support this review, this evaluation has been funded as part of EPA's annual Program Evaluation
Competition. EPA has contracted with Industrial Economics, Inc. (lEc) to conduct a process evaluation of
the grant program, to assess how grants are currently handled, and to make suggestions so grants can
be addressed consistently, expeditiously, and effectively throughout all media programs.
An important component of the evaluation will be an assessment of the process maps developed for
each grant program. lEc will first determine if the process maps accurately describe the way that the
grants are issued and managed, in practice. If processes are followed, we will evaluate their
effectiveness. If actual practices vary from the steps laid out in the process maps, we will examine the
differences, develop revised process maps, and make suggestions as to whether the practices need to
be changed, or whether they are still effectively meeting the program goals. Additionally, we will seek
stakeholder input and benchmark against other EPA Regions.
lEc will conduct a process evaluation only, focusing on the Agency's Tribal grant allocation, approval, and
management practices, rather than on grant products or activities conducted under the grants by Tribes.
As an independent contractor, lEc will coordinate with EPA but will conduct its own analysis and make
its own recommendations.
This evaluation methodology describes the purpose for the evaluation, identifies the evaluation
questions and sources of data to answer the questions, and presents an analytic approach for
24
-------
interpreting the data. We also describe how the evaluation results will be reported and include a
project schedule.
THE PROGRAM LOGIC MODEL
As an essential first step in laying out the structure of the evaluation, we worked with EPA to develop a
program logic model that outlines the basic structure and design of the Tribal grant process, as shown in
Exhibit 1. This model shows the steps involved in awarding and managing the grants, as well as
outcomes due to specific activities that the Tribes undertook with the grants. Grant activity outcomes
are shown in the logic model for the sake of completeness, but will not be addressed by lEc during its
evaluation. The process portion of the logic model, which is supported by the series of program-specific
process maps that are described above, serves as the basis for the evaluation questions (see next
section) and will inform lEc's data collection efforts.
Key components of the logic model include:
• Resources: Basic inputs of funds, staffing, and knowledge dedicated to the program.
• Activities: Specific actions taken to achieve program goals. For example, EPA issues grants and
develops grant guidance and performance criteria.
• Outputs: The immediate products that result from the activities, such as grant applications,
guidance, and negotiations.
• Process Outcomes: Include short- and long-term process outcomes, such as intended changes in
awareness, attitudes, understanding, knowledge, and skills resulting from program outputs. For
example, the grant guidance is designed to help inform Tribal members about the procedures and
rules governing the grant program. Long-term outcomes parallel the overarching goals of the
program and include the provision of clear and understandable rules and guidance, and a Tribal
community satisfied with the process.
• Grant Activity Outcomes: Include short- and long-term outcomes due to specific activities that the
Tribes undertook with the grants (e.g., increased Tribal capacity to administer environmental
programs or improved environmental conditions).
25
-------
Exhibit 1. Logic Model for EPA R7 Tribal Grant Processes7
Resources/
Inputs
• Funding for
.
grant offices
• EPA Region
7 technical
and
administrative
personnel:
- Project
officers
- Grant
administrators
• Regional
Quality
• EPA IT
systems:
• Integrated
Management
- Financial
Data
Warehouse
Activities
* Grant application
and support
• Grant application
negotiation, review, and
decision {incl. workplan
and budget):
- Threshold criteria
and programmatic
compliance
and additional approvals
(if necessary)
• Grant management.
support
• Quality assurance
review
Outputs
Grants.gov, or in Federal Register:
- Notices of funding availability
- Grant application templates
• Inputs to IGMS
• Notifications of application
receipt
• Application feedback and
technical advice
• Negotiation milestones
• Commitment notices
• Grant approvals and signatures
• Notification and mailing of grant
awards to Tribes, or notification of
• Updates to IGMS Post-Award
Database and grant files
• Feedback on quarterly progress
and final performance reports
(incl. requests for revisions, if
necessary)
« Phone calls and in-person visits to
Tribes for technical support
• QAPP approval (or requests for
revisions, if necessary)
Process
Outcomes
• Fair, simple,
consistent, time-
efficient, and
transparent grants
processes across
Tribes and
program areas
• Effective
communication
and coordination
between EPA and
Tribes
• Valuation of
Tribal priorities
and environmental
needs
• EPA and Tribal
satisfaction
Grant Activity
Outcomes
• Efficient and
Tribes
• Increased Tribal
capacity to
administer
programs and
tackle
challenges
to monitor and
improve
environmental
conditions
• Enhanced
environment and
human health
7 Note: Grant activity outcomes are shown in the logic model for the sake of completeness, but will not be addressed by lEc during its evaluation. lEc will
conduct a process evaluation only, focusing on the Agency's Tribal grant allocation, approval, and management practices, rather than on grant products or
activities conducted under the grants by Tribes.
26
-------
EVALUATION QUESTIONS AND SUB-QUESTIONS
EPA presented lEc with three interrelated draft evaluation questions in the contract work assignment.
Following discussions with EPA and the development of a logic model, lEc revised the evaluation
questions and developed the following three major questions:
1. How can Region 7 make its Tribal grant processes more efficient, consistent, and
transparent across Regional program areas as well as Tribes?
2. How can Region 7 improve its communication and coordination with Tribes throughout the
entire grants process, from negotiation of grant applications to the management of
activities conducted under the grants?
3. Are there ways for the different Region 7 programs to coordinate and collaborate for
improved (e.g., consistent) grants management and Tribal relations, while still effectively
utilizing their specific programmatic expertise?
In addition, we have identified a number of more targeted sub-questions that fall under each of the
three key evaluation questions. These sub-questions will help form the basis for the interview guides
that we develop, as well as help direct our other research. For the interview guides, which will be
tailored to the interviewee, we will modify the sub-questions into interview questions that encourage
participants to provide exampled and detailed responses (i.e., more than just "yes/no" responses).
Table 1 below lists the documents we plan to review and analyze and the individuals we will interview to
help answer each of the evaluation questions and sub-questions. The overarching evaluation questions
are highlighted in bold.
TABLE 1.
QUESTIONS AND RESEARCH METHODS
QUESTION
REVIEW AND ANALYSIS OF MATERIALS
INTERVIEWS
How can Region 7 make its Tribal grant
processes more efficient, consistent, and
transparent across Regional program areas as
well as Tribes?
- Region 7 Tribal grants process maps
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- Region 7 Tribal communications feedback
evaluation
- National program guidance from each
media program
- GAP guidance
- Other EPA Regional guidance materials
- Grant NOFAs/RFPs
- Region 7
- EPA HQ
- Other EPA
Regions
- Tribes
How are the Region 7 programmatic grant
processes currently structured? What are the
overarching grant requirements for all the
programs (e.g., number of days to close out a
grant, etc.)?
- Region 7 Tribal grants process maps
- Region 7 Tribal assistance protocols and
roles/responsibilities
- Region 7
What causes the grants negotiation and
performance reporting processes to extend longer
than they should? At which stages do the
processes get held up? Are clear milestones and
schedules set from the start?
- Region 7 Tribal grants best management
practices
- Region 7 Tribal communications feedback
evaluation
Region 7
Tribes
27
-------
QUESTION
REVIEW AND ANALYSIS OF MATERIALS
INTERVIEWS
What are the differences in work plan and
quarterly report requirements for the different
types of Tribal grants (i.e., GAP grants and media-
specific grants)?
- Region 7 Tribal grants process maps
- Region 7 Tribal assistance protocols and
roles/responsibilities
- National program guidance from each
media program
- GAP guidance
- Other EPA Regional guidance materials
- Region 7
- EPA HQ
- Other EPA
Regions
- Tribes
How are the Tribes made aware of the different
work plan and quarterly report requirements for
the different types of Tribal grants? Is this
communication effective?
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- Region 7 Tribal communications feedback
evaluation
- Grant NOFAs/RFPs
- Region 7
- Tribes
Are the same programmatic grants processes
applied to each Tribe that receives funding?
- Region 7
Are consistent programmatic grants processes
applied each year/funding cycle? If there are
differences, why?
How can Region 7 improve its communication
and coordination with Tribes throughout the
entire grants process, from negotiation of grant
applications to the management of activities
conducted under the grants?
- Region 7 Tribal grants process maps
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
roles/responsibilities
- Region 7 Tribal communications feedback
evaluation
- Grant NOFAs/RFPs
- Region 7
- Region 7
- Tribes
How are Tribal priorities and environmental needs
incorporated into grant processes?
- Region 7 Tribal grants process maps
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- Region 7 Tribal communications feedback
evaluation
Region 7
Tribes
Do Region 7 project officers and Tribes
communicate on a consistent basis, or only when
issues arise?
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
roles/responsibilities
- Region 7 Tribal communications feedback
evaluation
Is there ever confusion about roles and
responsibilities?
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
roles/responsibilities
- Region 7 Tribal communications feedback
evaluation
Region 7
Tribes
Region 7
Tribes
28
-------
QUESTION
REVIEW AND ANALYSIS OF MATERIALS
INTERVIEWS
Are there ways for the different Region 7
programs to coordinate and collaborate for
improved (e.g., consistent) grants management
and Tribal relations, while still effectively
utilizing their specific programmatic expertise?
- Region 7 Tribal grants process maps
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- National program guidance from each
media program
- GAP guidance
- Other EPA Regional guidance materials
- Region 7
- EPA HQ
- Other EPA
Regions
Which grant steps are mandated by EPA
Headquarters or statutes/regulations? Are there
different grants steps in the other EPA Regions?
Are there any steps than could be removed or
altered without having an adverse impact on the
programmatic grant processes? At what level
would these changes need to be made (i.e.
headquarters or R7)?
- Region 7 Tribal grants process maps
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- National program guidance from each
media program
- GAP guidance
- Other EPA Regional guidance materials
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- Region 7
- EPA HQ
- Other EPA
Regions
Are Region 7 project officers aware of differing
grant processes for the different programs?
How does Region 7 coordinate across programs?
When a program issues a grant to a particular
Tribe, does it always know which other grants the
Tribe may be receiving, and does it coordinate?
What kind of coordination is possible? Is this
coordination happening? Can EPA's IT systems
(e.g., IGMS, Financial Data Warehouse) be used to
help?
- Region 7 Tribal grants best management
practices
- Region 7 Tribal assistance protocols and
roles/responsibilities
- Region 7
- Region 7
Are there programmatic or environmental
justifications for having some differences in the
requirements for the different types of Tribal
grants?
- Region 7 Tribal grants process maps
- Region 7 Tribal assistance protocols and
ro les / res po nsi bi li ties
- National program guidance from each
media program
- GAP guidance
- Other EPA Regional guidance materials
- Region 7
- EPA HQ
- Other EPA
Regions
We discuss our data collection approach in more detail in the next section of this document.
DATA COLLECTION METHODOLOGY
lEc plans to use two data collection approaches: (1) a review and analysis of existing materials from EPA,
including the draft Tribal grants process maps and Tribal grants best management practices; and (2)
interviews and surveys with key technical and administrative personnel (e.g., project officers and grant
administrators) in Region 7, EPA Headquarters, and other EPA regions, as well as with Tribes in Region 7.
Review of Existing Materials
lEc will review materials from Region 7, including but not limited to program process maps, grant
documents, Tribal assistance protocols and roles/responsibilities, the Tribal grants best management
practices, and the Tribal communications feedback evaluation. Region 7 will provide lEc with these
materials, if they are not publically available online. These materials will be especially useful in helping
29
-------
lEc comprehend how Region 7's Tribal grant processes vary between programs and how they are
intended to function. lEc understands that the process maps and Tribal grants best management
practices are currently in draft form and will recommend any necessary modifications at the conclusion
of the evaluation. lEc also expects to review materials from the EPA Headquarters' Grants Office as well
as from other EPA regions, such as National Program Guidance documents, GAP Guidance documents,
and Regional guidance documents. These materials will help lEc benchmark Region 7's Tribal grants
processes against the processes used elsewhere within EPA and identify possible lessons or best
practices that can be transferred to Region 7 or from Region 7 to other Regions. If there are time or
resource constraints, lEc will focus primarily on benchmarking for water grant programs, as water issues
are the most significant environmental concern of the Tribes in Region 7.
Interviews and Surveys
lEc will rely heavily on direct feedback provided through interviews with key technical and
administrative personnel in Region 7 as well as interviews with Tribes in Region 7, EPA Headquarters,
and other EPA regions. lEc expects to conduct most of the interviews by phone and will provide
interviewees with the questions in advance of the interviews. Each interview should take between one
and two hours, and will be tailored to the interviewee. lEc may conduct follow-up interviews, if
necessary, and will also encourage additional feedback from interviewees after they are interviewed. If
we cannot reach certain individuals due to budgetary, scheduling, or time constraints, we will consider
the use of a short survey document, recognizing that this format will limit our ability to ask open-ended
questions.
Table 2 below shows the anticipated number of interviews within each category of interviewee. Group
interviews count as one interview for the purposes of this table.
TABLE 2.
ANTICIPATED INTERVIEWS FOR EACH INTERVIEWEE CATEGORY
INTERVIEWEE CATEGORY
Region 7
Tribes
EPA HQ (OITA, Program Offices, and OGD)
Other EPA Regions
Total
APPROXIMATE NUMBER OF INTERVIEWS
10-15
3-5
3-4
5-8
21-32
lEc plans to interview key technical and administrative personnel in Region 7, EPA Headquarters, and
other EPA regions. Region 7 interviewees will include staff and management in the Water, Air,
Superfund, Tribal, and Grants offices, as well as Counsel. In particular, lEc plans to interview at least a
random representative sample of Region 7's:
Regional Management (incl. program branch chiefs and other senior staff)
- Administrative Project Officers
- Technical Project Officers
- Technical Advisors
30
-------
Grants Specialists
Grant Management Officers
These staff and management will provide an insider's perspective on Region 7's Tribal grant processes,
allowing lEc to determine what the existing processes are (as compared to the process maps) and to
gain insight into how the processes can be made more efficient and consistent.
lEc also plans to interview representatives from OITA, National Program Offices (e.g., AIEO, OW, OAR,
Superfund, Brownfields, OPP, etc.) and OGD, as well as Regional personnel who have significant
experience in Tribal grant processes. In particular, lEc plans to interview staff from EPA Regions 1, 2 and
4, which have a similar number of federally recognized Tribes as EPA Region 7. By interviewing EPA
employees outside of Region 7, lEc will be able to benchmark against Region 7's processes and
incorporate knowledge and experience gained in other parts of the Agency. lEc will work closely with
the EPA COR and other members of the evaluation team to identify the appropriate personnel to
interview from these Regions.
There are nine federally recognized Tribes in Region 7, the majority of which are in receipt of active EPA
grants (see Table 3 below -Tribes that do not currently receive any EPA grants are marked with an
asterisk and will not be interviewed). lEc intends to interview a representative sample of individuals
from the Region 7 Tribes who are familiar with Region 7's Tribal grant processes. lEc will first interview
representatives from the Tribes that serve on the Regional Tribal Operations Committee (RTOC) Grants
Subgroup, and, in the event that these do not cover all types of grants, representatives from other
Tribes that receive different types of grants. Through its interviews of Tribes, I EC will be able to verify
tribal concerns firsthand and receive suggestions from the perspective of the grant recipient.
TABLE 3.
FEDERALLY RECOGNIZED TRIBES IN REGION 7
TRIBE
Meskwaki Nation (Sac & Fox Tribe of the Mississippi in Iowa)
Iowa Tribe of Kansas and Nebraska*
Kickapoo Tribe in Kansas
Prairie Band Potawatomi Nation
Sac & Fox Tribe of Missouri in Kansas and Nebraska
Omaha Tribe of Nebraska and Iowa*
Ponca Tribe of Nebraska
Santee Sioux Nation
Winnebago Tribe of Nebraska
TENTATIVE PROJECT SCHEDULE
A tentative project schedule is shown in Table 4 below.
31
-------
TABLE 4.
TENTATIVE SCHEDULE FOR EPA REGION 7 TRIBAL GRANTS EVALUATION
TASK
Participate in conference calls with EPA
Attend RTOC meeting in Lawrence, KS
Provide updates to RTOC
Review background documents
Develop draft logic model
Develop draft evaluation plan, incl. evaluation questions and
methodology
Revise evaluation plan
Interview Guides
Conduct interviews with EPA Regions, HQ, and with Tribes
Initial Findings
Draft final written report
Revise final written report
Provide oral presentation to EPA HQ and R7
APPROXIMATE TIME FRAME
Weekly throughout the evaluation
Complete
Quarterly throughout the evaluation
Complete; will re-review as necessary throughout the
evaluation
Complete
Complete
Complete
By March 2011
March-May 2011
Ongoing, but first cut by end of May 201 1
Early summer 2011
Late summer 201 1
Late summer 201 1
PRESENTATION OF RESULTS
During the course of the evaluation, we will provide status updates to EPA, generally through telephone
conferences. We can also provide updates to the Tribes in Region 7 during the quarterly RTOC
meetings, either over the phone or in person. In addition, we will present our final findings and
recommendations in a written report, consolidated fact sheet, and oral presentation to EPA. We will
use the logic model and evaluation questions to frame our presentation, and the findings and
recommendations will be directly organized around the evaluation questions. Region 7 will share the
final evaluation results and recommendations with the RTOC, and will determine possible steps for
implementation.
After developing the initial findings and survey recommendations, the evaluators will draft the full
evaluation report for EPA review. The evaluation report is expected to be structured as follows
(however this may change somewhat depending on the evaluation findings):
Executive Summary
I. Introduction (approx. 1 page)
A. Purpose/objectives of the evaluation
B. Evaluation questions
C. Structure of the report
32
-------
II. Methodology and Approach (approx. 1-2 pages)
A. Study design
B. Data sources
C. Analytical approach
III. Results (approx. 3-5 pages)
A. Summary response to each research question (combining information drawn from all data
sources)
IV. Findings and Conclusions (approx. 3-5 pages, in addition to graphics)
A. Description of the current programmatic grant processes
B. Strengths and weaknesses of current processes
C. Proposed changes to processes/"ideal future state"
D. Recommendations/path to implement proposed changes and reach "ideal future state"
33
-------
APPENDIX D:
LITERATURE REVIEWED
U.S. Environmental Protection Agency. Final guidance on awards of grants to Indian tribes under section
106 of the Clean Water Act for FYS 2007 and beyond.
U.S. Environmental Protection Agency. Indian General Assistance Program 2006 grant administration
guidance (Effective February 24, 2006)
U.S. Environmental Protection Agency Region 7 Tribal Group. Draft recommendations. October 2006.
U. S. Environmental Protection Agency, Office of Air and Radiation. Review of authorities available for
Tribal program financial assistance awards. November 2006
U.S. Environmental Protection Agency. The Tribal air grants framework: a menu of options. October
2007.
U.S. Environmental Protection Agency, Region 7. Draft region 7 protocol for tribal assistance
agreements. September 2006.
U.S. Environmental Protection Agency, Region 7. Region 7 tribal study-final report. December 2010
U.S. Environmental Protection Agency Region 7. Tribal grants process maps as provided by program
offices. August 2010.
U.S. Environmental Protection Agency, Region 7. Tribal Air Non-competitive grant opportunity FY-2011
the Clean Air Act 103 Tribal notice of funds available.
U.S. Environmental Protection Agency Region 7 Tribal/EPA Grants and Communications Workgroup.
Tribal/EPA best management practices guide for grants and communication. February 2011.
U.S. Environmental Protection Agency Region 7 Tribal/EPA Grants and Communications Workgroup.
Meeting notes follow up. July 2010
34
-------
APPENDIX E:
INDIVIDUALS INTERVIEWED FOR EVALUATION
Karen Sherrill; Grant Management Section Chief; EPA Region 7
Jacob Nicholls; Tribal Grants Specialist; Grants Management Section; EPA Region 7
Wolfgang Brander; Senior Regional Indian Program Manager; EPA Region 7
Heather Hamilton; GAP Project Officer, Office of Tribal Affairs; EPA Region 7
Stanley Holder; GAP Project Officer, Office of Tribal Affairs; EPA Region 7
Amanda Halstead; Sustainable Communities Partnership Coordinator; Office of Policy and Management,
EPA Region 7
Steve Kovac; Branch Chief for Watershed Planning and Implementation; Water, Wetlands and Pesticides
Division; EPA Region 7
Heather Duncan; CWA 106 Coordinator; Water, Wetlands and Pesticides Division; EPA Region 7
Amy Shields; Tribal Water Quality Standards Program Coordinator; Water, Wetlands and Pesticides
Division; EPA Region 7
Amy Algoe-Eakin; Section Chief for Community Partnerships; Air Planning and Development Branch; EPA
Region 7
Kim Olson; Project Officer; Air Planning and Development Branch; EPA Region 7
Stanley Walker; Branch Chief for Superfund Technical Assistance and Reuse; EPA Region 7
Ann Keener; Grant Coordinator; Superfund Division; EPA Region 7
Deborah Kennedy, Project Officer, Superfund Division, EPA Region 7
Jennifer Morris, Project Officer; Superfund Division; EPA Region 7
Dan Cozza; Water Quality Coordinator; Water Pollution Control Grant Program; EPA Region 5
David Horak; Water Quality Coordinator; Water Pollution Control Grant Program; EPA Region 5
Betty Winter; Tribal Project Officer; Office of Policy and Management; EPA Region 4
Kickapoo Tribe in Kansas: Luke Terry, Tej Attili, Mike Kelley
Sac & Fox Tribe of Missouri in Kansas & Nebraska: Rick Campbell, Teresa Ausberger, Scott Weir, Mathue
Fasching
Prairie Brand Potawatomie Nation: Virginia LeClere, Sharron Bosse, Adam Irvin
Winnebago Tribe of Nebraska: Joseph Painter, Denise Jensen, Molly Feldick
Santee Sioux Tribe: Felix Kitto, Eric Miller, Joel Valburg, Jeramie Key, Anthony (Sam) Kitto
Ponca Tribe of Nebraska: Julia Sage
35
-------
APPENDIX F:
SAMPLE INTERVIEW GUIDE
EPA Region 7 Tribal Grants Evaluation
Interview Guide for EPA Region 7 CWA 106 Staff
CONTACT INFORMATION
Name:
Title:
Email:
Phone number:
INTRODUCTION
Thank you for participating in this interview process. As you are aware, EPA Region 7 awards grants to
Tribal nations across its various media programs. As you may know, in order to ensure that the process
is as clear and efficient as possible, the Region 7 Tribal program and the Regional Tribal Operations
Committee (RTOC) have formed a subgroup charged with reviewing and improving communications
around the Tribal grants, specifically focusing on improving transparency in grant implementation.
Parallel to this effort, EPA has contracted with Industrial Economics, Inc. (lEc) to conduct a third-party
evaluation of the grant program. Through this evaluation, EPA is seeking to understand how the Tribal
grants process currently works and how it could be improved.
This will be an evaluation of Region 7's processes only, focusing on the Agency's Tribal grant allocation,
approval, and management practices. EPA Region 7 intends to use the results of the evaluation to
improve its internal processes and will share the results with the RTOC.
Throughout the interview, we would like to know about the processes with which you are most familiar.
We also encourage you to raise any items or topics you think are important to our evaluation. Notes
from specific interviews will be treated as confidential information, and information collected through
the interviews will be summarized thematically and will not be attributed to specific individuals in the
evaluation report. In presenting findings from the interviews, lEc may attribute findings to groups of
interviewees, (e.g., a regional staffer), but we will not attribute findings or quotes to individuals.
The following interview questions are intended to serve as a guide for our conversation and are
provided in advance to spur your thinking and responses. Where possible, please be prepared to
provide specific examples. If you prefer, you may provide comments in written form, although we
would like to have a brief follow-up call with you if any clarifications are needed. Your responses are
important, and we look forward to speaking with you. If you have any questions or would like to
provide any additional feedback or information, please contact:
Andy Schwarz, Industrial Economics, Inc.
ams@indecon.com
617-354-0074
36
-------
INTERVIEW QUESTIONS
Grant Process Steps and Roles
1. Please briefly describe your position at Region 7 and your role as it relates to Tribes and Tribal
grants. Which Tribal grants do you manage?
2. Please discuss the steps you typically go through when working on a Tribal grant for the CWA
106 program. To assist in your response, please keep in mind the following typical grant stages
and activities:
• Proposal Solicitation and Eligibility
• Negotiating Work Plans and Budgets
• Awarding Funds
• Reviewing Performance Reports / Post-Award Monitoring
• Closing Out Grants
3. What do you see are the successes in your Tribal grant process? What's working well? What
are the key points in the process that have the biggest influence on achieving a positive
outcome?
4. What are opportunities for improvement? What are your biggest obstacles to success? Where
in the process do problems occur that can prevent a positive outcome?
5. When major issues arise, who do you elevate them to or ask for their assistance with?
• How do you communicate with your Division Director?
• How do you communicate with the Grants Office?
• How do you communicate with the Office of Tribal Affairs in Region 7? What issues do
you elevate to them or ask for their assistance with?
• Who is responsible for the ultimate resolution of the issue (i.e., staff or management)?
Efficiency and Consistency of Programmatic Grant Processes
6. What milestones are developed to keep the CWA 106 grants process on schedule?
• To the extent that grant processes (e.g., work plan negotiation, awarding funding,
review of performance reports, closing out, etc.) extend longer than anticipated, what
do you see as the primary causes for these delays?
• What system, if any, is in place to track the timing of the process?
7. We understand that the CWA 106 program is currently under transition at Region 7. Has this
had any effect on grant processes?
37
-------
8. What type of guidance informs your Tribal grant process (e.g., statutes, Region 7 policy, EPA
Headquarters guidance, etc.)? Are there parts of the process that are at the Region's
discretion?
• In your opinion, how could you do things differently to meet the requirements of the
process?
9. Are you aware of differing CWA 106 grant processes or grant resources in the other EPA
regions? Do you talk with your colleagues in other regions about the grants process? Could
Region 7 incorporate any elements from the other EPA regions to improve its CWA 106 grant
process?
Communication and Coordination with Tribes
10. Please describe how you communicate with the Tribes during the CWA 106 grant process.
• When and over what issues do you communicate (e.g., work plan and budget
negotiation, performance reports, etc.)? Who is responsible for ensuring that
communication takes place?
• What methods do you use to communicate (e.g., email, conference call, in-person
meetings/site visits, etc.)?
• Do you communicate in a similar manner and frequency with each Tribe? Do most
Tribes prefer the same method of communication? Do you prefer a particular method
of communication?
• Are there areas where the communication could be improved? If so, please explain.
Coordination and Collaboration Across Region 7
11. How does Region 7 coordinate across programs?
• Do you know which other grants the Tribe may be receiving, and do you coordinate?
• What type of coordination is possible for Tribes receiving other grants (e.g., GAP,
Wetland Program Development, or CWA 319 Grants)?
• Do you feel coordination across programs is done efficiently and effectively? What
opportunities do you see for increased coordination, if necessary?
• What tools, such as EPA's IT systems (e.g., IGMS, Financial Data Warehouse), could be
used to help in coordinating grant efforts?
12. How would you rate the level of communication that exists between grants managers and staff
across Region 7? How could this communication be improved? What currently works well?
38
-------
Additional Information
13. Please discuss any other issues affecting the efficiency or effectiveness of the CWA 106 grants
process. Do you have any other suggestions to improve the process, from either the
perspective of EPA or the Tribes?
39
-------
APPENDIX G:
R7 PROCESS MAPS
DRAFT PROGRAMMATIC PRE-AWARD PROCESS MAP OF CAA 103 TRIBAL GRANTS
EPA Region 7 issues NOFA using tribal air guidance. Region 7's budget guidance to tribes depends
on yearly funding availability from EPA Headquarters.
(February)
The Region 7 PO evaluates proposals and discusses allocations with immediate supervisors and
the Environmental Services Division prior to meeting the Region 7 director of the Air and Waste
Management Division. Region 7 distributes funds based on work plan activity and prior grantee
performance.
Memorandum with final decision is signed by the director of AWMD.
After Region 7 has decided to fund the project, work plan negotiations begin. This is often an
iterative process between PO and tribe (see below).
(May-July)
Tribe submits final, complete grant application package includingfinal work plan to Region 7
grant office where it is entered into IGMS. (application due mid to end of July)
Region 7 PO prepares funding recommendation in IGMS, attaching final work plan.
(award to be approved in August)
40
-------
DRAFT PROGRAMMATIC PRE-AWARD PROCESS MAP OF CERCLA 128(a) TRIBAL GRANTS
EPA Headquarters issues annual funding guick
(November - December)
Tribe provides Region 7 PO with request for funding
(by January 31)
Region 7 engages in internal discussion of requests for funding and sends a consolidated
response to EPA HQ
EPA HQ conference call with Regions to discuss allocations
Region 7 PO learns of allocation to Tribe from EPQ HQ and informs Tribe
(April or May)
Tribe submits draft workplan to Region 7 PO
Tribe submits full award application . Region 7 PO prepares funding recommendation, attaching
final workplan..
(at least 2 weeks prior to July 1 award date)
41
-------
DRAFT PROGRAMMATIC PRE-AWARD PROCESS MAP OF GAP TRIBAL GRANTS
Tribe provides Region 7 PO with initial draft of the workplan in national format. Grants start
dates are staggered throughout the year.
(4-6 months before the current grant expires)
Region 7 receives ~$990,QOO in GAP funds annually from EPA AIEO, but available funds vary from
year to year. Region 7 sends out a Notice of Funding Availability.
(typically in March)
Tribe submits draft workplan to Region 7 PO.
inalized workpl;an reviewed by Region 7
Tribal Program Coordinator reviews it.
Tribe resubmits
evised workplan
to Region 7 PO
While workplan is being finalized,
the Tribe prepares the application
for Tribal Council signature.
The Tribe sends the completed application to the Region 7 grants office. If the final workplan is
not attached to the application, Region 7 GAP PO will attach it in-house.
I
lication processed by the Region 7 grants office and the Region 7 PO prepares the Funding
Recommendation and Commitment Notice for signature and award bytheGMO.
42
-------
DRAFT PROGRAMMATIC PRE-AWARD PROCESS MAP OF CURRENT
CWA 106 TRIBAL GRANTS
Tribe and Region 7 engage in preliminary discussions about funding availability and possible
project activities.
Tribe submits draft workplan and budget proposal to Region 7 PO.
(4 months before fun ding start date)
While
workpfan is
being
finalized,
the Tribe
finalizes
the budeet.
When
budget is
final, the
Tribe
submits its
award
application.
(60 days
before the
funding
L—I
L»B|
Region 7 reviews
the revised
workplan.
(within 30 days of
receipt)
Tribe resubmits
revised workplan
to Region 7 PO.
(within 30 days of
r^rAlnt nf PH
comments)
Region 7 PO sends
comments to Tribe
via email. Later
rounds of revision
may include
follow-up via
phone or face-to-
face meetings.
(With in 30 days of
Region 7 PO prepares funding package for signature by Watershed Planning and
Implementation Branch Chief and Water Wetlands and Pesticides Division Director.
The award follows the same process as other program awards.
43
-------
APPENDIX H:
TRIBAL BEST MANAGEMENT PRACTICES GUIDE
This 'Best Management Practices Guide' is not prescriptive. It is up to each tribal environmental
director and EPA Project Officer (PO) to determine the best methods for interacting and to work
better as partners with "give and take." Experience has suggested that the Practices in this Guide may
be useful for effective management of tribal grants and cooperative agreements. Effectiveness and
timeliness of work products and processes depends completely upon cooperation and communication
between all parties involved in tribal grants management.
DEFINITIONS
Assistance agreement: The legal instrument used when the principal purpose of the relationship is the
transfer of money, property, or anything of value to a State or local government or other eligible
recipient to accomplish a public purpose of support or stimulation authorized by Federal Statute. An
assistance agreement is an overarching term for both grants AND cooperative agreements.
Grant agreement (which may also be called an assistance agreement) does not substantially
involve EPA in the project and the recipient has the authority and capability to complete all elements of
the program.
Cooperative agreement includes substantial involvement between EPA and the recipient during
performance of the contemplated activity.
Performance Partnership Grant enables grantee to combine funds from more than one
environmental program grant into a single grant with a single budget.
Award: the document, when signed by both parties (EPA and the Tribe) allows for funds to be
distributed and drawn.
General Assistance Program (GAP): Funding for federally recognized tribal governments to protect
environmental and public health in Indian Country in the following areas: legal, enforcement and
compliance, technical and non-administrative, communications, administrative, solid and hazardous
waste.
Joint Evaluation: Evaluation that is carried out by representatives of multiple organizations or groups
working together that have a stake in the findings.
Program Grants: Grants listed under 40 CFR Part 35.101. Examples: Clean Water Act 106; Underground
Storage Tanks, Clean Air Act 105.
Project Grants - The funding, for fixed or known periods, of specific projects or the delivery of specific
services or products, without liability for damages for failure to perform. Project grants include
fellowships, scholarships, research grants, training grants, traineeships, experimental and demonstration
44
-------
grants, evaluation grants, planning grants, technical assistance grants, survey grants, construction
grants, and unsolicited contractual agreements.
BEST MANAGEMENT PRACTICES - Grants
1. BMP: In order to allow enough time for workplan negotiation, review, tribal council approval of
application, submission of application to EPA, and the award process, negotiation should begin at least
120 days (four months) prior to project period start date.
2. BMP: Draft workplans should be submitted to the EPA grants office with the application at least 60
days prior to the project period start date. By the time applications are received by the EPA grants
office, workplans should be essentially complete. By statute Brownfields and Superfund programs have
90 days to fund or return an application
3. BMP: Where not mandatory, tribes have the flexibility to use the template they choose for workplans
and reports, as discussed with the EPA PO. This is true for nearly all EPA programs (except for GAP).
Sample templates are available from EPA program offices.
4. BMP: To prevent long negotiation times for workplans, all POs, technical advisors, and tribal contacts
will be accountable to review and provide comment to the workplan as soon as possible but within no
more than 30 calendar days from receipt of draft workplan submission. If this timeline cannot be met,
please communicate this at your earliest convenience.
Workplans should not take more than two drafts to reach the third and final document. If consensus is
not reached, a new approach is necessary (meet face-to-face in order to create a 3rd draft.)
5. BMP: To prevent long review times for performance reports, all POs, technical advisors, and tribal
contacts will be accountable to review and provide comment to the report as soon as possible but
within no more than 30 calendar days from receipt of draft report submission. For the purposes of this
document, performance reports refer to quarterly and final reports. If this timeline cannot be met,
please communicate this at your earliest convenience.
6. BMP: Workplan negotiations should begin with the EPA and tribal counterparts agreeing to
negotiation milestones and discussing workplan ideas up front. The milestones should include a face-to-
face kickoff meeting including all program staff and managers, or representatives of managers (i.e.,
Section Chiefs) as resources allow, who have activities in the workplan.
At this meeting a draft workplan should be nearly complete. Meeting in person and discussing
workplans verbally are more productive; e-mail should be used for sharing documents, documenting
conversations for the grant file, and resolving minor issues.
Workplans should focus on content and results, accuracy, and workplan format; grammar and
punctuation, are secondary.
Budget numbers are an estimate of anticipated workplan activities, but they should be at least within 10
percent of the final application amount. Remember that by the time an application is submitted,
workplans should be essentially final.
7. BMP: When announcing funding opportunities for tribes, EPA (when possible) needs to be as clear
and precise as possible regarding how much funding is projected to be available in order to help the
tribes develop an appropriate project in the workplan based on the funding that will be received.
45
-------
Funding opportunities include but are not limited to "Notice of Funding Availability (NOFA),"
competitive announcements, or Request for Proposals. EPA personnel may not talk to the tribes once
these announcements are made except in very general terms. Prior to the announcement, EPA may
provide more specific instruction
8. BMP: Grant applications need to be passed by tribal council resolution prior to submitting the
application to the EPA grants office. Tribal councils generally meet at least once per month. Workplans
and award amounts need to be close to final in time for tribal council meetings. This should be
considered when developing the workplan negotiation milestones at the beginning of the workplan
negotiation process.
9. BMP: Create project schedules (on a calendar) to stay on task and to better manage tribal grants and
balance priorities. What gets done is what gets measured.
10. BMP: Be aware that grant project periods may not be on tribal fiscal years. This needs to be
considered when negotiating workplans.
11. BMP: Project periods should be up to but not more than two years to reduce administrative burden
and reduce the time it takes to negotiate a workplan (workplans and budget will need to cover both
years of the project, although, funding is only available annually).
12. BMP: Draft workplans in Microsoft Word format are more useful and make EPA and Tribal
review/comment easier as notes can be added to the document rather than create a new document and
have longer more complicated comments.
13. BMP: Workplans should be written clearly, concisely, and to the point. The workplan should be an
accumulation of discussions and should stand on their own.
BEST MANAGEMENT PRACTICES - Communications
14. BMP: GAP POs may hold regularly scheduled calls with tribes and EPA POs and technical advisers to
discuss progress and various issues. If practical, include the call schedules in the GAP workplan.
15. BMP: Tribal counterparts should call or email PO discuss/notify of changes or considered changes
to a project. This could also be done during the monthly calls listed previously.
16. BMP: When EPA is made aware of visits to tribal offices, the Tribal Environmental Department
Director will be notified three days prior to visit via phone and written correspondence (e-mail is fine).
This is especially important for visits to utility departments or Casinos when the Environmental
Department may not be in the loop.
17. BMP: EPA and Tribal personnel will notify in writing (e-mail is fine) when there is a change in
personnel.
18. BMP: EPA and Tribal personnel will acknowledge receipt of e-mails.
46
------- |