SERA
May 2007
Evaluation of the
Tribal General
Assistance
Program
Promoting Environmental Results
•4
Through Evaluation
-------
This evaluation was performed by Industrial Economics, Incorporated
(lEc) for EPA's Office of Environmental Policy Innovation under
Contract EP-W-04-023 between EPA and lEc. The lEc evaluation
team included Tracy Dyke-Redmond and Heather Posner and Allison
Barasz. Rodges Ankrah, Teresa Kuklinski and Candi Schadel of
EPA's Office of Water, American Indian Environmental Office and
Yvonne M. Watson of EPA's Office of Environmental Policy
Innovation as the technical advisor.
This report was developed under the Program Evaluation Competition,
co-sponsored by EPA's Office of Policy, Economics and Innovation
and the Office of the Chief Financial Officer. To access copies of this
or other EPA program evaluations, please go to EPA's Evaluation
Support Division's website at http:www.epa.gov/evaluate.
-------
TABLE OF CONTENTS
EXECUTIVE SUMMARY ES-1
CHAPTER 1: INTRODUCTION 1-1
I. Overview of The Tribal General Assistance Program 1-1
II. Purpose/Objectives of The Evaluation 1-5
III. Evaluation Questions 1-6
IV. Structure of The Report 1-8
CHAPTER 2: GAP EVALUATION METHODOLOGY 2-1
I. Evaluation Design 2-1
II. Steps for Conducting the Evaluation 2-2
III. Quality Assurance Procedures 2-18
IV. Strengths and Weaknesses of The Evaluation Design 2-18
CHAPTERS: GAP EVALUATION FINDINGS 3-1
I. Federally Recognized Tribes' Access to GAP 3-2
II. Tribal Utilization of Resources Provided by GAP 3-3
III. Indicators of Tribal Environmental Capacity 3-12
IV. GAP Process Outputs to Achieve Tribal Goals and Priorities 3-26
V. GAP Support for EPA's Strategic Goal of Increasing Tribes'
Ability to Build Environmental Program Capacity 3-28
CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS 4-1
Appendix A: GAP Evaluation Questions
Appendix B: Interview Guide for Regional GAP Project Officers
Appendix C: Discussion Guide for Tribes
Appendix D: List of Tribes in Evaluation Sample
Appendix E: GPRA Codes and Activity Categories
Appendix F: Tribal Population Statistics
Appendix G: Activity Comparison Across Tribe Sample Groups
Appendix H: Regional GAP Project Officers Interviewed
Appendix I: Tribes Participating in Panel Discussions
Appendix J: Sources of Non-GAP Funding and Program Support
-------
EXECUTIVE SUMMARY
Introduction
The EPA is responsible for administering Federal environmental statutes on all U.S.
lands, including Indian country. The EPA recognizes tribal governments as the primary
parties for making environmental policy decisions and implementing environmental
programs that affect Indian communities. GAP was established under the authority of the
Indian Environmental General Assistance Program Act of 1992. The primary purpose of
GAP is to help federally recognized tribes and intertribal consortia build the basic
components of a tribal environmental program, which may include planning, developing,
and establishing the administrative, technical, legal, enforcement, communication, and
outreach infrastructure.
In 2004, EPA's American Indian Environmental Office, which manages GAP, applied for
funding assistance from EPA's Office of Policy, Economics, and Innovation for a
program evaluation to determine how effective GAP has been in building Tribal
environmental capacity. For the purpose of this evaluation, Tribal environmental
capacity is defined as administrative, legal, technical and enforcement capability of
Tribes to develop and implement a Tribal environmental program, and communications
capability to work with Federal, State, Local, Tribal, and other environmental officials.
This evaluation is the result of that request and is designed to answer the following five
groups of questions:
1) Is the GAP being accessed by all federally-recognized tribes? If not, why are
some tribes not involved in GAP? Are there tribes that received GAP grants at
one time but which no longer receive GAP grants? If so, why?
2) Are tribal governments using the resources (technical, fiscal, and programmatic)
provided as a component of GAP?
a) How often are they accessed?
b) How are tribes using these resources?
c) To what extent have tribes met program expectations for grants management,
execution of administrative functions, and carrying out proposed activities?
d) How does participation in GAP increase understanding of the process required
to develop a tribal environmental program?
3) What indicators of tribal environmental capacity exist?
a) To what extent have tribes achieved environmental capacity as suggested by
the presence of these indicators?
b) What factors contribute to the achievement of environmental capacity, and
what is the impact of these factors?
c) What is the relative contribution of GAP toward achieving capacity?
4) Is the GAP process providing adequate outputs to achieve tribal goals and
priorities?
ES-1
-------
5) To what degree does GAP support EPA's strategic goal of increasing tribes'
ability to build environmental program capacity?
Methodology
To address these questions, the evaluation drew on several sources of information,
including existing databases file reviews for a sample of 111 tribes in the nine EPA
regions with federally recognized tribes. Databases reviewed included:
1. GAP Accountability Tracking System, which contains records from a sample
of GAP recipients who had received GAP funding prior to 2003 and provides
documentation of their programming efforts under GAP;
2. Grants Information and Control System, which contains records of all EPA
grants, including all GAP grants;
3. Audit Database, which contains information on audits of government grants to
tribes and states, including audit findings and dates for EPA grants awarded to
tribes; and
4. Strategic Goals Reporting System, which contains records on how GAP grants
support EPA's Strategic Goal 5, Objective 5.3, which is to build tribal
environmental capacity.
The database reviews were supplemented with reviews of regional files containing grant
documents (e.g., quarterly reports provided by tribes funded under GAP) in order to
ensure adequate representation of tribal grants reviewed across EPA regions. Note that
GAP database and file review data represents GAP grantee activity from October 2000 -
September 2004, and is therefore somewhat out of date.
In addition to database and file reviews, the evaluation was based on discussions with key
stakeholders, including panel discussions with tribal representatives at three regional
tribal meetings and interviews with GAP project officers in eight regions. These
discussions provided a more recent perspective on the extent to which GAP is supporting
development of tribal environmental capacity.
The draft evaluation methodology was peer reviewed by EPA, tribal representatives, and
academic evaluation experts. The draft methodology was modified to address comments
from the peer reviewers, including changes such as:
• Inviting greater tribal input during the interviews of Tribal representatives;
• Asking tribal representatives about their definition of environmental capacity; and
• Assessing the extent to which organizations other than EPA, as well as program
areas within EPA other than GAP, have helped Tribes develop environmental
capacity.
ES-2
-------
Findings
Federally Recognized Tribes' Access to GAP:
During 1994-2004, 89 percent the 561 federally recognized tribal governments in the
United States received at least one GAP grant. The tribes that did not receive GAP grants
had various reasons for not accessing this funding, such as:
• A policy of not accepting federal grant money
• Very small tribes without basic infrastructure to apply
• Non-reporting or fiscal mismanagement made some tribes ineligible
• In one instance, lack of regional staff to process applications.
• Lack of GAP funding hindered the development of these tribes' environmental
programs (except for tribes with significant financial resources of their own).
Tribes' Use of GAP Resources
Between 1994 and 2004, the 111 tribes in the sample received an average of seven GAP
grants; on average the value of each grant was $102,472. A majority (76 percent) of
tribes in the sample accessed technical resources, such as workshops or training. GAP
facilitates contact and networking with regional tribal staff, other tribes, EPA media
program offices, and non-EPA agencies and organizations. A minority (23 percent) of
tribes in the sample accessed programmatic resources, defined as GAP-specific grants
management or fiscal administrative training.
How Tribes Are Using GAP Resources
Tribes use GAP funds and technical and programmatic resources primarily to establish
and maintain a tribal environmental presence in Indian country, which many tribes define
as having a qualified staff person available on the reservation to respond to
environmental issues of concern to their tribal council and members. In addition, tribes
use GAP resources to participate in a variety of activities that help build their
environmental capacity and expand their environmental presence. We examined the
types of activities conducted by the 96 tribes in our sample for which we were able to
obtain activity data from the either the GAP database or file reviews. Nearly all tribes
(98 percent) participated in activities related to the general management and
administration of their environmental programs. A majority of tribes also participated in
land activities (84 percent), water activities (73 percent), and grant writing activities (65
percent). A smaller proportion of tribes conducted air activities and special emphasis
activities.
A further analysis of tribal activities funded by GAP by activity type shows that more
than 80 percent of tribes in the sample participated in program development or
establishment, staffing, and communication activities. Approximately two-thirds of
ES-3
-------
tribes engaged in baseline assessment and grant writing activities. Considerably fewer
tribes conducted activities associated with media-specific programs, such as the
development of Quality Assurance Project Plans and monitoring capacity; the
development of legal tools such as codes, ordinances, standards, and permitting authority;
and the administration of grants received in support of these programs. Only two tribes
participated in database development activities.
How Tribes Are Meeting EPA Expectations
The award of GAP grants brings with it EPA's expectation that tribes will fulfill the
requirements of GAP for demonstrating accountability in the utilization of funds as well
as for grants management and performance reporting, detailed in the 2000 GAP
Guidelines. Based on our interviews with regional POs, we found that, overall, tribes are
meeting regional expectations for grants management, the execution of administrative
functions, and carrying out proposed activities. Tribes continue to improve the
timeliness, quality, and completeness of their GAP work plans and progress reports.
Currently, most tribes in a majority of regions are submitting their work plans and
progress reports on time.
We also examined the results of A-133 audits conducted for the 111 tribes in the sample
to assess tribes' ability to execute administrative functions. Approximately 25 percent of
the 111 tribes in our sample had been audited during the period 1997-2004. Of these 27
tribes, the audits for 24 resulted in at least one reportable condition, material weakness, or
material noncompliance outcome. Audit findings mainly cited problems with tribes'
ability to correctly track and document expenditures. Note that because participation in
an A-133 audit is required only when a tribe's total annual expenditures of federal funds
exceed a high threshold, most of the tribes in the sample would not likely have to
undergo such an audit. As a result, the tribes in the sample that were audited and cited
with a reportable condition, material weakness, or a material non-compliance may not be
representative of the ability of the tribes not audited to execute administrative functions
pertaining to GAP grants. It may also be the case that since A-133 audits include a
review of all federal expenditures for a tribe, the findings recorded in the Audit Database
for tribes in our sample may not be related to tribal fiscal performance under GAP.
Regional Project Officers for GAP identified only a few tribes that had received a major
finding on an Al33 audit.
Influence of GAP on Tribes' Understanding of Environmental Program Development
In addition to assessing the direct resource outputs provided by GAP, the evaluation
seeks to discern how tribal participation in GAP and utilization of GAP resources has
influenced, 1) tribes' understanding of the process required to develop an environmental
program, and, 2) the way tribes approach the various administrative and programmatic
functions associated with the development process. Tribal representatives emphasize that
instead of changing tribal understanding of how to develop an environmental program,
GAP facilitates tribes' ability to develop a program that is responsive to each tribe's
unique environmental conditions and priorities. GAP resources enable tribes to establish
an environmental presence, which in turn provides the foundation upon which each tribe
ES-4
-------
can build an environmental program tailored to meet its needs. From EPA's perspective,
however, GAP may influence and clarify tribal priorities as tribal environmental staff
acquire training, learn about specific environmental conditions on tribal lands, and
become more aware of concrete program opportunities through their interactions with
EPA regional tribal and media program contacts.
Indicators of Tribal Environmental Capacity
Many tribal representatives offered definitions of capacity and indicators that fall within
the GAP category of technical capability, such as: hiring and training of qualified
environmental professionals and expansion of tribal environmental programming efforts
to include media-specific components. A few tribes linked environmental capacity with
legal or enforcement capability.
Regional project officers identified many of the same key indicators of capacity as tribes,
such as tribes' ability to establish an environmental presence; retain qualified,
knowledgeable staff over the long-term; and diversify their environmental programming.
Achievement of Tribal Environmental Capacity
In order to determine the extent to which tribes in the sample have achieved
environmental capacity as defined by GAP, we examined tribal capability in each of the
five indicator areas - technical, legal, enforcement, administrative, and communication.
We identified a set of coded activity types for each indicator and equated tribal capability
in that area with a tribe's participation in one or more related activity. Exhibit ES-1 lists
the activity types selected to demonstrate tribal capability for each indicator and the
proportion of the 96 tribes that participated in activities within each type during 2000-
2004.
Exhibit ES-1: Tribal Achievement of Environmental Capacity, 2000-2004 (n = 96)
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental Capacity
Developed a Code, Ordinance, or Standard
Participated in an Activity to Increase Legal Capacity
Adopted a Code, Ordinance, or Standard
Participated in an Activity to Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to Increase Fiscal Administration Capacity
Participated in Internal Communication Activities
Participated in External Communication Activities
Participated in General Communication Activities
Number
of Tribes
25
24
7
25
86
70
70
47
15
68
66
28
Percent of
Tribes
26%
25%
7%
26%
90%
73%
73%
49%
16%
71%
69%
29%
ES-5
-------
Factors that Contribute to the Achievement of Environmental Capacity
In addition to requesting tribal and regional input on the most important indicators of
tribal environmental capacity, we asked tribes and POs to identify factors that impact
environmental capacity and describe how they influence tribal efforts. As ranked by
project officers, the following factors almost always or often influence environmental
capacity:
• Tribal council support for environmental programs
• Qualifications of tribal environmental director and/or staff
• Turnover rate of tribal environmental director and/or staff
• Clear tribal environmental priorities
• Degree of information sharing among tribes
• Access to funding outside of GAP.
Factors tribal representatives consider most influential include:
• Stability of knowledgeable tribal environmental staff
• Effective communication between tribes and EPA regions (and between the EPA
regional tribal offices and media programs)
• Support of tribal council for planning and funding environmental programs
relative to other tribal priorities.
Relative Contribution of GAP Toward Achieving Environmental Capacity
There are many factors that can potentially affect tribes' achievement of environmental
capacity such as GAP funding and technical assistance, the stability of tribal leadership
and staffing, and the degree to which Tribal council members focus on environmental
concerns. Another potential factor is tribal access to other sources of funding (e.g., EPA
media programs, other federal and state agencies, and tribes themselves).
Interviews with tribal representatives and POs make clear that they perceive GAP
funding as essential to achieving environmental capacity. Many tribes say that without
GAP funding, they would be able to do very little environmental work. They stress that
GAP is the foundation for their environmental programs, and GAP resources enable them
to establish a basic program infrastructure, through which they can apply for other types
of environmental funding. This view supports a basic premise of the GAP program,
namely, that as GAP helps tribes build their environmental capacity, tribes will be able to
access other sources of funding to support their environmental programs.
A comparison of tribes that first accessed GAP early in the program's existence (1994 -
1999) to those that first accessed GAP in later years (2000 - 2004), shows that tribes that
accessed GAP earlier have acquired a greater proportion of non-GAP EPA funding
relative to GAP funding. This supports the view that GAP is helping tribes expand their
sources of environmental funding, which suggests that tribes have increased their
environmental capacity accordingly. However, tribes that accessed GAP earlier have not
achieved a greater number of indicators of environmental capacity with GAP funds
ES-6
-------
compared to tribes that adopted GAP later. The results of this analysis run counter to the
hypothesis that tribes that have had GAP funding for a longer period of time would be
further along in the process of developing environmental capacity, compared to tribes
that had received GAP funding for a shorter period of time.
Sufficiency of GAP Outputs to Achieve Tribal Goals
Many of the tribal representatives interviewed stated that a key goal for tribes is having
an environmental presence on tribal land, i.e., a qualified staff person who can coordinate
the tribe's environmental programs, maintain a cohesive program, and be a point of
contact for members of the tribal community and neighboring communities. GAP
enables tribes to establish this environmental presence by providing the funds to hire,
train, and retain professional and technical environmental staff. Tribes emphasize that
GAP provides a foundation for tribal environmental programs.
While tribal representatives state that GAP funding is vital for establishing and
maintaining an environmental presence, many perceive current levels of GAP funding as
insufficient. When asked about additional resources tribes need to develop their
environmental programs, several tribes indicate that above all else, they need sustained,
consistent funding over time to enable them to hire and retain sufficient qualified staff,
and thereby retain institutional knowledge. GAP provides an important source of
sustained funding, although some tribes note that the requirement to re-apply for GAP
funding every year takes away from the stability of the GAP program and the staff that
GAP supports.
Tribes sometimes have goals and priorities that GAP does not address, either because
GAP funding is insufficient to meet these goals, or because these goals involve
implementation of environmental programs. Most tribes interviewed say that in order to
meet tribal goals, they need to be able to use GAP funding for program implementation
and maintenance, and they need additional funding to support this additional effort. In
addition, while most tribes perceive overall consistency between GAP goals and tribal
priorities, they also pointed out that it is difficult to mesh the cultural and traditional
values of the tribes with the bureaucratic and regulatory guidelines and definitions
established by GAP.
GAP Support of EPA7 s Strategic Goals
EPA's 2003 - 2008 Strategic Plan includes Objective 5.3, the percent of tribes that "had
access to an environmental presence." This indicator increased from 36 percent in 1996
to an estimated 90.4 percent in FY 2006, with a peak of 97 percent in FY 2004. Access
to an environmental presence is defined as the annual dollar value of GAP funding that
AIEO determines is needed to establish an environmental presence.
EPA's updated 2006 - 2011 Strategic Plan includes new targets for building program
capacity, including: increasing the percent of tribes conducting EPA-approved
ES-7
-------
environmental monitoring and assessment activities and increasing the percent of tribes
with an environmental program. Further data is needed to conclusively assess progress
toward these new targets.
Conclusions
Based on the findings from this evaluation, the evaluators conclude that:
• The extent of capacity-building varies across indicator areas for tribes in the
sample that received GAP grants. These tribes have relatively well-developed
technical and communications capabilities, but less developed legal, enforcement,
and administrative capacity.
• GAP has done much in recent years to clarify grant expectations and
administrative requirements for tribes, and tribes in turn are increasingly meeting
these expectations and requirements.
• Tribes report that restrictions on GAP grants that preclude using GAP funds for
program implementation are now hindering tribal environmental program
development.
• Tribes emphasize that GAP funding is essential to achieving their environmental
goals, but perceive that current levels of funding are insufficient to address tribal
priorities.
Recommendations
The evaluation team makes the following recommendations for EPA based on the
findings and conclusions from the evaluation:
Consider developing a mechanism to support tribal program implementation.
• EPA HQ and Regions could continue to promote and expand the use of
Performance Partnership Grants (PPGs) by tribes.
• AIEO could eventually establish a second tier of GAP funding - "GAP plus" - to
fund program implementation for those Tribes that show they have met key
indicators of capacity under GAP.
• Another approach could be to establish a block grant for tribes similar to those
established for U.S. territories.
Consider working more directly with tribes and regions to enhance administrative, legal,
and enforcement capacity.
• To help build administrative capacity, AIEO could coordinate with regions to
ensure that programmatic resources provided keep pace with tribal needs.
• EPA regions could offer legal support to help tribes enact their own codes,
ordinances, and standards.
ES-8
-------
• In cases where tribes feel that they cannot or do not wish to implement their own
environmental laws and regulations, AIEO should consider developing a coherent
plan for working with tribes to protect the environment, while respecting tribal
sovereignty.
Raise awareness of innovative environmental policy approaches that complement
traditional codes and standards.
• Tribes may benefit from a greater emphasis on pollution prevention education,
self-certification, and compliance assistance inspections.
• Tribes could leverage the considerable experience of EPA and states in
developing innovative policy tools and approaches, as well as specific outreach
materials.
• AIEO and regions could help tribes by raising awareness of innovative policy
approaches, readily available materials, and potential funding sources.
Acknowledge cross-cultural differences, and continue working with tribes to maintain a
respectful dialog.
• A key difference in perspective is that Tribes see GAP funding as an extension of
EPA's trust responsibility.
• EPA views tribes as grantees that must meet certain requirements to show that
they are accountable for funds.
• To foster greater understanding, Tribes suggest hiring more Native Americans to
serve as regional POs and tribal coordinators.
• More frequent site visits to tribes by AIEO and EPA regional program staff could
help underscore the diversity of tribal perspective, priorities, and approaches to
environmental protection.
Track progress toward achievement of the new 2006-2011 strategic goals and targets.
• AIEO needs to ensure that its data collection systems allow for the effective
capture and tracking of indicators related to the updated strategic targets.
• AIEO should consider the degree to which its proposed performance measures
align with tribal priorities and perspectives, and the feasibility of tribes' of
achieving them.
• The five-year cycle for setting strategic goals and targets may be too short to
effectively track and measure tribal progress. AIEO should consider keeping
consistent goals for a longer period of time.
ES-9
-------
CHAPTER 1: INTRODUCTION
This report summarizes the evaluation of the tribal General Assistance Program (GAP)
conducted between 2005 and 2007 by Industrial Economics, Inc (ffic). The first section in this
chapter begins with an introduction and overview of the tribal GAP, including a logic model that
describes the program's design. The next section describes the purpose and objectives of the
evaluation, and provides a review of the specific questions the evaluation is designed to answer
and the relationship between the evaluation questions and elements of the logic model. The
chapter concludes with a description of the structure of the evaluation report.
I. OVERVIEW OF THE TRIBAL GENERAL ASSISTANCE PROGRAM
The EPA is responsible for administering Federal environmental statutes on all U.S. lands,
including Indian country. The EPA recognizes tribal governments as the primary parties for
making environmental policy decisions and implementing environmental programs that affect
Indian communities. GAP was established under the authority of the Indian Environmental
General Assistance Program Act of 1992. The primary purpose of the GAP is to help federally
recognized tribes and intertribal consortia build the basic components of a tribal environmental
program, which may include planning, developing, and establishing the administrative, technical,
legal, enforcement, communication, and outreach infrastructure. Total GAP funding has
increased from initiation of the program, starting at a total of $5.4 million in 1992 to $60.4
million in 2005, although funding for 2006 was projected to decline slightly to $56.9 million.1
As shown in Exhibit 1-1, average funding per tribe has ranged from a low of $26,738 in 1996 to
a high of $106,623 in 2004. Note that in recent years (2005 and 2006), the average amount of
funding per tribe has declined. Note also that not all eligible tribes receive GAP funding, and
funding is not distributed evenly between tribes, so these figures provide only a general idea of
the level funding available to tribes over time. Chapter 3 provides further details about the
amount of funding provided to GAP grantees.
1 Data is drawn from the Goal 5, Objective 5.3 Reporting System, available online at
https://iasint.rtpnc.epa.gov/TATS/tats_prv/tats securitv.login form?p mode=reports. under Target 1 Program
Performance Report. Last accessed April 2007.
1-1
-------
Exhibit 1-1: History of GAP Funding*
Year
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
Total GAP
Funding
$15,000,000
$28,000,000
$38,500,000
$42,000,000
$42,000,000
$52,000,000
$52,000,000
$56,150,000
$60,991,000
$60,404,000
$56,900,000
Number of
Federally
Recognized Tribes
561
562
565
565
567
572
570
572
572
572
572
Average Amount of
Funding per Tribe
$26,738
$49,822
$68,142
$74,336
$74,074
$90,909
$91,228
$98,164
$106,628
$105,601
$99,476
Percent Increase (Decrease) from Prior
Year in Average Amount of Funding
Per Tribe
—
86%
37%
9%
0%
23%
0%
8%
9%
(1%)
(6%)
*Data on number of eligible entities, and therefore amount of funding per tribe, is not available for 1992 to 1995.
The GAP provides annual grant funding to federally recognized tribes and intertribal consortia
through a negotiated process administered by each EPA region. The grant funds may be used by
tribes to plan and carry out any number of capacity-building activities including the development
of administrative procedures; quality assurance/quality control systems; sampling and laboratory
capabilities; baseline environmental assessments; enforcement programs; legal procedures;
communications plans; computer information systems; and staff qualifications and expertise.
GAP may not be used for the ongoing implementation of media-specific environmental programs
once established, with the exception of solid waste program implementation activities. Since
GAP funds may generally not be used for implementation activities, the GAP program defines its
outcomes as changes in knowledge and behavior (i.e., short-term and intermediate outcomes, as
described below).
To illustrate the various components of the GAP, EPA and lEc developed a logic model (i.e., a
graphical representation of the relationships between program inputs, outputs, and intended
outcomes), presented in Exhibit 1-2. Key components include the following:
• Resources are the basic inputs of funds, staffing, and knowledge dedicated to the program.
• Activities/Outputs are the specific actions taken to achieve program goals and the
immediate products that result. Under the GAP, these products include grant funds, technical
assistance, training, and grant oversight.
• Customers are the users of the activities and outputs (fiscal, technical, administrative)
provided. They are the tribal governments that receive GAP grants and the environmental
employees hired with GAP funds.
1-2
-------
• Short-Term Outcomes are changes in awareness, attitudes, understanding, knowledge, and
skills resulting from program outputs. Technical and grant management training
opportunities provided to tribal environmental employees through the GAP increase
understanding of the processes required in developing a tribal environmental program. Note
that outcomes listed in italics are intended as illustrative examples of the intended effects of
tribes' increased understanding for how to develop a tribal environmental program.
• Intermediate Outcomes involve changes in behavior that are broader in scope than short-
term outcomes. Intermediate outcomes often build upon the progress achieved in the short-
term. Under the GAP, changes in tribal awareness, understanding, and skill level pave the
way for planning, development, and initiation of capacity-building activities. The logic
model includes examples of activities that represent increased capability in the legal,
enforcement, technical, communications, and administrative arenas. Note that outcomes
listed in italics are intended as illustrative examples of the effects of tribes' increased legal,
enforcement, technical, communications, and administrative capability.
• Long-Term Outcomes parallel the overarching goals of the program and are the
environmental improvements and public health benefits that flow from the behavioral,
procedural, and operational changes.
• Contextual/External Variables are factors, not directly controlled by the program or its
entities, which may affect program performance. For example, changes in tribal policy and
budgetary priorities may influence the ability of tribes to sustain environmental efforts.
Elements of the logic model, noted by the circled letter codes, are referenced in section III in the
discussion of the evaluation questions.
1-3
-------
Exhibit 1-2. GAP Grant Program Evaluation Logic Model
Legend: Letter codes are used to connect elements of the logic
model to the evaluation questions, as shown in Exhibit 1-3.
i Resources
i Activities ! j Outputs ! j Customers
i Outcomes i
Short-Term I
Intermediate
Long-Term
Provide
Funding to
Tribes
.
©
GAP Grants
.
k.
^
Tribal
Executives
Inter-tribal
Consortia
Executives
Agency
Technical
Expertise
A
Dedica
Staffin
FTE-
Regior
Nation
Progra
Manag
k
ted
g;65
AIEO,
is, and
al
m
ers
Prov
Tecr
Assi
Trib
ide
stance to /D^N
3S \ J
r
^
w
Pro's
Rec
Gra
Ove
Trib
External Factors:/ s
tribal leadership, (*-'
vision, continuity,
priority of
environmental issues,
education levels, staff
turnover, and
resource levels.
Assumptions: GAP
funding facilitates
ability of tribes to
acquire other EPA
grants that also
contribute to
capacity-building.
/ide
pient/
it ^^^
rsight to £ '
es ^~—S
L
Technical and Environmental
Media-Specific \~W\ Employees
Trainings fn\ Funded by GAP
Site Visits
Grant
Management
Trainings
GAP Data
Collected and
Performance
Tracking
Performed
J
k
_fe
Increased
understanding of the
process required in
the development of a
tribal environmental
program.
Increased
awareness of
multiple media
program areas.
Increased
knowledge of grants
management
requirements and
approaches.
Tribes develop legal and
enforcement infrastructure, i.e.,
codes, regulations, ordinances, and
standards that can be used to
implement management policies
and guidelines.
Tribes demonstrate ability to
perform the inventories,
monitoring, and inspected needed
to ensure compliance with
environmental policies and
guidelines.
Tribes develop technical skills for
environmental management such as
monitoring and analysis, baseline
assessment, data management,
quality assurance procedures, and
emergency response systems.
Assumption or creation of media-
specific programs.
Tribes demonstrate ability to
communicate about environmental
issues with the community, tribal
executives, the regulated
community, and other government
entities.
Tribes establish ability and
procedures for managing and
accounting for program funds,
including procedures for staffing
and training, management of office
resources and personnel, and
communication with other tribal
government agencies.
Planning for tribal codes
and ordinances
Ratification of tribal
codes and ordinances
Certification of tribal
programs
Approval of tribal grant
work plans
:::,
Public education and
outreach programs
Established
communication with other
players
Established staffing for
media-specific
Acquisition of equipment
Reductions in
questionable audits
Established capacity to plan,
develop, implement, and
manage environmental
programs.
Achievement of EPA's
Strategic Plan Section 5.3
Goals
Compliance with federal
statutes and regulations
Sustainability of tribal
environmental programs
Improved environmental
conditions in Indian Country
( K J
Consistent Communication between AIEO, regions and tribes about Challenges and Performance
-------
II. PURPOSE/OBJECTIVES OF THE EVALUATION
Since the passage of the Government Performance and Results Act of 1993, federal agencies are
expected to regularly conduct evaluations of their programs as part of a larger effort to promote
results-oriented government.2 EPA's 2003-2008 Strategic Plan reflects a "sharpened focus on
achieving measurable results" and notes that program evaluations are used, "to identify areas
needing improvement, more effective strategies for achieving established goals, and ways to
improve data collection or better measure program results."3 One area where EPA is seeking to
measure its results is in its tribal program.
In 2004, EPA's American Indian Environmental Office (AIEO), which manages GAP, applied
for funding assistance from EPA's Office of Policy, Economics, and Innovation (OPEI) for a
program evaluation to be conducted by an independent evaluator. This evaluation is the result of
that request. AIEO's goal in seeking this evaluation support was to, "... determine how effective
GAP has been in building tribal environmental capacity with those tribes receiving funds[,]...to
see if we are reaching the tribes to change their knowledge and behavior, and to determine how
permanent those changes are."4 The 2000 GAP guidelines establish elements of a core tribal
environmental protection program. These include "establishing] the administrative, legal,
technical and enforcement capability of tribes to develop and implement a tribal environmental
program...[and] [establishing a Tribal communications capability to work with Federal, State,
Local, Tribal, and other environmental officials."5 Having capability in each of these areas
constitutes having environmental capacity for the purpose of this evaluation.
AIEO is also considering whether to expand the GAP beyond its current mandate of building
tribal environmental capacity. Before proceeding with an expansion of the GAP, AIEO has
decided to evaluate the existing program with regard to its impact on tribal capacity development
and its relevance to tribal needs and to Section 5.3 of EPA's 2003-2008 Strategic Plan.
Industrial Economics, Inc. (ffic) was selected to assess the impact of the GAP on tribal
environmental capacity and help AIEO understand which elements of GAP contribute to the
establishment of multi-media environmental programs. The results of this evaluation are
intended to help the tribal program demonstrate its successes to stakeholders and identify
opportunities for improvement. The evaluation results are expected to be of particular interest to
GAP stakeholders involved in awarding grants, providing technical assistance and oversight, and
planning and executing tribal environmental programs. Participation by tribal environmental
National Research Council, Decision Making for the Environment: Social and Behavioral Science
Research Priorities, Washington, D.C.: National Academies Press, 2005.
3 U.S. Environmental Protection Agency (EPA), 2003-2008 EPA Strategic Plan: Direction for the Future.
Washington, D.C., EPA, 2005.
4 American Indian Environmental Office (AIEO), Improving Results: Program Evaluation Competition
Application, Unpublished document provided by AIEO, 2004.
5 U.S. Governmental Protection Agency (EPA), Indian Environmental General Assistance Program:
Guidelines on the Award and Management of General Assistance Agreements for Indian Tribes. Washington, D.C.,
EPA, 2000.
1-5
-------
staff in the evaluation process is intended help build EPA's awareness of tribal priorities and
progress toward achieving environmental goals. The results are expected help AIEO and
Regional GAP project officers more effectively target outreach to tribes and identify the level of
resources (administrative, financial, and technical) necessary to promote and sustain tribal
environmental initiatives. Tribal environmental managers may find the evaluation results useful
in influencing community decision-makers and further raising the profile of environmental
protection programs among tribal members. More broadly, the evaluation results will be of
interest to policy planners at EPA Headquarters tracking progress on the objectives EPA's
strategic plan.6
III. EVALUATION QUESTIONS
The evaluation is designed to answer the following five groups of questions:
1) Is the GAP being accessed by all federally-recognized tribes? If not, why are some tribes not
involved in GAP? Are there tribes that received GAP grants at one time but which no longer
receive GAP grants? If so, why?
2) Are tribal governments using the resources (technical, fiscal, and programmatic) provided as
a component of GAP?
a) How often are they accessed?
b) How are tribes using these resources?
c) To what extent have tribes met program expectations for grants management, execution
of administrative functions, and carrying out proposed activities?
d) How does participation in GAP increase understanding of the process required to develop
a tribal environmental program?
3) What indicators of tribal environmental capacity exist?7
a) To what extent have tribes achieved environmental capacity as suggested by the presence
of these indicators?
b) What factors contribute to the achievement of environmental capacity, and what is the
impact of these factors?
c) What is the relative contribution of GAP toward achieving capacity?
4) Is the GAP process providing adequate outputs to achieve tribal goals and priorities?
5) To what degree does GAP support EPA 's strategic goal of increasing tribes' ability to build
environmental program capacity?
6 This evaluation was specifically designed to assess progress on Section 5.3 of EPA's 2003-2008
strategic plan. After this evaluation was designed, EPA issued an updated 2008-2011 Strategic Plan, with updated
goals and targets for AIEO. While the evaluation was not specifically designed to address progress toward these
updated goals and targets, Chapter 3 does describe findings from this evaluation in relation to the 2008-2011
Strategic Plan.
7 The order of the three sub-questions listed for this evaluation question has been changed in Chapter 3 to
facilitate the presentation of the evaluation findings.
1-6
-------
Exhibit 1-3 lists the final evaluation questions and the components of the logic model to which
they correspond.
Exhibit 1-3:
Relationship Between Evaluation Questions and Logic Model
Evaluation Question
la. Is GAP being accessed by all federally recognized tribes?
Ib. Why are some tribes not involved in GAP?
Ic. Are there tribes that received GAP grants at one time but
which no longer receive GAP grants? If so, why?
2a. Are tribal governments using the resources (technical, fiscal,
and programmatic) provided as a component of GAP? How
often are GAP resources accessed?
2b. How are tribes using GAP resources?
2c. To what extent have tribes met program expectations for
grants management, execution of administrative functions,
and carrying out proposed activities?
2d. How does participation in GAP increase understanding of
how to develop a tribal environmental program?
3 a. What indicators of tribal environmental capacity exist?
3b. To what extent have tribes achieved environmental capacity
as suggested by the presence of these indicators?
3c. What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
Tribal Priorities
Tribal Staffing
Tribal Funding
Communication
Regional Activities
3d. What is the relative contribution of GAP toward achieving
capacity?
4. Is the GAP providing adequate outputs to achieve tribal goals
and priorities?
5. To what degree does GAP support EPA's strategic goal of
increasing tribes' ability to build environmental program
capacity?
Component of the Logic Model
©
(A)
©
© © ©
© © © ® © ©
©
©
® © ® ® ® ®
©
®
® ©
®
(M)
Overarching question
Not directly shown in logic model
®
1-7
-------
IV. STRUCTURE OF THE REPORT
Following this introduction, Chapter 2 of the report presents the methodology used in conducting
the evaluation, including study design, data sources, a plan for data analysis, quality assurance
procedures, and strengths and weaknesses of the methodology. Chapter 3 presents the evaluation
findings, organized by the five evaluation questions described in section III above. Chapter 4
presents the conclusions resulting from the evaluation findings and recommendations to AIEO
for future improvements to GAP.
1-8
-------
CHAPTER 2: GAP EVALUATION METHODOLOGY
This chapter summarizes key aspects of the methodology used to evaluate the General
Assistance Program (GAP). The methodology begins with an overarching evaluation design,
followed by a series of tasks undertaken in conducting the evaluation, including gathering and
analyzing data, interpreting findings, and reporting results. The methodology also addresses
quality assurance procedures used, and comments on the strengths and weaknesses of the
evaluation design.
I. EVALUATION DESIGN
Most program evaluations, including the present study, are designed to address two overarching
questions: 1) what are the program's outcomes, effectiveness, and impacts; and 2) how or why is
a program effective or ineffective.1 It is typically difficult to answer the first question
definitively because it is often not possible to infer causal relationships between programs and
measured outcomes and long-term impacts. Although randomized, controlled trials or
experiments can isolate causal effects, it is very rare to be able to conduct a randomized,
controlled study of government environmental programs. For example, in the present evaluation,
in order to develop a randomized, controlled trial, it would be necessary to award GAP grants to
only a subset of tribes chosen on a random basis, and then compare the outcomes of tribes that
did receive grants to those that did not. Such a design would be impractical and questionable on
legal and ethical grounds.
Since it is not possible to conduct a randomized, controlled experiment for this evaluation, we
chose an alternate evaluation design. In this case, our evaluation design options were limited.
For example, we could not conduct direct controlled trials, since we had no ability to control
variables (e.g., the turnover rate and qualifications of tribal environmental staff, or the grantee
caseload for EPA project officers) that might affect program outcomes. Moreover, we could not
choose a quasi-experimental design, since that methodology relies on selecting comparison
groups (i.e., comparing tribes that received a GAP grant to those that did not). It is not feasible
to make this comparison since the vast majority of eligible tribes have received GAP grants.
Those that have not received such funding are atypical and therefore would not present a valid
basis of comparison (e.g., they are very small tribes, and thus may have inherently different
abilities to achieve environmental capacity with or without GAP grants).
In light of these limitations, we conducted a non-experimental direct analysis. This type of
evaluation examines only the subject group receiving the "intervention" (in this case, a GAP
grant). Our evaluation incorporated aspects of a longitudinal study, which examines conditions
of the study group over time. Specifically, we compared tribes that had GAP grants for a longer
period of time compared to tribes that had GAP for a shorter period of time, to see whether tribes
1 Office of Management and Budget (OMB), What Constitutes Strong Evidence of a Program's
Effectiveness? Washington, D.C., OMB, 2004. Available from http://www.whitehouse.gov/omb/part/2004
program eval.pdf. Accessed July 2005.
2-1
-------
with more years of GAP funding had more effectively developed environmental capacity. For
tribes that had GAP grants for a shorter period of time, we also compared an indicator of
capacity (percentage of EPA funding derived from sources other than GAP) before and after
GAP grants were awarded, to see if access to GAP grants led to an increase in funding
diversification.
This evaluation addresses short term and intermediate outcomes achieved by GAP. However,
GAP grants are limited to capacity-building and are not intended to cover program
implementation (with the exception of solid and hazardous waste programs.)2 Therefore, this
evaluation focuses on short-term and intermediate outcomes (i.e., changes in tribal knowledge
and behavior), rather than long-term outcomes, such as environmental and public health
improvements, that would be achieved only through the implementation of tribal environmental
programs.
II. STEPS FOR CONDUCTING THE EVALUATION
The four major steps taken to conduct this evaluation include: 1) identifying the information
needed to answer the evaluation questions, 2) collecting and analyzing data from existing
databases and files, 3) collecting and analyzing data from interviews panel discussions, and 4)
reporting results and conclusions. Exhibit 2-1 lists the detailed tasks completed under each of
these steps. The four steps are then discussed in more detail in the following sections.
The purpose of GAP grants, which are defined by statue, are explained in the Indian General Assistance
Program 2006 Grant Administration Guidance, available at http://www.epa.gov/indian/pdfs/gap2006.pdf. Accessed
April 2007.
2-2
-------
Exhibit 2-1: Steps for Conducting the Evaluation
A. Identify Information Needed and Prepare for Data Collection:
1) Establish data indicators.
2) Identify data sources.
3) Develop data collection tools (guides for interviews with EPA regional staff and
focus groups with tribal representatives).
B. Collect and Analyze Data from Existing Databases and Files:
1) Select a sample of tribal GAP recipients.
2) Develop an evaluation database.
3) Collect quantitative and qualitative data from databases and grantee file reviews.
4) Categorize and code qualitative data.
5) Summarize data through descriptive statistical analysis.
C. Collect and Analyze Data from Interviews and Panel Discussions:
1) Identify regional GAP coordinators to interview.
2) Schedule and conduct interviews.
3) Select a sample of tribes for participation in group interviews.
4) Schedule and conduct group interviews.
5) Code responses from regional GAP interviews.
6) Analyze trends and patterns in data from interviews and group interviews.
D. Prepare Final Evaluation Report, in Accordance with EPA Guidelines:
1) Introduce GAP program and the purpose of the evaluation.
2) Describe methods for data collection and analysis.
3) Summarize key findings from quantitative and qualitative data analyses, and consider
relationship between quantitative and qualitative findings.
4) Develop conclusions and identify lessons learned.
5) Propose recommendations.
A. Identify Information Needed and Prepare for Data Collection
In preparing this evaluation methodology, ffic identified specific types of information that could
help answer each evaluation question (Appendix A). lEc developed this list of data needs in
consultation with staff from AIEO and regional GAP Project Officers (POs). Indicators of
environmental capacity were initially selected based on the statutory definition of tribal capacity,
which was informed by tribal perspectives at the time that the statute was written (although tribal
perspectives may have changed since that time). AIEO and the regional POs offered feedback
on key indicators of tribal environmental capacity, guidance in defining the overarching indicator
of capacity for tribes, and insight into the factors that may potentially impact tribes' achievement
of environmental capacity. Tribal representatives had an opportunity to review and comment on
the evaluation methodology, which included a description of the proposed indicators of
environmental capacity. The final indicators of capacity were selected with all of this input in
mind.
20
-3
-------
Regional POs also identified several databases that could supply needed data on tribal
environmental programming efforts, including:
1. GAP Accountability Tracking System (GAP database) - This database, maintained by
AIEO, contains records from a sample of GAP recipients who had received GAP
funding prior to 2003 and provided documentation of their programming efforts
under GAP. The database records include details on the grants awarded, activities
conducted with GAP funds, and positions funded by GAP during the period October
2000 through September 2004.3 A total of 92 tribes are included in one or more of
the tables in the GAP database, representing at least 20 percent of tribes in eight of
EPA's ten regions.4
2. Grants Information and Control System (GICS) - This database, maintained by EPA's
Office of Administration and Resources Management, contains records of all EPA
grants, including all GAP grants. We received a pull of all GICS records of GAP and
other EPA grants awarded to tribes from 1994 - 2004. The GICS data includes for
each grant, the tribe name; region; award amount and date project end date, budget
end date, and closeout date; and a description.5
3. Audit Database - (http://harvester.census.gov/sac/dissem/entity.html). This database,
maintained by the U.S. Census Bureau, contains information on audits of government
grants to tribes and States, including audit findings and dates for EPA grants awarded
to tribes.
4. Strategic Goals Reporting System (5.3S-D) - (https://oasint.rtpnc.epa.gov/
TATS/tats_prv/entry_page). This database, maintained by AIEO, contains records on
how GAP grants support EPA's Strategic Goal 5, Objective 5.3, which is to build
tribal environmental capacity. The database contains records on the number of tribes
per region that have achieved Treatment as a State (TAS), Direct Implementation
Tribal Cooperative Agreements (DITCA), GAP Grants, Quality Assurance Project
Plans (QAPPs), Tier III Tribal-EPA Environmental Agreements (TEAs), Performance
Partnership Grants (PPGs), and other agreements (such as Memoranda of
Understanding (MOUs), Memoranda of Agreement (MOAs), and Tier I & Tier II
TEAs).
3 This time period corresponds with GAP work plan fiscal years 2000, 2001, 2002, and 2003. Unlike a
federal fiscal year, which is named for the year ending in September, a GAP work plan year is named for the year
beginning in October. For example, GAP work plan year 2000 began on October 1, 2000 and ended on September
30, 2001.
4 According to AIEO, 20 percent of tribes in most EPA regions were selected for inclusion in the GAP
database, however, where needed, additional tribes were selected to make sure that at least two tribes were included
in the GAP database from each Region.
5 In addition, POs recommended that we also review the IGMS database, which is a subset of the GICS
database. We received and reviewed a data pull from IGMS of grants made to tribes from 1992 to 1999. However,
since GICS contains more complete records, we used the data pull from GICS in lieu of the IGMS data.
2-4
-------
In addition, AIEO and POs provided suggestions for other sources of information to supplement
the data found in existing databases, including:
• Reviews of regional and/or tribal files, such as grant documents, correspondence, and
related materials;
• Discussions with AIEO staff;
• Interviews with regional EPA staff that oversee GAP grants;
• Interviews with tribal representatives; and
• Review of EPA's strategic plan as it relates to GAP.
For each type of information needed to answer the evaluation questions, lEc worked with AIEO
and the regional POs to identify the data source(s) most likely to provide pertinent information.
Finally, lEc developed data collection tools to use in gathering data during interviews with POs
and panel discussions with tribal representatives. These interview and discussion guides were
designed to collect the types of information that could not be gathered through an analysis of
existing databases, and are attached to this methodology as Appendices B and C.
B. Collect and Analyze Data from Existing Databases and Files
The existing databases provided by AIEO include many of the types of information needed to
answer the evaluation questions. The databases include both quantitative information (e.g., the
dollar amount of each grant), which can be summarized mathematically, as well as narrative
descriptions (e.g., types of activities conducted with GAP funds). The GAP database represents
a sample of tribes, thus, we used this existing sample and augmented it with information on
additional tribes, as described below.
Sample Selection
The GAP database provided the starting point for our sample selection, since this database
provides information specifically on GAP-funded activities conducted by tribes, without which
we could not address the evaluation questions. AIEO initially populated the database from a
random selection of tribes in Regions 1, 2, 4, 5, 6, 7, 8, and 9 that had received GAP funding
prior to 2003 and provided documentation of their programming efforts under GAP. AIEO
chose their sample for the GAP database by sampling at an equal rate within each EPA region to
ensure that the representation of tribes in the database is consistent with the distribution of tribes
in different parts of the country. (In Region 10, only a few tribes were included in the GAP
database, representing only one percent of the tribes in the GICS database in the region that
received GAP grants. There are no federally recognized tribes in Region 3, so none were
included in the GAP database from this region.) We felt it was important to maintain this
geographic representation in the sample selected for this evaluation, since tribes in different parts
of the country vary in size, resources, and other characteristics that may affect tribal
environmental capacity.
2-5
-------
Two tables within the GAP database - the Activity and Position tables - contain information
needed to help answer the evaluation questions pertaining to use of GAP funds. As shown in
Exhibit 2-2, a total of 66 tribes that are included in the GICS database are listed in both of these
tables.6 These 66 tribes represent eight EPA regions and 13 percent of the 500 tribes that have
received GAP grants between 1994 and 2004, according to the GICS database.7 Note that
Regions 2 and 10 are considerably underrepresented in the Activity and Position tables. Were
these regions excluded from the analysis, the records in the GAP database would represent 22
percent of tribes that received GAP grants between 1994 and 2004. In order to provide each
region with approximately equal representation in the evaluation sample, we set the final number
of tribes to be selected from each region as close to 22 percent as possible. For some regions (2,
9, and 10) with fewer tribes in the Activity and Position tables, we randomly selected additional
tribes not included in the GAP database to include in the final sample. We used grant files and
related documents for these additional tribes obtained from the POs to fill in the types of data
that would otherwise be included in the GAP database. For other regions (5, 6, 7, and 8), we
randomly selected from among tribes in the Activity and Position tables to ensure approximately
22 percent of each region's tribes would be included in the sample. Appendix D lists the tribes
included in our sample.
Where records in the GAP database included in our sample turned out to be incomplete or
inadequate for a particular question, we did not analyze the data for that tribe for that question.
The same held true with file reviews: if a file had incomplete data, we analyzed the available
data, rather than select a different record for inclusion in the sample. We characterized any
significant data gaps as part of our analysis.
6 There is one tribe included in the GAP activity and position tables that is not included in the GICS
database. Since information from all three of these sources is important to the analysis, we did not include this tribe
in our sampling pool, and it is not reflected in the totals in Exhibit 2-2. Also note that we did not included intertribal
consortia in our sample (although they are represented in the GICS and GAP databases), since including these
consortia could result in double-counting of tribes that received GAP both independent of and as part of a consortia.
Note that Region 10 tribes eligible for GAP grants, and therefore included in this analysis, do not include
Alaska Native regional or village corporations. The reason for this exclusion is explained in 2000 GAP Guidelines,
Indian Environmental General Assistance Program Guidelines on the Award and Management of General
Assistance Agreements for Indian Tribes, available at http://www.epa.gov/indian/pdfs/gap2000.pdf last accessed
May 2007.
2-6
-------
EXHIBIT 2-2: COUNT OF TRIBES RECEIVING GAP GRANTS
COMPARED TO COUNT OF TRIBES IN THE GAP DATABASE, BY REGION
EPA Region
1
2
4
5
6
7
8
9
10
Total
Total excluding
Regions 2 and 1 0
Number of
Tribes
Receiving GAP
Grants 1994-
2004
-------
from existing databases and that gathered from file reviews.8 A list of the codes used is included
in Appendix E.
We used statistical methods to summarize and analyze the data in the existing databases. Most
of the evaluation questions could be answered by descriptive statistics - including the range,
mean, median, standard deviation, and/or proportion - for key indicators. For example, to
answer the question, "How often are GAP resources accessed?" we considered the percentage of
tribes in our sample that participated in technical and programmatic training opportunities. We
also used statistics to draw inferences about the total population of tribes receiving GAP
grants.9'10 For qualitative data, such as EPA media program categories or staff titles, we
calculated counts and percentages to describe patterns and identify trends.
The one evaluation question that cannot be adequately addressed by descriptive statistics is,
"What is the relative contribution of GAP toward the achieving environmental capacity?" In
order to answer this question, we compared tribes that had GAP grants for a longer period of
time ("early adopters") to tribes that had GAP for a shorter period of time ("late adopters"), to
see whether tribes with more years of GAP funding had more effectively developed
environmental capacity, as measured both by the percent of EPA funding attributable to GAP
and achievement of specific indicators of environmental capacity. For late adopters, we also
compared the percentage of EPA funding derived from sources other than GAP before and after
GAP grants were awarded, to see if access to GAP grants led to an increase in funding
diversification.
Exhibit 2-3 summarizes each type of information that we gathered from the existing databases
and the type of analysis we conducted for each type of data. The primary unit of analysis in this
evaluation is the tribe. We used descriptive statistics to summarize the status of tribes with
regard to evaluation questions 1, 2, and 3a. For the sake of brevity in the table, we refer to
calculating "averages" and "percentages." Where it is noted that we calculated an average, we in
fact calculated the sample range, mean, median, and standard deviation, in order to provide a full
8 The coding categories can be found in Appendix A of the report EPA American Indian Environmental
Office Gap Accountability Tracking System, Developed for AIEO, draft October 6, 2003. The document was
provided by AIEO to lEc.
9 Our statistical calculations assumed that we had obtained a large or approximately random sample of the
population. In cases where these conditions were not met, we did not develop inferences about the whole population
of tribes.
10 We analyzed all tribes across Regions in a single set of descriptive statistics. Where it seemed relevant,
we also calculated separate sets of descriptive statistics for tribes in each Region.
-------
description of the data. We also estimated the population mean, which is reported with the
confidence level and confidence interval. Where it is noted that we calculated a percentage, we
calculated the sample proportion as well as an estimate of the population proportion, which is
reported with the confidence level and confidence interval. The results of this inferential
analysis are included in Appendix F.
2-9
-------
EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
Evaluation Question
Analytic Approach'1
Analytical Method
la. Is GAP being accessed by all
federally recognized tribes?
• Number and percentage of federally recognized tribes
that have received GAP funds between 1994 and 2004.
• Calculate the number and percentage of tribes based on the total
population of GAP grants in the GICS database, rather than a sample.
2a. Are tribal governments using the
resources (technical, fiscal, and
programmatic) provided as a
component of GAP? How often
are GAP resources accessed?
Amount and type of GAP resources that have been
delivered to and accessed by tribes:
• Fiscal resources: GAP funding provided to tribes
• Technical resources: Technical assistance and
media specific trainings
• Programmatic resources: Grants management
training
• Calculate average amount of grant award(s).
• Calculate percentage of tribes that have received technical assistance
and media specific trainings among GAP-funded activities.
• Calculate percentage of tribes that have received programmatic
resources (e.g. grants management training) and calculate the average
number of grants management trainings described among each tribe's
GAP-funded activities.
2b. How are tribes using GAP
Tribal staff and activities funded through GAP
resources >.
Categorize and code activities, and then calculate percent of tribes
conducting each type of activity. Include solid waste implementation
in GAP-funded activities.
2c. To what extent have tribes met
program expectations for grants
management, execution of
administrative functions, and
carrying out proposed activities?
• Timing of grant end date vs. final close out of the grant
- according to regional POs, the shorter the period of
time between grant end date and final closeout, the
more likely that tribes met expectations. Note that
other factors such as EPA project officer turnover, lack
of FTE, or lack of EPA's emphasis on closeouts could
affect this indicator.
• Results of administrative post award monitoring audits.
Calculate the average number of months between grant end data and
final closeout data for those grants with an action code "FC" in the
GICS database, meaning the grant has been closed out.
Calculate the percent of tribes that were audited and had a reportable
condition, material weakness, and/or material non-compliance finding
as a result of the audit.
2-10
-------
EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
Evaluation Question
Analytic Approach'3
Analytical Method
3b. To what extent have tribes
achieved environmental capacity
as suggested by the presence of
indicators of environmental
capacity?
Note, the indicators of
environmental capacity were
selected based on the purpose of
GAP as described in the GAP
program guidelines as well as
input from regional POs early in
the evaluation process and
feedback from tribal
representatives on the draft
evaluation methodology.
Overarching Indicator of Environmental Capability
• The sequence of GAP and non-GAP grant funding
secured by GAP recipients over time
Describe the sequence of GAP funding relative to non-GAP funding
received by tribes; calculate the number and percentage of tribes that
received non-GAP funding before, during, and after receiving GAP
funding.(b)
Legal Capability
• GAP recipients that have developed tribal codes,
standards, and/or enforcement programs to control
pollution
• GAP recipients that have adopted or implemented
tribal codes, standards, and/or enforcement programs
to control pollution
Query the activities field in the database to identify developing codes,
standards, and similar activities in the database, and then calculate the
number and percentage of tribes that mention these among their GAP
activities. To the extent possible, distinguish between codes,
standards, etc. that have been developed versus those that have been
adopted or implemented.
Enforcement Capability
• Presence of tribal environmental staff person(s)
charged with enforcement duties
Calculate the number and percentage of tribes for which records show
enforcement or inspection among the activities funded by GAP, or
that have a position description that mentions enforcement or
inspection.
Technical Capability
• GAP recipients with one or more staff specifically
tasked with managing environmental programs
• Size and composition of tribal environmental staff
• Environmental programs being carried out in different
media annually by tribes
• GAP recipients that have taken environmental training
Calculate the percentage of tribes that have a position of
"Environmental Director" or an equivalent term, or that have a
position with a "Professional" category code.
Calculate average number of staff members listed by each tribe in the
sample, and the percentage of overall positions that are classified as
Professional, Administrative, Technical, Legal, Clerical, or Other.
Query the database to identify activities associated with air, water,
and waste programs, and then calculate the number and percentage of
tribes conducting each type of activity.
Calculate the number and percentage of tribes that list environmental
training as among their activities.
2-11
-------
EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
Evaluation Question
Analytic Approach'3
Analytical Method
Communications Capability
• GAP recipients that have conducted community
education and outreach, based on the grant work plan
• GAP recipients that have executed agreements with
other jurisdictions for management of on- or off-
reservation resources
• Tribal participation in EPA or tribal workgroups and/or
Task Forces
Calculate the number and percent of tribes that list community
education, outreach, or similar terms among their activities.
Calculate the number and percent of tribes that list inter-governmental
agreements among their activities.
Calculate the number and percent of tribes that list participation in
workgroups or task forces among their activities.
3c. What factors contribute to the
achievement of environmental
capacity, and what is the impact
of each factor?
Tribal Funding
• GAP funding amounts
GAP funding consistency over time
Calculate average amount of GAP award.
• Calculate the average number of consecutive GAP grants received per
tribe over time.
3d. What is the relative contribution
of GAP toward achieving
capacity?
Non-GAP grant funding amounts
Sequence of EPA grants received by tribes
• Calculate average amount of funding from non-GAP, EPA grants.
Compare percentage of non-GAP EPA funding between early and late
adopters, and for late adopters, before and after receiving a GAP
award.
• Calculate the percent of tribes that received GAP funding before
securing other funding. Note, we do not have data to assess funding
from outside of EPA, e.g. grants from BIA or tribes' internal funds,
which limits the conclusions that we can draw.
2-12
-------
EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
Evaluation Question
Analytic Approach'3
Analytical Method
5. To what degree does GAP support
EPA's strategic goal of increasing
tribes' ability to build
environmental program capacity?
Summary of GAP goals and objectives
Summary of EPA's strategic goal 5, objective 5.3, and
related targets for building tribal capacity
Summary of progress towards EPA's strategic targets
under objective 5.3, as reported in the Strategic Goals
Reporting System
Summary of findings on the extent to which GAP
appears to support tribes' development of
environmental capacity.
Review GAP goals and objectives contained in EPA's Indian
Environmental General Assistance Program: Guidelines on the
Award and Management of General Assistance Agreements for Indian
Tribes (U.S. EPA, 2000) snA Report to Congress: The Indian
Environmental General Assistance Program (U.S. EPA, 2001)
Compare Goal 5.3, Build Tribal Capacity, and other strategic goals
with relevance to tribes that are included in EPA's 2003-2008
Strategic Plan (U.S. EPA, 2003) with the range of tribal
environmental programs documented in the EPA 5.3S-D.
(a) Information to be collected from GAP database, GICS, Audit Database, and 5.3 S-D, supplemented by File Reviews for Regions 2, 9, and 10.
(b) Securing funding in addition to GAP has been identified as both an indicator of tribal environmental capacity and a factor influencing achievement of environmental capacity
by regional POs. Through interview discussions and focus groups, we will try to understand whether securing other sources of funding is more a cause or effect of success in the
GAP program.
2-13
-------
A potential source of bias in our analysis is the selection of tribes for our sample based on the
availability of comprehensive tribal records. If the tribes with data in the Activity and Position
tables tend to differ systematically from tribes that are not represented in these tables, then our
sample would not be truly representative. This could also be true if tribes in the GAP database
are not represented in the Activity and Position tables because their records are incomplete,
suggesting that perhaps these tribes have less administrative capacity than those that are
included. On the other hand, if the Activity and Position tables are incomplete for reasons
unrelated to tribal characteristics (e.g., the time and resources available to EPA to populate the
database) there might not be a bias in our sample. To assess this potential bias, we compared
tribes in our sample that are not included in the Activity and Position tables to tribes that are
included in those tables to see if they differ significantly for the analysis of activities funded by
GAP. The results of this analysis are included in Appendix G. Overall, the tribal participation
rates for different types of activities funded by GAP seem similar for tribes included in the GAP
database tables and those for which we conducted file reviews. The only substantial difference is
that tribes assessed through the file reviews appear to conduct more communication and baseline
assessment activities than tribes included in the database analysis. We do not believe this
difference is sufficiently large to limit our ability to draw inferences about the whole population
of GAP grants. Due to the scope of the evaluation, it was not possible to conduct further tests on
the GAP database to rule out potential sources of bias. However, we reviewed the description of
the methodology used in preparing the database, and we did not find any notable sources of bias.
We are aware that AIEO is making an effort to improve its data tracking over time, and the GAP
database was developed as a prototype of a more comprehensive database that AIEO is now
compiling.
C. Collect and Analyze Data from Interviews with Regions and Tribes
To complement the analysis of data from the EPA databases, we conducted individual interviews
with EPA staff from eight regions and panel discussions with representatives from an array of
tribes. As discussed above, we used the data from existing databases to develop an initial
analysis of the extent of tribes' use of GAP, how tribes are using GAP resources, the degree to
which tribes are meeting GAP grants management expectations, the extent to which tribes have
achieved environmental capacity as demonstrated by key indicators, and the degree to which
different factors may be associated with key indicators of environmental capacity. However, our
ability to accurately answer the evaluation questions was greatly enhanced by soliciting feedback
on and interpretation of this initial analysis by regional POs and tribal representatives who have
personal experience issuing and implementing GAP grants. We presented POs and tribal
representatives with an abbreviated summary of our analysis and asked for their help in
understanding the data, and identifying any situations where they felt the data may be misleading
or incomplete. In addition, we relied on POs and tribal representatives to provide information for
a number of questions that are not addressed at all in the existing databases. Our approach to
sample selection, data collection, and data analysis are described in detail below.
Sample Selection
We conducted one-hour telephone interviews with EPA POs located in eight regions, and one
cross-regional representative. (The names of specific interviewees are included in Appendix H.)
In order to be sure that the interviewees had sufficient experience with GAP grants to be able to
2-14
-------
answer our questions, we interviewed current POs if they had a minimum of two years of
experience working with GAP grants in the region they represent. If a current PO did not have
this minimum level of experience, we interviewed a former PO for that region, if available.
We also conducted 2-3 hour panel discussions with representatives from a subset of nine tribes
that have received GAP grants in the past five years. The interviews were held in conjunction
with existing gatherings of GAP grantees, in order to facilitate participation of tribes. Each
session was scheduled for a different part of the country, to provide geographic diversity among
the tribes participating. The tribal panelists were selected by the regional EPA staff organizing
each meeting to achieve two objectives: to include different types of tribes and ensure that the
representatives had detailed knowledge of GAP grants and tribal environmental programming.
In addition to tribal representatives on the panel, we invited other tribes attending the gathering
to attend the session if they wished. For each set of interview questions answered by the
panelists, ffic opened the discussion to include comments and questions from the tribal
representatives assembled in the audience. This approach allowed for a greater range of tribal
input on the interview questions for those tribes that wished to provide comments. We also
provided each invited tribal representative with the set of interview questions in advance, so they
could solicit feedback from other tribes in their region if they wished. The names of tribes
participating in the panel discussions are included in Appendix I.11
We recognize that relying on EPA to select tribal representatives from among attendees at an
EPA-sponsored GAP event did not provide a sample representative of the population of GAP
grantees; we assumed that the tribes that participate in such an event are more likely to be active
partners in GAP and other EPA programs than tribes that eschew them. Tribes that actively
collaborate with EPA and other tribes on GAP projects may have different experiences and
perspectives than those that do not. Despite this likely source of bias from the group interviews,
we believe this approach was the most feasible way to gather information from tribal
representatives. Moreover, while recognizing the limitations of this proposed sample of tribal
representatives, we believe that this group was able to provide insights and illustrative examples
about how GAP grants work for tribes that are especially engaged with EPA on the GAP
program.
Data Collection and Analysis
The interviews and panel discussions address aspects of the evaluation questions that are not
covered or cannot be fully addressed in our analysis of the EPA databases (see Exhibits 2-4 and
2-5). In addition, these forums provided POs and tribal representatives with the opportunity to
offer their interpretations of and feedback on the results of the preliminary analysis we conducted
with information from the EPA databases. Interviews with POs included structured and open-
ended questions; Appendix B includes the interview guide for POs. Appendix C includes the
tribal discussion guide.
11 Due to inclement weather, we were unable to conduct a session with tribal representatives in the Eastern
region of the U.S. We instead conducted individual telephone interviews with each of the three panelists selected
for that session.
2-15
-------
EXHIBIT 2-4:
INFORMATION GATHERED IN INTERVIEWS WITH REGIONAL GAP PROJECT OFFICERS
Evaluation Question
Ib. Why are some tribes not involved
in GAP?
Ic. Are there tribes that received
GAP grants at one time but which
no longer receive GAP grants? If
so, why?
2c. To what extent have tribes met
program expectations for grants
management, execution of
administrative functions, and
carrying out proposed activities?
3 a. What indicators of tribal
environmental capacity exist?
3b. To what extent have tribes
achieved environmental capacity
as suggested by the presence of
these indicators?
3c. What factors contribute to the
achievement of environmental
capacity, and what is the impact
of each factor?
5. To what degree does GAP support
EPA's strategic goal of increasing
tribes' ability to build
environmental program capacity?
Information Sought from Regional Project Officers to Help Answer
Evaluation Questions
• Perceptions as to why tribes may not seek GAP funding;
perceptions on ability of tribes to initiate environmental programs
without it
• Perceptions as to why tribes may have dropped off the GAP grant
rolls
• Regional POs' perceptions about the quality, timeliness, and
completeness of work plans and progress reports received
Administrative Capability:
• How long has the tribal staff person tasked with managing
environmental programs (e.g. Environmental Director) been in that
position?
• Major findings on A133 audits
• Results of on-site grants management review
• Degree of tribal council support for environmental programs
• Does tribe have clear environmental priorities?
• Rate of change in tribal governments - how often do Council
members change?
• Turnover rate of tribal Environmental Director and/or staff
• Qualifications of tribal Environmental Director and/or staff
• Degree of information sharing among tribes
• Degree to which tribes request information from EPA
• Structure of EPA regional office - specifically, at what level the
tribal office located (in the administrators office or elsewhere)
• Experience and longevity of POs
• How do GAP's goals currently align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• How do GAP's objectives align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• How do tribal GAP activities align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
2-16
-------
EXHIBIT 2-5:
INFORMATION GATHERED IN PANEL DISCUSSIONS WITH TRIBAL REPRESENTATIVES
Evaluation Question
Information Sought from Tribal Representatives to Help Answer
Evaluation Questions
Ib. Why are some tribes not involved
in GAP?
Perceptions as to why tribes may not seek GAP funding;
perceptions on ability of tribes to initiate environmental programs
without GAP
2d. How does participation in GAP
increase understanding of how to
develop a tribal environmental
program?
Serf reported increase in knowledge and understanding about the
necessary steps in developing a tribal environmental program
Serf-reported increase in skills needed to develop tribal
environmental programs
Serf-reported change in awareness and commitment to
environmental programs in tribes
3 a. What indicators of tribal
environmental capacity exist?
3b. To what extent have tribes
achieved environmental capacity
as suggested by the presence of
these indicators?
• Tribes' perceptions of key indications of environmental capacity.
Communications Capability:
• Extent of tribal environmental staffs communication of with tribal
Council
Other capabilities (legal, enforcement, technical, administrative) not
fully addressed in the analysis of EPA databases.
3c. What factors contribute to the
achievement of environmental
capacity, and what is the impact
of each factor?
Degree of tribal Councils' support for environmental programs
Do tribes have clear environmental priorities?
Rates of change in tribal governments - how often do Council
members change?
Turnover rates of tribal Environmental Director and/or staff
Qualifications of tribal Environmental Director and/or staff
Degree of information sharing among tribes
Availability of non-grant tribal funding
3d. What is the relative contribution
of GAP toward achieving
capacity?
Sequence of grants received by tribes (i.e., do tribes receive GAP
grants before other grants? If so, this would suggest that GAP
may contribute to ability to receive other grants)
Availability of non-grant tribal funding
4. Is the GAP providing adequate
outputs to achieve tribal goals and
priorities?
Tribes' perceptions about whether GAP is providing the type and
amount of resources that they need to meet their environmental
goals and priorities
Are there additional resources that tribes feel they would need to
have in order to address their goals and priorities?
Is environmental capacity building a priority for tribes receiving
GAP grants?
The first step in analyzing the data collected through the interviews and panel discussions
involved broadly categorizing and summarizing responses and using them to qualify the results
of our quantitative analysis. We also captured comments that, in our judgment, synthesize the
views expressed by interviewees and panel discussion participants. In this way, our analyses and
conclusions reflected the insights and perspectives provided by POs and tribal representatives.
2-17
-------
D. Prepare Final Evaluation Report
This report constitutes the final evaluation report, which has been prepared in accordance with
EPA guidelines.
III. QUALITY ASSURANCE PROCEDURES
Two parts of this evaluation required a quality assurance review. We first needed to ensure that
the way we coded data during our review of tribal GAP files was consistent with the coding of
data already in the GAP database. To meet this requirement, we used the same coding categories
used by EPA to create the GAP database, and we reviewed these categories with AIEO staff to
be sure we understood how to interpret them. We created a standardized code sheet to use for all
file reviews, and coordinated between lEc staff conducing the reviews to ensure coding
consistency (Appendix E). Finally, in cases where we had questions about interpretation, we
referred the questions to AIEO staff.
The second part of this evaluation requiring a quality assurance review is the analysis itself (e.g.,
calculation of descriptive statistics, populations proportions, and population means, as well as
characterization of qualitative information gathered in the interviews and panel discussions). All
quantitative computations and analyses have been reviewed by at least one manager to ensure
accuracy. With regard to the qualitative information from interviews and panel discussions, lEc
used interview guides to ensure consistency in the way we asked questions during discussions
with EPA staff and tribal representatives. lEc also had a staff person present to type notes during
each interview (this recorder was usually not the same person who conducted the interviews). In
most cases the notes recorded were comparable to a transcript (although some information was
summarized to keep pace with the flow of the conversation). We compiled the interview notes
into summary documents, and grouped together responses to each interview question to facilitate
their characterization. Where interviews produced quantitative information (e.g., ranked
responses to questions posed along a Leichart scale), we analyzed these responses
mathematically. We made the draft summary of our results available to POs and tribal
representatives that we interviewed to allow them to correct any inaccuracies in our
interpretation of their comments, however no corrections were submitted.
IV. STRENGTHS AND WEAKNESSES OF THE EVALUATION DESIGN
The strengths of this analysis are that it answers the specific evaluation questions posed by
AIEO, and it draws on both quantitative and qualitative data to answer them. The methodology
underwent extensive peer review (with comments provided by EPA staff, tribal representatives,
and academic reviewers) and we modified the evaluation design in light of the comments
received. Academic reviewers noted that while the evaluation would not meet the requirements
of a formal academic evaluation, the analytical rigor is appropriate for the budget and time
available to conduct it.
The final evaluation methodology varies only slightly from the proposed methodology that
underwent peer review. The most significant change is that the original methodology proposed
conducting a regression analysis to assess the correlation between index of environmental
2-18
-------
capacity and key indicators, such as the amount of GAP funding. In retrospect, pursuing this
analysis would have been ill advised. Based on what we heard in interviews, we believe there
are many confounding factors (e.g., tribal population and land base, degree of cohesiveness of
tribal land holdings, and tribal socio-economic status) that could influence a tribe's achievement
of environmental capacity, in addition to the indicators that we can measure, such as the amount
of GAP funding. Accounting for all these factors was beyond the scope of this investigation.
We opted instead to compare early vs. late adopters of GAP, to see if tribes' access to GAP for a
longer period increased the degree to which they had achieved capacity. If true, this would
suggest that access to GAP contributes to tribes' ability to develop environmental capacity.
A key limit of this methodology, both in its original and final form, is that it does not prove that
GAP caused the changes in tribal environmental capacity described in Chapter 3. However,
qualitative interviews suggest that GAP has indeed been an essential resource in helping tribes
build their environmental programs. Another limitation of our analysis is that since the tribes
included in the panel discussions are not necessarily representative of all GAP grantees, we
cannot rely on information from these sessions to make inferences about GAP grantees as a
whole. However, given the time and budget limitations of the analysis, and U.S. Information
Collection Request (ICR) restrictions, we were not able to interview a larger sample of tribes that
would be representative of GAP grantees as a whole.
Although this evaluation design significantly limits the types of conclusions and our ability to
generalize them to the larger population of GAP grantees, in our judgment, it is the best
methodology available given the characteristics of the program under evaluation. Moreover, this
evaluation design is well adapted for understanding how and why a program is effective, which
can provide useful information for program management.
2-19
-------
CHAPTER 3: GAP EVALUATION FINDINGS
This chapter presents the findings from EC'S evaluation of the General Assistance Program
(GAP). The findings are based on a review of records for a sample of 111 tribes awarded GAP
grants, stored in four federal databases: the EPA GAP Accountability Tracking System and
Strategic Goals Reporting System, both maintained by EPA's American Indian Environmental
Office (AIEO); the Grants Information and Control System (GICS) maintained by EPA's Office
of Administration and Resources Management; and the Audit Database maintained by the U.S.
Census Bureau. Where needed, records in the databases were supplemented by file reviews of
GAP progress reports maintained by EPA regions. Statistics presented in the narrative are based
on the sample of 111 tribes. Appendix F includes inferences based on this sample about
characteristics for the entire population of GAP grantees. Information obtained from these
databases, with the exception of the GAP database, span the years 1994-2004, unless otherwise
noted. Due to the limitations of the version of the GAP database used for this evaluation, tribal
activity and position data only cover the period October 2000 through September 2004.l The
activity table in the database contains records of discrete activities conducted by tribes during
this period but does not track the overall effort and time expended by tribes to carry out each
activity.2
In addition to information from the databases and file reviews, these findings are informed by
interviews with EPA project officers (POs), and a series of panel discussions and interviews with
tribal representatives. Information from these interviews and discussions is drawn from the
experience of the interviewee, which encompasses recent years and may date back to 2000 or
earlier.
The findings are organized around the five evaluation questions, as follows:
I. Is the GAP being accessed by all federally recognized Tribes? If not, why are some
Tribes not involved in GAP? Are there Tribes that received GAP grants at one time but
which no longer receive GAP grants? If so, why?
II. Are Tribal governments using the resources (technical, fiscal, and programmatic)
provided as a component of GAP?
A. How often are they accessed?
B. How are Tribes using these resources?
C. To what extent have Tribes met program expectations for grants management,
execution of administrative functions, and carrying out proposed activities?
1 This time period corresponds with GAP work plan fiscal years 2000, 2001, 2002, and 2003. Unlike a
federal fiscal year, which is named for the year ending in September, a GAP work plan year is named for the year
beginning in October. For example, GAP work plan year 2000 began on October 1, 2000 and ended on September
30, 2001. Thus, the time period for which GAP data are available extends from October 2000 through September
2004.
2 This data limitation may create the impression that tribes which reported fewer activities in a given
program area were less active in building capacity than tribes which reported more activities in that area, however,
certain capacity building activities are ongoing, and thus may represent a substantial effort for the tribe even if they
are only counted in the database once for any given reporting period.
3-1
-------
D. How does participation in GAP increase understanding of the process required to
develop a Tribal environmental program?
III. What indicators of Tribal environmental capacity exist?
A. To what extent have Tribes achieved environmental capacity as suggested by the
presence of these indicators?
B. What factors contribute to the achievement of environmental capacity, and what is
the impact of these factors?
C. What is the relative contribution of GAP toward achieving capacity?
IV. Is the GAP process providing adequate outputs to achieve Tribal goals and priorities?
V. To what degree does GAP support EPA's strategic goal of increasing Tribes' ability to
build environmental program capacity?
Based on these findings, the next chapter discusses recommendations and conclusions of the
evaluation.
I. FEDERALLY RECOGNIZED TRIBES' ACCESS TO GAP
GAP is utilized by a significant majority of federally recognized tribes. From 1994-2004, 500 of
the 5613 federally recognized tribal governments in the United States received at least one GAP
grant. This means that approximately 89 percent of tribes received a GAP grant during this time
period.4
lEc interviewed EPA POs from eight regions, who work directly with tribes to administer their
GAP grants. We asked POs whether they knew of any federally recognized tribes in their
regions that had not received GAP grants since 1994, and if so, why these tribes did not receive
GAP funding. The POs indicated that only a few tribes in their regions had not received GAP
grants. Tribes in four regions had adopted a policy of not accepting federal grant money. In
addition, POs in two regions shared their perspective that some tribes have not applied for GAP
funding because they are too small and do understand how to use GAP, or because they do not
have the basic infrastructure to apply for the funding. POs in two other regions explained that
tribal performance issues, such as non-reporting or fiscal mismanagement, had made some tribes
ineligible for EPA funds. Lastly, in one region, there was a time when the regional office did not
have sufficient staff to process all incoming GAP applications, and the region ended up denying
grants to tribes whose proposals it deemed insubstantial.
We also asked POs whether any tribes in their regions had at one time received GAP grants, but
no longer receive GAP funding, and if so, what caused them to drop off the GAP grant rolls.
3 Bureau of Indian Affairs (BIA). 2007. Bureau of Indian Affairs. Washington, D.C., BIA. Available
from http://www.doi.gov/bureau-indian-affairs.html. Accessed April 2007.
4 These results were obtained through an analysis of EPA's Grants Information and Control System (GICS)
database records between 1994 and 2004, and are based on an analysis of GAP funding provided to all 561 federally
recognized tribes.
3-2
-------
Many POs responded that one or more tribes in their region had received a GAP grant between
1994 and 2004, but had since dropped off the GAP grant rolls. At least two tribes in one region
had their own financial resources from successful casinos and decided not to apply for GAP
funding in order to free up that funding for other tribes. In another region, staff turnover left
some small tribes without staff to manage their GAP grants. In a few cases, performance issues
or fiscal mismanagement left tribes temporarily ineligible for GAP funding. Once tribes resolved
these issues they became eligible for GAP grants again. Finally, one tribe lost its federal
recognition, and thus was no longer eligible for GAP funding.
We then asked POs if the lack of GAP grants hindered the development of environmental
programs for tribes that did not receive GAP grants. The POs explained that except for those
tribes with significant financial resources of their own, the lack of GAP funding did hinder the
development of tribal environmental programs. Tribes without GAP funding or substantial
revenue streams of their own have not been able to devote sufficient effort to their environmental
programs. These tribes missed networking opportunities and had to rely on EPA for permitting
and enforcement. One PO explained, however, that the temporary loss of GAP grants due to
fiscal mismanagement ultimately benefited certain tribes, because it led them to reevaluate their
efforts and make administrative improvements to access GAP in the future.
II. TRIBAL UTILIZATION OF RESOURCES PROVIDED BY GAP
GAP provides funding as well as technical and programmatic assistance to tribes. Technical
assistance may include: EPA-sponsored training, linking tribal staff with EPA media program
contacts, EPA review of tribal proposals for establishing programmatic capability, and site visits.
For the purpose of this evaluation, we have defined programmatic assistance as any EPA-
sponsored training or outreach directed toward improving tribal administration of GAP grants.
One objective of the evaluation is to determine whether tribes are accessing the array of
resources (fiscal, technical, and programmatic) provided as a component of GAP, and how often
they access these resources. The evaluation also seeks to assess how tribes' participation in GAP
and their use of GAP resources influences the way tribes approach developing and administering
their environmental programs.
A. Tribes Access of GAP Resources
GAP funds received by tribes constitute the most critical resource provided by GAP. The 111
tribes in the sample received a total of 754 GAP grants between 1994 and 2004. The funding
amounts awarded for these grants ranged from $1,918 to $423,000; the mean GAP award for the
sample of tribes was $102,472. These funds have supported various components of a core tribal
environmental program, including salaries for environmental program staff, access to training
and networking opportunities, equipment, outreach and education, and contract services. GAP
funds have remained important to tribal environmental programming efforts over time, even after
tribes secured additional funds from other EPA and non-EPA sources. Exhibit 3-1 illustrates the
number of GAP grants received by tribes. Just over three quarters of the tribes (79 percent)
received nine or fewer GAP grants during this 10-year period. The maximum number of GAP
grants received by a tribe was 19; the average across tribes was seven.
3-3
-------
Exhibit 3-1: Number of GAP Grants Received by Tribes, 1994-2004 (n=lll)
Qno/, _
JU
C" 9Wn -
&
£
P&
H
-------
and outreach available to tribes. Some regions hold an annual GAP conference at which GAP
administrative training may or may not be provided, while other regions routinely offer GAP-
specific workshops. One region indicated that it has had more success with assisting tribes with
the fiscal administration of GAP on a one-on-one basis than through formal training.
B. Tribes' Use of GAP Resources
Tribes use GAP funds and technical and programmatic resources primarily to establish and
maintain a tribal environmental presence in Indian country, which many tribes define as having a
qualified staff person available on the reservation to respond to environmental issues of concern
to their tribal council and members. In addition, tribes use GAP resources to participate in a
variety of activities that help build their environmental capacity and expand their environmental
presence. We examined the types of activities conducted by the 96 tribes in our sample for
which we were able to obtain activity data from the either the GAP database or file reviews.
Exhibit 3-2 presents the percentage of tribes that participated in different categories of activities.5
Nearly all tribes (98 percent) participated in activities related to the general management and
administration of their environmental programs. A majority of tribes also participated in land
activities (84 percent), water activities (73 percent), and grant writing activities (65 percent). A
smaller proportion of tribes conducted air activities and special emphasis activities.
5 Activities listed in the GAP database are organized into categories that correspond to six main program
areas. Within each category, activities are grouped into more narrow subcategories. For example, land activities
may include activities associated with asbestos, emergency response, Superfund, hazardous waste, lead, pesticides,
solid waste and recycling, or underground storage tanks (UST). Activities conducted within each subcategory may
be further classified by type (in Exhibit 3-3), such as general program development, staffing, communication,
baseline assessment, development of monitoring capacity, development of codes, ordinances, or standards,
developing permitting/licensing authority, development of QAPPs, grant administration, and database development.
For a complete list of categories and subcategories, see Appendix E.
3-5
-------
Exhibit 3-2: Activities funded by GAP, Organized by Major Category, 2000-2004 (n = 96)
g, 100% -,
&
s
_ 80%
fj
«
a
.=
& ^»0%
•e
u
^
•C
SS 4no/,
s
.Q
L^ 90%
o
1
V
PH 0%
General Land Activities Water Activities Grant Writing Air Activities Special Emphasis
Management and Activities
Administration Category of Activity
Exhibit 3-3 shows tribal participation in activities further classified by activity type. More than
80 percent of tribes participated in program development or establishment, staffing, and
communication activities. Approximately two-thirds of tribes engaged in baseline assessment
and grant writing activities. Considerably fewer tribes conducted activities associated with
media-specific programs, such as the development of Quality Assurance Project Plans (QAPPs)
and monitoring capacity; the development of legal tools such as codes, ordinances, standards,
and permitting authority; and the administration of grants received in support of these programs.
Only two tribes participated in database development activities.
3-6
-------
Exhibit 3-3: Activities funded by GAP, Organized by Activity Type, 2000-2004 (n = 96)
100% -,
13
u
-S 60% -
w
^g
^
•-C
u
£
*£ 9n%
H
=g 0% J
1
n n n n
II II II II 1 1
1 |5l 1 1 | 1 ^ S * °si^c^cg a gg
£ £s| | i i £ is |i| |& |l.^ i it
ill I j a 1 ft ill P 111 1 ^
S | Q§Q^° ig
Activity Type
We also asked tribes directly for details about the activities they conduct with GAP funding. The
activities tribes mentioned are consistent with the categories and activity types described above,
although many also reflect the specific interests and environmental priorities of individual tribes.
For example, one tribe performed an assessment of medicinal plant species on the reservation.
Another has conducted environmental education integrated with tribal culture and language.
Noting the subsistence diet of its members, another tribe used GAP funds to address water
quality and increased mortality rates potentially linked to the consumption of contaminated fish.
A number of tribes have also used GAP to participate in training and planning activities for
emergency response programs that include natural disaster assistance, such as forest fire
prevention.
The range of activities conducted by tribes suggests both the depth and breadth of capacity-
building within and across program areas. One example is water quality programs. Tribes
recalled using GAP funds for wetlands surveys, riparian zone protection activities, surface water
and well monitoring, and obtaining staff certifications for drinking water and wastewater
treatment systems. A few tribes mentioned using GAP to hire contractors to conduct specialized
activities including hazardous waste cleanups, water sampling and analysis, program planning
for solid waste and emergency management, and electronics recycling.
Under the general program management and administration category, tribes have engaged in
activities to further establish their legal and enforcement capability. These activities include the
3-7
-------
development of ordinances for solid waste and recycling, open burning, zoning, and underground
storage tanks, as well as the establishment of water quality standards. GAP has enabled at least
one tribe to issue wetlands permits and conduct its own inspections to ensure compliance with
tribal regulations. For tribes that do not have permitting or enforcement authority for programs
such as UST, Underground Injection Control (UIC), or the National Permit Discharge
Elimination System (NPDES), GAP provides funds for tribal environmental staff to accompany
federal inspectors and gain knowledge and expertise.
C. Expectations for Grants Management, Execution of Administrative Functions, and
Carrying Out Proposed Activities
The award of GAP grants brings with it EPA's expectation that tribes will fulfill the
requirements of GAP for demonstrating accountability in the utilization of funds as well as for
grants management and performance reporting, detailed in the 2000 GAP Guidelines.6 The
Guidelines also specify requirements for the preparation of work plans, financial reports, and
performance reports that demonstrate progress toward the achievement of deliverables stated in
the grant work plans.
Based on our interviews with regional POs, we found that, overall, tribes are meeting regional
expectations for grants management, the execution of administrative functions, and carrying out
proposed activities. When asked to rate tribal fulfillment of regional expectations along a five-
point Lei chart scale, POs said that tribes almost always or often meet their region's expectations
for grants management and the execution of administrative functions, and almost always
complete the activities proposed in their work plans. According to POs, tribes continue to
improve the timeliness of their GAP work plans and progress reports. Currently, most tribes in a
majority of regions are submitting their work plans and progress reports on time. In one region,
however, only 20 percent of tribes are submitting timely work plans and progress reports. POs
attribute significant improvements in the timeliness of tribal submittals in part to efforts
undertaken by regional staff to improve their ability to track submittals and to raise tribes'
understanding of reporting expectations. They also acknowledge that timeliness tends to
improve as tribes gain experience preparing work plans and progress reports, although setbacks
can occur when there is high turnover among tribal environmental directors. Delays in work
plan submittal may also occur when EPA does not announce award amounts until late in the
year. Because GAP has traditionally received its appropriation toward the end of the second
quarter, the time period tribes have for preparing and submitting work plans is often extended.
One region addressed this problem by establishing a timetable for tribes that specifies when their
work plans would be due and when regional staff would respond to them. This region
emphasized to tribes that their GAP awards could be jeopardized if they did not submit their
work plan by the deadline.
The quality and completeness of GAP work plans and progress reports has improved
concurrently with submission timeliness. POs indicated that while most tribes submit quality
work plans initially, regions must still enter into subsequent negotiations with some tribes to
improve both the quality and completeness of their work plans. One PO said that only 10
percent of the tribes in the region submit initial work plans of sufficient quality; however,
6 Available online at www.epa.gov/indian/pdfs/gap2000.pdf. Last accessed May 2007.
3-8
-------
approximately 90 percent show quality improvements in subsequent submittals. In addition,
work plans and progress reports may fulfill the basic criteria for completeness but still lack
adequate detail. Work plan and report templates have contributed to this problem in one region.
A few POs view the use of the standard work plan format recently implemented nationally by
EPA as having positively influenced quality, but others report that the new format has caused
some declines in quality due to the need for time to become familiar with the new format, as well
as tribal concern over changing expectations. Negotiations conducted with tribes during the
development of earlier standard formats in two regions have facilitated tribal acceptance of the
new formats, and have helped ease the transition to using them. As with timeliness, the quality
and completeness of work plans and progress reports has continued to improve with the
increased experience and stability of tribal environmental staff.
To further assess tribal execution of administrative functions, we examined the length of time
tribes took to close out their GAP grants after the date the grants ended. Our assumption, based
on prior interviews with EPA staff, is that tribes that close out their GAP grants relatively
quickly after the end of the grant period have fewer administrative requirements to address at the
end of the grant, and demonstrate greater capability to execute administrative functions. Of the
754 GAP grants awarded to the 111 tribes in our sample from 1994 through 2004, EPA's GICS
database indicates that tribes closed out 175 grants (23 percent) during this period. The amount
of time that passed between the end date and final closeout date for each grant ranged from
approximately minus 3 months (i.e., the grant was closed out prior to the grant's end date) to 51
months. On average, it took tribes about a year (13 months) to close out their GAP grants.
We also examined the results of A-133 audits conducted for the 111 tribes in the sample to
assess tribes' ability to execute administrative functions.7 As illustrated in Exhibit 3-4,
approximately 25 percent of the 111 tribes in our sample had been audited during the period
1997-2004. Of these 27 tribes, the audits for 24 resulted in at least one reportable condition,
material weakness, or material noncompliance outcome.8 Exhibit 3-5 presents the proportion of
tribes with any of these three major audit findings organized by region. The greatest number of
tribes audited was in Region 10, followed by Regions 9, 8, and 6. Audits that resulted in a
Reportable Condition also resulted in a Material Weakness. All the tribes audited in Regions 7,
8, and 9 had audit findings in these categories. Fewer tribes had audits that resulted in Material
Non-Compliance. Note that because participation in an A-133 audit is required only when a
tribe's total annual expenditures of federal funds exceed a high threshold, most of the tribes in
our sample would not likely have to undergo such an audit. As a result, the tribes in our sample
that were audited and cited with a reportable condition, material weakness, or a material non-
compliance may not be representative of the ability of the tribes not audited to execute
administrative functions pertaining to GAP grants. It may also be the case that since A-133
7 In accordance with Office of Management and Budget (OMB) Circular No. A-133 (68 FR 38401), non-
federal entities that expended $300,000 (or $500,000 for fiscal years ending after December 31, 2003) or more in a
year of federal awards, such as grants, are required to have a single or program-specific audit conducted on an
annual basis. Available from: http://www.whitehouse.gov/omb/circulars/al33/al33.html. Accessed April 2007.
8 Reportable conditions, as defined in the OMB Circular No. A-133 (68 FR 38401), constitute deficiencies
in internal control over major programs. Reportable conditions may be individually or cumulatively material
weaknesses. Material noncompliance means noncompliance with the "provisions of laws, regulations, contracts, or
grant agreements related to a major program."
3-9
-------
audits include a review of all federal expenditures for a tribe, the findings recorded in the Audit
Database for tribes in our sample may not be related to tribal fiscal performance under GAP.
Exhibit 3-4: A-133 Audit Findings, 1997-2004 (n = 111)
Tribes Audited but no
Problems Identified
3%
Tribes Not Audited
76%
Tribes Reporting a Material
Weakness, Reportable Condition, or
Material Non-Compliance
22%
Exhibit 3-5: A-133 Audit Findings by Region, 1997-2004 (n = 111)
EPA
Region
1
2
4
5
6
7
8
9
10
Number of
Tribes in
Sample
2
1
1
7
14
2
6
30
48
Percent of
Tribes in the
Sample that
were Audited
0%
0%
0%
29%
29%
50%
83%
20%
19%
Percent of Tribes
Audited with a
Reportable
Condition
-
-
-
50%
75%
100%
100%
100%
78%
Percent of Tribes
Audited with a
Material
Weakness
-
-
-
50%
75%
100%
100%
100%
78%
Percent of Tribes
Audited with a
Material Non-
compliance
-
-
-
50%
50%
0%
60%
50%
33%
Percent of Tribes
Audited with no
problems
reported
-
-
-
50%
0%
0%
0%
0%
22%
3-10
-------
Regional POs identified only a few tribes that had received a major finding on an A133 audit.
POs said these audit findings mainly cited problems with tribes' practices in tracking and
documenting expenditures, such as incomplete time and attendance records for environmental
staff on the GAP payroll. In addition, audits revealed some concerns about tribes' ability to
track the non-Federal share of their funding and the use of GAP funds when those funds were
combined with other grants (e.g., when GAP funds and other grants were pooled to purchase
equipment). At the regional level, the grants administration office often assumes responsibility
for reviewing tribes' management of GAP grants while POs focus their reviews of grantee
progress on activities funded by GAP. For regional on-site GAP grant management reviews,
POs report that a majority of tribes receive positive reviews, but in the small number of instances
where problems are identified, the regions work directly with the tribes to correct them. In one
region where GAP grants are incorporated into tribal PPGs, all regional program staff involved
in the PPG, including senior managers, participate in the reviews and the development of follow-
up action items with the tribes.
D. How Participation in GAP Influences Understanding of the Process Required to
Develop a Tribal Environmental Program
In addition to assessing the direct resource outputs provided by GAP, the evaluation seeks to
discern how tribal participation in GAP and utilization of GAP resources has influenced, 1)
tribes' understanding of the process required to develop an environmental program, and, 2) the
way tribes approach the various administrative and programmatic functions associated with the
development process. Tribal representatives emphasize that instead of changing tribal
understanding of how to develop an environmental program, GAP facilitates tribes' ability to
develop a program that is responsive to each tribe's unique environmental conditions and
priorities. GAP resources enable tribes to establish an environmental presence, which in turn
provides the foundation upon which each tribe can build an environmental program tailored to
meet its needs. From EPA's perspective, however, GAP may influence and clarify tribal
priorities as tribal environmental staff acquire training, learn about specific environmental
conditions on tribal lands, and become more aware of concrete program opportunities through
their interactions with EPA regional tribal and media program contacts.
Although tribes' maintain the view that GAP has not directly changed their environmental
priorities or their understanding of what is needed to achieve them, they acknowledge that on a
practical level, GAP has helped them develop a planning perspective, specifically, establishing
and refining the annual goals in their work plans to support their priorities. One tribe noted that
the progress made toward the goals outlined in its GAP work plan in a given year influences the
priorities it establishes for the next year. As another tribe observed, however, the goals EPA
establishes for GAP in a given year might target an emerging EPA priority, such as children's
health, which may not correspond with tribal environmental priorities.9 Under these
circumstances, tribes may enter into negotiations with EPA to resolve differences but end up
modifying their approach to align with the GAP goals in order to secure needed funding.
9 The divergence between tribal and EPA goals may be more of an issue for tribes seeking media-specific
funds from EPA, rather than GAP funding, since GAP funds may be used to address tribal priorities.
3-11
-------
With regard to the influence of GAP on the administration of tribal environmental programs,
tribes stress that the continuity of GAP funding provides them with the means to hire, train, and
keep qualified, environmental professionals. Retaining a qualified, knowledgeable staff person
allows tribes to effectively manage their GAP funds, carry out activities proposed in their work
plans, and seek opportunities to further expand and diversify their environmental programming
in response to tribal priorities. GAP facilitates tribes' awareness of the broader environmental
resources and infrastructure available to assist them. Through contact with GAP POs and other
personnel in each region's tribal office, tribal environmental staff learn about the media-specific
programs within EPA and external to EPA that can provide funding and technical assistance to
complement GAP activities.
GAP also facilitates the creation of partnerships among tribes, and between tribes, and outside
agencies and organizations, that can give tribes a voice in addressing environmental concerns on
and off the reservation. Most tribal representatives assert that in addition to promoting external
communication, GAP has influenced how tribes communicate internally to tribal members and
council about environmental concerns. Tribes use GAP funds to develop environmental and
cultural education programs for tribal youth and prepare outreach materials, such as monthly
newsletters and radio announcements. The education and outreach conducted by tribal
environmental staff helps to build community recognition for environmental activities on the
reservation and raises the credibility of the tribal environmental program and its
accomplishments. For example, one tribal representative mentioned that GAP assistance helped
the tribe gain recognition in the community for its environmental leadership. The environmental
director and staff formed committees to raise awareness among tribal members and involve them
in establishing environmental priorities on the reservation.
III. INDICATORS OF TRIBAL ENVIRONMENTAL CAPACITY
An important objective of the evaluation is to determine how effective GAP has been in building
tribal environmental capacity among tribes receiving GAP funds. For the purpose of this
evaluation, we are using a definition based on EPA's 2000 GAP Guidelines: "environmental
capacity" means that a tribe has established the administrative, legal, technical, and enforcement
capability necessary to develop and implement a tribal environmental program, as well as the
communications capability to work with federal, state, local, tribal, and other environmental
officials. This section attempts to answer this question in detail by: 1) identifying the indicators
of environmental capacity, as defined by GAP, tribes, and regional POs; 2) assessing the extent
to which tribes have "achieved" environmental capacity given the presence of these indicators;
3) identifying other factors that may influence the development of tribal environmental
programs; and 4) determining how GAP has contributed to tribal environmental capacity relative
to these other factors.
A. Tribal and Regional Perspectives on Indicators of Environmental Capacity
Not withstanding the GAP definition of environmental capacity and the five indicators
incorporated within, we asked tribal representatives and regional POs to provide their
perspective on what it means for a tribe to have attained environmental capacity. Tribes tend to
define environmental capacity more generally than GAP and equate it with a variety of key
3-12
-------
indicators, as shown in Exhibit 3-6. Many of the tribal definitions of capacity and indicators fall
within the GAP category of technical capability, e.g., the hiring and training of qualified
environmental professionals and the expansion of tribal environmental programming efforts to
include media-specific components. A few tribes linked environmental capacity with legal or
enforcement capability. One tribal representative equated environmental capacity with three
broad indicators: 1) the ability of tribes to recognize environmental problems, 2) the ability of
tribes to address the problem and take immediate action to protect health and the environment,
and, 3) the ability (i.e., knowledge and capability) of tribes to prevent the problem from
happening again. By that person's estimation, 90 percent of tribes have met the first indicator,
35-50 percent have met the second, and only 10 percent have met the third.
Exhibit 3-6: Tribal Definitions of Environmental Capacity and Key Indicators
Definitions
Key Indicators
Having an environmental presence - the ability to
respond quickly to environmental issues and
implement environmental initiatives in response to
tribal concerns as they arise.
The ability to attract, hire, train, and sustain
qualified environmental staff to regulate tribal
environmental programs across all media areas.
The ability to build tribal awareness and
understanding of environmental conditions on tribal
lands.
The ability to carry out environmental protection via
the development of codes and ordinances.
The ability to enforce environmental permits and
conduct compliance actions on reservation lands.
Progression over time from a single multi-media
program with one staff person to multiple program
areas with support staff.
Trained staff and growth of programs with media
grants.
The degree of institutionalization of environmental
programs and the reliance of the tribe on the
technical expertise of its environmental director and
staff.
A functioning environmental program with all five
individual components - administrative, legal,
technical, enforcement, and communications.
Establishing a water quality system and standards,
receiving Treatment As a State (TAS), and being
able to run the program efficiently for the
betterment of the tribe.
Convergence of the tribe's environmental program
with its economic development strategy.
Protection of tribes' sovereign right to protect water,
land, air.
Regional POs identified many of the same key indicators of capacity as tribes, such as tribes'
ability to establish an environmental presence; retain qualified, knowledgeable staff over the
long-term; and diversify their environmental programming. These POs see the longevity and
expertise of staff as the primary means by which tribes achieve success. Seasoned staff are adept
at finding solutions to environmental problems, they participate more fully in regional
networking opportunities (e.g., RTOC calls), they provide mentoring to other tribes, and they are
likely to have comprehensive programs funded by multiple grants. POs also pointed to the
development of legal and enforcement capability as an important indicator; one PO
acknowledged, however, that while many tribes have developed codes and ordinances, few have
the capability to enforce them. Another PO cited direct positive results, such as when a tribe's
efforts lead to a demonstrable change in behavior or environmental outcomes for tribal members,
as an important indicator.
3-13
-------
B. The Extent to Which Tribes Have Achieved Environmental Capacity
In order to determine the extent to which tribes in our sample have achieved environmental
capacity as defined by GAP, we examined tribal capability in each of the five indicator areas -
technical, legal, enforcement, administrative, and communication. We identified a set of coded
activity types for each indicator and equated tribal capability in that area with a tribe's
participation in one or more related activity. Exhibit 3-7 lists the activity types selected to
demonstrate tribal capability for each indicator and the proportion of the 96 tribes that
participated in activities within each type during 2000-2004. For legal capability, we identified
three activity types: activities to increase legal capability and the development and/or adoption of
codes, ordinances, and standards.10 For enforcement and administrative capability, we identified
only one activity type for each: activities to increase enforcement capability and activities to
increase fiscal administrative capability, respectively.11 For technical capability we identified
four activity types: the hiring of a professional employee, and participation in water, waste, or
air activities.12 For communications capability we identified three activity categories: general,
internal, and external communication activities.
Exhibit 3-7: Tribal Achievement of Environmental Capacity, 2000-2004 (n = 96)
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental Capacity
Developed a Code, Ordinance, or Standard
Participated in an Activity to Increase Legal Capacity
Adopted a Code, Ordinance, or Standard
Participated in an Activity to Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to Increase Fiscal Administration Capacity
Participated in Internal Communication Activities
Participated in External Communication Activities
Participated in General Communication Activities
Number
of Tribes
25
24
7
25
86
70
70
47
15
68
66
28
Percent of
Tribes
26%
25%
7%
26%
90%
73%
73%
49%
16%
71%
69%
29%
10 The development of codes, ordinances, and standards includes the development of permitting/licensing
authority and water quality standards.
11 Enforcement activities include all activity descriptions containing the keywords "enforce" or "inspect."
Activities to increase fiscal administration capability comprise activities related to the development of standards for
property management, procurement, and general fiscal administration.
12 Water activities include ground water, non-point source, point source, source water protection, UIC,
watershed, and wetlands. Waste activities include hazardous waste, recycling, and solid waste. Air activities
include external and indoor air.
3-14
-------
Exhibit 3-8 presents the number and percent of tribes that achieved at least one indicator for each
category of environmental capacity during the period 2000-2004. For example, in Region 1, one
of the two tribes in the region achieved at least one indicator of technical capacity, while in
Regions 10, 33 of the 34 tribes in the region achieved at least one indicator of technical capacity.
This exhibit shows that, overall, the vast majority tribes have achieved at least one measure of
capacity in the areas of technical and communications capability. Approximately one-third of
tribes have achieved at least one measure of legal capability, while fewer tribes have achieved
enforcement and administrative capacity.
Exhibit 3-9 shows the number and proportion of tribes that demonstrated capability in multiple
categories of capacity during the same time period. For example, in Region 1, one of the two
tribes did not demonstrate capability in any of the indicator categories (legal, enforcement,
technical, administrative, or communications capability). The second tribe in the region
demonstrated capability in only one of these categories. Forty-four percent of tribes
demonstrated capability in two categories of capacity, 24 percent demonstrated capability in
three categories, and 21 percent demonstrated capability in four categories of capacity.
3-15
-------
Exhibit 3-8: Number and Percent of Tribes that Achieved at Least One Indicator for Each Category of Environmental Capacity,
by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Total
No. Tribes in
Sample with
Data
2
1
1
7
14
2
6
29
34
96
Legal
Number
0
0
1
2
2
0
2
12
14
33
Percent
0%
0%
100%
29%
14%
0%
33%
41%
41%
34%
Enforcement
Number
0
1
1
4
3
0
2
10
4
25
Percent
0%
100%
100%
57%
21%
0%
33%
35%
11%
26%
Technical
Number
1
1
1
7
14
2
6
29
33
94
Percent
50%
100%
100%
100%
100%
100%
100%
100%
97%
98%
Administrative
Number
0
0
0
0
0
1
1
9
4
15
Percent
0%
0%
0%
0%
0%
50%
17%
31%
12%
16%
Communications
Number
0
1
1
4
10
1
4
27
32
80
Percent
0%
100%
100%
57%
71%
50%
67%
93%
94%
83%
Exhibit 3-9: Number of Categories of Environmental Capacity for which Tribes Achieved at Least One Indicator, by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Total
No. Tribes in
Region with
Data
2
1
1
7
14
2
6
29
34
96
Zero
Number
1
0
0
0
0
0
0
0
0
1
Percent
50%
0%
0%
0%
0%
0%
0%
0%
0%
1%
One
Number
1
0
0
0
2
1
1
1
3
9
Percent
50%
0%
0%
0%
14%
50%
17%
3%
9%
9%
Two
Number
0
0
0
4
10
0
3
9
16
42
Percent
0%
0%
0%
57%
71%
0%
50%
31%
47%
44%
Three
Number
0
1
0
3
1
1
0
8
9
23
Percent
0%
100%
0%
43%
7%
50%
0%
28%
27%
24%
Four
Number
0
0
1
0
1
0
2
11
5
20
Percent
0%
0%
100%
0%
7%
0%
33%
38%
15%
21%
Five
Number
0
0
0
0
0
0
0
0
1
1
Percent
0%
0%
0%
0%
0%
0%
0%
0%
3%
1%
3-16
-------
Technical Capability
Ninety-eight percent of the 96 tribes in our sample participated in activity types associated with
technical capability. Within this indicator category, 90 percent of the 66 tribes for which we had
GAP position data hired at least one professional or technical employee with GAP funding.
Exhibit 3-10 presents the number of professional and technical full-time equivalent (FTE) staff
supported by GAP. Tribes hired more professional staff (an average of 0.8 FTEs), which
included environmental program directors or managers, than technical staff (an average of 0.4
FTEs). These results suggest that tribes depend primarily on GAP funding to hire full-time
environmental directors and managers but may rely on GAP in combination with other funding
sources to hire technical personnel for specific programs or to perform discrete tasks.
Exhibit 3-10: Number of Full Time Equivalent Positions (FTEs) funded by GAP,
2000-2004 (n = 66)
Category
Professional
Technical
Total Professional and
Technical FTEs
Total FTEs provided by GAP
53.4
27.7
81.2
FTEs per Tribe
0.8
0.4
1.2
To assess the scope of tribal technical capability supported by GAP, we examined tribes'
participation in activities in three media-specific program areas. As shown in Exhibit 3-11,
nearly three-quarters (73 percent) of the 96 tribes participated in either a water or waste activity.
Approximately half participated in an air activity. Exhibit 3-12 displays the number and
proportion of tribes in each region that participated in these media-specific areas. Regions 2, 5,
6, 8, 9 and 10 had tribes that participated in activities in all three media areas; Regions 2 and 5
had the highest rates of tribal participation across the three areas. The rates of tribal participation
in Regions 1, 2, 4, and 7 may be a function of the small number of tribes included in the sample
for each region.
Exhibit 3-11: Number and Percent of Tribes with Media-Specific Activities,
2000-2004 (n = 96)
Media Area
Water
Waste
Air
Number of Tribes
70
70
47
Percent of Tribes
73%
73%
49%
3-17
-------
Exhibit 3-12: Number and Percent of Tribes Participating in Activities in Each Media Area,
by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Number of Tribes
with Activity
Information
2
1
1
7
14
2
6
29
34
Water Activities
Number
1
1
1
6
10
2
6
21
22
Percent
50%
100%
100%
86%
71%
100%
100%
72%
65%
Waste Activities
Number
0
1
0
7
8
0
4
22
28
Percent
0%
100%
0%
100%
57%
0%
67%
76%
82%
Air Activities
Number
1
1
0
6
7
1
4
12
15
Percent
50%
100%
0%
86%
50%
50%
67%
41%
44%
Tribes engage in many activities specific to the media program areas, although there appears to
be a greater diversity of activities in the water and waste categories than in the air activity
category. Exhibit 3-13 lists examples of the kinds of activities conducted in each area, as
described by tribal representatives.
Exhibit 3-13: Examples of Media-Specific Program Activities Conducted by Tribes with GAP Funding
Water Activities
Waste Activities
Air Activities
Water quality planning
Development of water quality
standards
Surface water monitoring
Well monitoring and
compliance
Water sample analysis
Riparian zone protection
Sewage treatment permitting
Wetlands - survey, permitting
Mercury testing in fish
Certification of staff
QA/QC for macroinvertebrate
identification
Development of solid waste,
recycling, and burn barrel
ordinances
Solid waste - open dump
cleanup; waste inventory
Recycling - electronics, motor
oil
School chemical cleanup
Hazardous spill cleanup
Indoor air quality monitoring
Communications Capability
Eighty-three percent of tribes in our sample participated in one or more activity types associated
with communication capability (Exhibit 3-9). As shown in Exhibit 3-8, 71 percent of tribes
participated in internal communication activities, (e.g., with the tribal council, tribal members,
schools, and other tribal offices). A nearly equal proportion, 69 percent, participated in external
communication activities (e.g. with other tribes, EPA, other Federal and non-federal agencies,
and non-governmental organizations). Only about 30 percent of tribes reported participating in
general communication activities that could not be specified as either internal or external
communication. Regions 1, 4, 9, and 10 had the highest rates of tribal participation in activities
3-18
-------
associated with communications capability. Neither tribe in our sample from Region 1
participated in these types of activities. As discussed in Section II.D., GAP supports a range of
internal and external communications activities, from developing partnerships with federal
agencies to preparing outreach materials for tribal youth.
Legal and Enforcement Capability
Approximately a third of tribes (34 percent) in our sample met at least one indicator of legal
capability (see Exhibit 3-8). While approximately a quarter of tribes participated in general
activities to increase legal capacity or developed a code, ordinance, or standard, far fewer (7
percent) actually adopted a code, ordinance, or standard (see Exhibit 3-7). Regions 4, 9, and 10
had the highest proportion of tribes in our sample that demonstrated legal capability. Regions 1,
2, and 7 had no tribes that demonstrated this capability. That fewer tribes in our sample had
demonstrated legal capability compared to other indicators runs counter to the experience of
tribal representatives who participated in the panel discussions. Many tribes said they had
developed codes and ordinances for solid waste and water quality, among other programs, with
assistance from GAP. The difference may be a function of the limited time period, 2000-2004,
for which data were available on tribal activities. Depending on when tribes in the sample first
received GAP funding, they may have conducted activities related to legal capability,
particularly the development of tribal codes and standards, either prior to the start or after the
conclusion of this period. Tribes may have also participated in legal activities, including code
and standards development with support provided from non-GAP funding sources. According to
one tribal representative, the tribe provides its own legal staff to assist the environmental
program with its legal activities.
Fewer tribes in our sample demonstrated enforcement capability than legal capability. As shown
in Exhibit 3-7, just over a quarter of tribes participated in inspections or other enforcement-
related activities. Regions 2, 4, and 5 had the highest proportion of tribes that participated in
these activities. No tribes in Regions 1 or 7 participated in activities associated with the
development of enforcement capability from 2000-2004. Section II.B. describes some of
enforcement activities conducted by tribes with support from GAP. Tribal enforcement
capability may be a function of tribes' ability to develop and adopt their own codes, ordinances,
and standards. While the proportion of tribes shown to have demonstrated enforcement
capability is consistent with the assessments of POs, it is possible that tribes may have conducted
enforcement activities or inspections outside the period for which GAP activity data were
available or may have funded these activities with non-GAP support.
Administrative Capability
Only 16 percent of tribes in our sample participated in activities related to administrative
capability (Exhibit 3-8). Fewer tribes have demonstrated capability in the administrative area
than any other indicator. As mentioned above, for the purposes of this analysis we defined
administrative capacity as tribal participation in activities related only to fiscal administrative
capability, including the development of procurement and property management standards.
Our narrow focus on fiscal administrative capability likely omitted tribal participation in other
administrative activities, such as grants administration, which tribes may have conducted as a
component of their media-specific programs. The observations of the regional POs discussed in
3-19
-------
Section II.C. regarding the extent to which tribes are meeting expectations for grants
management and execution of administrative functions suggest that contrary to this finding,
tribes have increased their administrative capability over time.
C. Factors that Contribute to the Achievement of Environmental Capacity
In addition to requesting tribal and regional input on the most important indicators of tribal
environmental capacity, we asked tribes and POs to identify factors that impact environmental
capacity and describe how they influence tribal efforts. We found considerable overlap between
the key indicators of capacity identified by tribes and POs and the factors they cited, which
influence capacity. For example, the ability of a tribe to hire and retain knowledgeable
environmental staff is considered to be both a factor influencing capacity and an indicator of
tribal environmental capacity. In other words, tribes need committed, trained professionals to
develop their environmental programs, and tribes with established environmental programs are
more likely to have a stable, qualified staff.
We presented POs with a list of potential factors and asked them to rank each on a Liechart scale
according to the degree to which it influences the ability of tribes in their respective regions to
attain environmental capacity. Exhibit 3-14 lists the factors that a majority of POs identified as
more (almost always or often) and less (sometimes., seldom, or almost never) influential. The
rankings suggest that the tribal council's relationship with a tribe's environmental program and
its director, demonstrated by the level of support and clarity of direction it provides, significantly
impacts a tribe's ability to attain environmental capacity. POs reported that stable funding is the
most critical factor affecting the longevity of tribal environmental staff, but emphasized that the
actions of tribal council can also lead to staff turnover when there is a lack of communication
between a tribe's environmental director and council members, or when tribal elections result in
a change in leadership. The frequent departure or replacement of trained tribal environmental
staff can deprive tribes of the institutional expertise needed to sustain continuity and grow their
environmental programs. Staff turnover can also diminish other factors that build capacity, such
as information sharing among tribes, communication with EPA, and access to non-GAP funding.
Historically, POs report that the term of employment for a tribal environmental director has
varied across tribes, ranging from a low of <1 year to a high of 20 years, with an average of
approximately 2 to 10 years, depending on the region.
Another potential measure of a tribal council's support for the tribe's environmental program is
its commitment of tribal funding. According to one PO, a tribe's willingness to support the
environmental program with its own money means the tribe values the program enough to help it
succeed. For example, a tribe in the region conducted extensive community outreach among
tribal members to obtain their ideas for a creek restoration project. The tribal council's
investment of discretionary funds, in addition to their official endorsement, was critical in
providing the environmental program with the means to sustain community participation in its
planning and restoration efforts.
3-20
-------
Exhibit 3-14: Factors that Project Officers Said Influence Tribal Environmental Capacity
(in descending order of importance)
More Influential (Almost Always or Often):
• Tribal council support for environmental programs
• Qualifications of tribal environmental director and/or staff
• Turnover rate of tribal environmental director and/or staff
• Clear tribal environmental priorities
• Degree of information sharing among tribes
• Access to funding outside of GAP
• Experience, knowledge, and longevity of EPA POs
• Rate of change in tribal government leadership (e.g., council members)
• Frequency with which tribes request information from EPA
Less Influential (Sometimes, Seldom, or Almost Never):
• Year-to-year shifts in funding priorities due to changing priorities at EPA
• Cohesiveness of a tribe's land base
• Planning documents prepared by tribes (e.g., five year plans)
• Extent to which the regions consult with tribes
• Changes in the GAP funding process
• Different perceptions about funding priorities between tribes and EPA
• Status of tribal office in region (e.g., its location in the organizational chart)
The factors tribal representatives consider most influential correspond with many of the factors
that POs identified as affecting tribes' ability to build capacity, including the stability of
knowledgeable tribal environmental staff, effective communication between tribes and EPA
regions, and the support of tribal council for planning and funding environmental programs
relative to other tribal priorities (e.g., healthcare and economic development). Tribes also
identified factors related to EPA's activities that influence tribes' environmental capacity,
including the need of EPA regional tribal office staff to clearly convey their expectations for
GAP, provide timely responses to grant applications and requests for assistance, and be proactive
in consulting with other EPA media programs. Additional factors mentioned by tribes that
influence capacity include the existing tribal infrastructure and history of successful
implementation of tribal programs, jurisdictional issues, and the length of time a tribe has been
federally recognized.
Tribes stressed that consistent access to funding across a range of media areas is critical to their
ability to build environmental capacity and expand their programming to respond to tribal needs.
Since GAP does not fund implementation activities and many media-specific grants are
competitive, tribes cannot be assured that they will have the funds to maintain their programs
from year to year. One tribe indicated that the ability of EPA to find ways to fit a broader array
of tribal environmental activities within the GAP guidelines is one way to address funding
consistency and help tribes attain environmental capacity. Another suggested that the Treatment
as a State (TAS) designation, if tied to sustained funding such as states receive, could help
ameliorate this problem.
A number of the most influential factors, such as the level and consistency of GAP funding
available to sustain and train tribal environmental staff, the timely provision of technical
3-21
-------
assistance, access to non-GAP EPA funding, and effective communication with tribes, remain
within the purview of EPA. Other factors related to a tribe's internal structure and leadership,
including the clarity of tribal environmental priorities, the rate of change of tribal leaders, and the
support provided by tribal council for planning and funding environmental programs, are,
however, generally beyond EPA's direct control. Yet, given the significant role of GAP and
other EPA support in building, promoting, and sustaining tribal environmental programs, EPA
may still have the ability to indirectly influence these internal tribal factors.
D. Relative Contribution of GAP Toward Achieving Environmental Capacity
As discussed above, there are many factors that can potentially affect tribes' achievement of
environmental capacity such as GAP funding and technical assistance, the stability of tribal
leadership and staffing, and the degree to which Tribal council members focus on environmental
concerns. Another potential factor is tribal access to other sources of funding (e.g., EPA media
programs, other federal and state agencies, and tribes themselves).
Interviews with tribal representatives and POs make clear that they perceive GAP funding as
essential to achieving environmental capacity. Many tribes say that without GAP funding, they
would be able to do very little environmental work. They stress that GAP is the foundation for
their environmental programs, and GAP resources enable them to establish a basic program
infrastructure, through which they can apply for other types of environmental funding. This
view supports a basic premise of the GAP program, namely, that as GAP helps tribes build their
environmental capacity, tribes will be able to access other sources of funding to support their
environmental programs.
If this view is correct, it should be evident from the data on the types of grants tribes are
accessing to support their environmental programs. We would expect tribes that had accessed
GAP for a longer period of time would have built environmental capacity to a greater degree,
would have more established environmental programs, and as a result would be more able to
obtain greater amounts of non-GAP funding than tribes that had accessed GAP over a short time
period. To test this hypothesis, we conducted an analysis of the amount of non-GAP grants
receive by tribes in our sample. Note that because we only had access to data for EPA grants,
this analysis does not include any non-EPA grants (such as water resources funding available
through other federal agencies like the Bureau of Indian Affairs). Out of the 111 tribes in our
sample, 69 (62 percent) received at least one non-GAP EPA grant. Exhibit 3-15 lists the number
of non-GAP EPA grants in each category awarded to tribes in our sample (in descending order),
along with the total amount of funding awarded in each category, the number of tribes in our
sample that received each type of grant, the average grant amount awarded, the average
cumulative amount awarded to each tribe that received a grant, and the average number of grants
awarded to tribes in each category. Tribes in our sample received a total of $89,251,881 in non-
GAP EPA grant funding from 1994-2004. They most often received grants in the general water,
multimedia, and air categories. On average, individual tribes received the most total funding
from general water, water point-source, and multimedia grants. Appendix J lists examples of the
non-GAP EPA programs and other federal agencies identified by tribes that provided funding
and/or technical support.
3-22
-------
Exhibit 3-15: Number and Dollar Amount of Non-GAP EPA Grants Awarded to Tribes,
by Grant Category, 1994-2004 (n=lll)*
Category
Water
Multimedia
Air
Land
Land
Water
Land
Water
Land
Air
Water
Other
Special
Emphasis
Land
Other
Land
Water
Land
Special
Emphasis
Land
Land
Subcategory
Pesticides
Superfund
Wetlands
Lead
NFS
Solid Waste
IAQ
Source water
protection
Education
Environmental
Justice
UST
Brownfields
Point Source
Emergency
Response
Pollution
Prevention
Hazardous Waste
Persistent
Organic
Pollutants
Number of
Grants
386
198
93
93
59
57
52
52
43
41
36
28
26
24
23
7
7
6
6
4
1
Total Funding
Awarded
$28,349,377
$15,856,844
$8,559,419
$3,217,600
$4,824,484
$3,773,206
$2,887,537
$3,648,428
$1,237,792
$1,968,785
$5,998,683
$1,496,762
$737,370
$1,132,701
$1,283,880
$1,360,939
$2,271,025
$145,975
$207,889
$243,185
$50,000
Number of
Tribes with
Grant
56
41
24
16
19
29
19
13
20
13
16
16
19
8
10
6
5
6
4
2
1
Average
Grant
Amount
$73,444
$80,085
$92,037
$34,598
$81,771
$66,197
$55,530
$70,162
$28,786
$48,019
$166,630
$53,456
$28,360
$47,196
$55,821
$194,420
$324,432
$24,329
$34,648
$60,796
$50,000
Average
Total
Amount
Awarded per
Tribe
$506,239
$386,753
$356,642
$201,100
$253,920
$130,110
$151,976
$280,648
$61,890
$151,445
$374,918
$93,548
$38,809
$141,588
$128,388
$226,823
$454,205
$24,329
$51,972
$121,592
$50,000
Average
Number of
Grants per
Tribe
6.9
4.8
3.9
5.8
3.1
2.0
2.7
4.0
2.2
3.2
2.3
1.8
1.4
3.0
2.3
1.2
1.4
1.0
1.5
2.0
1.0
*Funding amounts are rounded to the nearest whole dollar.
We then analyzed the sequence of GAP and non-GAP EPA funding between 1994 and 2004 for
the 111 tribes in the sample. Based on our hypothesis, we anticipated that tribes would access
GAP funds first, followed by non-GAP funds. We found, however, that GAP funds tend to be
accessed year after year, and therefore, a majority of tribes in our sample (90 percent), received
non-GAP EPA funds concurrently with GAP funding. For the remaining tribes that did not
receive non-GAP EPA funds concurrently with GAP funds, about four percent them received
GAP funding first, while about seven percent received non-GAP EPA funding first.13 With
These figures do not add to 100 percent because of rounding error.
3-23
-------
regard to the sequencing of GAP and non-GAP EPA funding, one PO noted that non-GAP EPA
funding was the first type of funding available to tribes. Specifically, Clean Water Act 106 funds
became available in 1989, and this funding was the first grant program accessed by many tribes.
Once GAP became available in 1994, GAP began to offer more money to tribes than other
program funds. This may also explain, in part, why tribes have accessed significantly more
water-related grants than other types of non-GAP EPA grant. Another explanation for the large
number of water-related grants awarded to tribes may be the greater amount and variety of grants
offered by EPA's water programs relative to other media programs.
Since our analysis of the sequence of grants was not informative for the majority of tribes that
received GAP and non-GAP EPA funding concurrently, we conducted an additional analysis to
compare funding patterns for tribes that received GAP early in the program's existence (from
1994 to 1999), to tribes that received GAP later (from 2000 to 2004).14 Specifically, we would
expect tribes that began receiving GAP during the earlier period ("early adopters") to have
accessed a greater percentage of funding from non-GAP EPA sources compared to tribes that
began receiving GAP during the latter period ("late adopters"). If true, this would support the
claim that as tribes receive GAP funds over time, they are able to build environmental capacity
and basic program infrastructure, and then apply for and receive other EPA funds to support their
programs. Note that the small number of late adopters in our sample limits the degree to which
we can make inferences from this analysis, since it is possible that this sample of tribes may not
be representative of the overall population of tribes that received GAP funding from 2000 to
2004.
Our comparison of early vs. late adopters of GAP shows that in fact early adopters have received
a greater percentage of their funding from non-GAP EPA sources. Specifically, as presented in
Exhibit 3-16, only 44 percent of early adopters' total EPA funding between 1994 and 2004 came
from GAP, compared to 80 percent of GAP funding for late adopters. Moreover, there is a
marked increase (1,549 percent) in the amount of non-GAP EPA funding awarded to late
adopters after they received GAP, compared to before they received GAP. This supports the
view that GAP is helping tribes expand their sources of environmental funding, which suggests
that tribes have increased their environmental capacity accordingly.
14 Based on AIEO's suggestion, we have excluded Alaskan tribes from this analysis, because the Alaska
Native Claims Settlement Act treats Alaskan tribes differently than other tribes, and they do not have access to the
same funding as other tribes. When the Alaskan tribes are removed, the sample size is reduced to 70 tribes.
3-24
-------
Exhibit 3-16: Comparison of GAP and Non-GAP EPA Grant Funding for
Early vs. Late Adopting Tribes, 1994-2004 (n=70)*
Number of Tribes
Number of GAP Awards
Number of Non-GAP Awards
Amount of GAP Funding
Amount of Non-GAP Funding
Percent of Total Funding from GAP Grants
Percent of Total Funding from Non-GAP Grants
Average Non-GAP Funding Per Tribe
Median Non-GAP Funding Per Tribe
Total Amount of Non-GAP Funding Received Before
GAP
Amount of Non-GAP Funding received After GAP
Percent Change in Non-GAP Funding
Early Adopters (1994-1999)
61
563
1218
$69,161,568
$87,670,474
44%
56%
$1,437,221
$707,896
N/A
N/A
N/A
Late Adopters (2000-2004)
9
31
12
$3,426,273
$874,395
80%
20%
$97,155
$40,000
$50,000
$824,395
1,549%
*The percent change in non-GAP funding is calculated only for late adopters since data were not available on early adopter receipt of
non-GAP grants for the years preceding 1994.
We also compared achievement of specified indicators of environmental capacity between early
vs. late adopters (Exhibit 3-17). This analysis shows that for five indicators of environmental
capacity (those shown in bold in Exhibit 3-17), a greater percentage of early adopters had
achieved the indicators of capacity compared to late adopters. For example, more early adopters
are participating in waste and water activities, which are both indicators of technical capacity,
compared to late adopters. However, for the remaining seven indicators of capacity, a greater
percentage of late adopters had achieved the indicators. In some cases, this is consistent with our
understanding of how GAP is intended to work. For example, more late adopters than early
adopters have hired a professional employee with GAP funds. This may be because early
adopters are now using other grants to fund salaries for professional employees. In general,
however, the results of this analysis run counter to our hypothesis that tribes that have had GAP
funding for a longer period of time would be further along in the process of developing
environmental capacity, compared to tribes that had received GAP funding for a shorter period
of time.
3-25
-------
Exhibit 3-17: Comparison of Achievement of Environmental Capacity
Early vs. Late Adopting Tribes, 2000-2004 (n=70)*
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental
Capacity
Developed a Code, Ordinance, or
Standard
Participated in an Activity to
Increase Legal Capacity
Implemented a Code, Ordinance,
or Standard
Participated in an Activity to
Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to
Increase Fiscal Administration
Capacity
Participated in Internal
Communication Activities
Participated in External
Communication Activities
Participated in General
Communication Activities
Early Adopters (1994-1999)
Number of
Tribes
12
12
1
21
47
43
41
29
7
38
31
24
Percent of
Tribes
(out of 57)
21%
21%
2%
37%
82%
75%
72%
51%
12%
67%
54%
42%
Late Adopters (2000-2004)
Number of
Tribes
2
0
1
2
9
6
o
J
5
4
7
8
3
Percent of
Tribes
(out of 9)
22%
0%
11%
22%
100%
67%
33%
56%
44%
78%
89%
33%
IV. GAP PROCESS OUTPUTS TO ACHIEVE TRIBAL GOALS AND PRIORITIES
Many of the tribal representatives interviewed stated that a key goal for tribes is having an
environmental presence on tribal land, i.e., a qualified staff person who can coordinate the tribe's
environmental programs, maintain a cohesive program, and be a point of contact for members of
the tribal community and neighboring communities. GAP enables tribes to establish this
environmental presence by providing the funds to hire, train, and retain professional and
technical environmental staff. As discussed in the previous section, GAP funds the salary of the
environmental director or manager, and thus enables the coordination of all environmental
program work. One tribe noted that GAP funds the salaries of environmental specialists who
conduct virtually all of the tribe's permitting work, as well as training tribal staff in specialized
topics such as GIS and Autocad. In some cases, GAP is the primary source of funding for tribal
programs. In other cases, GAP funds only a portion of environmental programs (e.g., 25 percent
of environmental programs for two tribes in EPA Region 5); however, even in these cases, tribes
consider GAP funding essential.
3-26
-------
Tribes emphasize how GAP provides a foundation for tribal environmental programs. One
important way that GAP does this is by providing the framework to leverage and coordinate
other resources, such as other EPA grants. GAP also provides continuity for tribal
environmental programs over time, while at the same time allowing tribes to address new and
emerging environmental issues. Tribes stress the importance of GAP's flexibility: several tribes
perceive that GAP funding has fewer "strings attached" than other sources of funding, and
therefore can be used to address tribal priorities. One tribal representative compared
environmental capacity to a bush, growing in many different directions. The branches are the
various components of a tribes environmental programs and GAP is the root.
While tribal representatives state that GAP funding is vital for establishing and maintaining an
environmental presence, many perceive current levels of GAP funding as insufficient. Several
tribes say that they have received static levels of GAP funding since the inception of the GAP
program, even as they develop new environmental programs and carry out ever more diverse
environmental program activities. For example, one tribe notes that it wants to take on issues
like recycling, green buildings, and pollution prevention, but existing funding is not sufficient to
address these emerging areas of environmental concern. One Eastern tribal representative
expressed the view that GAP funding is primarily going to tribes in the West. This leads to
insufficient funding for the tribe, which results in one staff person having to assume multiple
responsibilities, and ultimately, limits the tribe's ability to maintain environmental capacity.
When asked about additional resources tribes need to develop their environmental programs,
several tribes indicate that above all else, they need sustained, consistent funding over time to
enable them to hire and retain sufficient qualified staff, and thereby retain institutional
knowledge. GAP provides an important source of sustained funding, although some tribes note
that the requirement to re-apply for GAP funding every year takes away from the stability of the
GAP program and the staff that GAP supports. In addition to funding, tribes say they need
infrastructure (especially for water and wastewater), equipment (e.g., computers, vehicles, and
sampling equipment), and more staff to develop their environmental programs. Several tribes
also mention the need for more support from EPA on enforcement programs, including both
funding and technical support. One tribe mentioned that it would be helpful if EPA regions had
more staff available to assist tribes with enforcement and developing codes and ordinances. A
few tribes noted the difficulties of preparing TAS applications, and one tribe asked for EPA to
speed its review of these applications. Another tribe explained that it needed help in preparing
environmental assessments or environmental impact statements under NEPA.
Tribes sometimes have goals and priorities that GAP does not address, either because GAP
funding is insufficient to meet these goals, or because these goals involve implementation of
environmental programs. For example, one tribe mentioned that it has a goal of establishing
regulatory programs, and tribal staff are looking for additional funding beyond GAP to support
this goal. Another tribe said that it has goals and ordinances in place, and has trained staff, but
now needs to start implementing its environmental programs and taking enforcement actions
where needed. Overall, most tribes interviewed say that in order to meet tribal goals, they need
to be able to use GAP funding for program implementation and maintenance, and they need
additional funding to support this additional effort.
3-27
-------
While most tribes perceive overall consistency between GAP goals and tribal priorities, they also
pointed out that it is difficult to mesh the cultural and traditional values of the tribes with the
bureaucratic and regulatory guidelines and definitions established by GAP. For example, some
of the terms GAP uses, such as a "Tribal Environmental Agreement," carry historical and
cultural connotations that raise concerns for tribes (e.g., concerns about signing a formal
agreement with the U.S. Government). Some tribal representatives are concerned that EPA's
interest in measuring tribal capacity will create a "gold standard" that EPA is expecting tribes to
meet, and this may interfere with tribal sovereignty. One tribe recommended that EPA hire more
American Indian POs, and suggested that POs visit the tribes and tribal councils more often to
learn about their culture and the differences among the different tribes. Tribes also point out
that EPA sometimes prioritizes specific programs (e.g., brownfields) that are not in response to a
tribal priority or that tribes are not yet ready to address.15 In other cases, tribal councils want
their environmental programs to do more than what is allowed under GAP funding. For
example, one tribe noted that GAP does not support a tribe's priority to protect sacred and
historical sites from development using NEPA. Tribes emphasize that tribal sovereignty is a
critical issue. While many tribal representatives say that EPA tries hard to address tribes' needs
and priorities, others feel that EPA is seeking "manifest destiny," and is trying to change Indian
ideals and erode tribal sovereignty. Finally, many tribal representatives emphasized that EPA
has a trust responsibility with regard to tribes, and the federal government has the responsibility
to help Tribal programs grow and fulfill their potential.
V. GAP SUPPORT FOR EPA'S STRATEGIC GOAL OF INCREASING TRIBES'
ABILITY TO BUILD ENVIRONMENTAL PROGRAM CAPACITY
EPA's 2003 - 2008 Strategic Plan includes Objective 5.3, which is to build tribal capacity.
Specifically, EPA seeks to "Assist all federally recognized tribes in assessing the condition of
their environment, help in building their capacity to implement environmental programs where
needed to improve tribal health and environments, and implement programs in Indian country
where needed to address environmental issues."16 The strategic plan discusses means and
strategies for achieving Objective 5.3, and mentions the GAP program as part of this discussion.
It specifically states that "EPA will continue to distribute Indian General Assistance Program
capacity building grants with the goal of establishing an environmental presence in all 57217
federally recognized tribes in the United States." AIEO has tracked progress towards this goal in
its Goal 5, Objective 5.3 Reporting System,18 and reports that the percentage of tribes that "had
access to an environmental presence," increased from 36 percent in 1996 to an estimated 90.4
percent in FY 2006, with a peak of 97 percent in FY 2004. Note that access to an environmental
15 It is not clear whether this comment was made specifically regarding GAP, or EPA funding sources in
general.
16 The 2003 - 2008 Strategic Plan is available online at http://www.epa.gov/ocfo/plan/2003sp.pdf. Last
accessed April 2007.
17 The number of federally recognized tribes has changed since the publication of the Strategic Plan: as of
this writing there are 561 such tribes.
18 The Goal 5, Objective 5.3 Reporting System is available online at https://oasint.rtpnc.epa.gov/TATS/
tats_prv/entry_page. Last accessed April 2007.
3-28
-------
presence is defined as an annual dollar value of GAP funding that AIEO determines is needed to
establish an environmental presence. That dollar value equaled $75,000 in FY 1996 - 1997, and
thereafter increased to $110,000. EPA calculates the percentage of tribes with access to an
environmental presence as the total amount of GAP funding awarded to all tribes divided by the
product of the number of federally recognized tribes and the allocation needed to establish an
environmental presence (i.e., $110,000 or $75,000, depending on the year).
EPA's updated 2006 - 2011 Strategic Plan includes an updated Objective 5.3 to "improve human
health and the environment in Indian country." Specifically, EPA is working to, "protect human
health and the environment on tribal lands by assisting federally-recognized tribes to build
environmental management capacity, assess environmental conditions and measure results, and
implement environmental programs in Indian country."19 EPA has modified its strategic targets
in light of this new goal. Of the three strategic targets in the updated plan, one relates directly to
implementation of environmental programs: increasing the percent of tribes implementing
federal environmental programs in Indian country. The other two strategic targets could
potentially be addressed through GAP:
• By 2011, increase the percent of tribes conducting EPA-approved environmental
monitoring and assessment activities in Indian country to 26 percent. (FY 2005 baseline:
20 percent of 572 tribes.)
• By 2011, increase the percent of tribes with an environmental program to 67 percent. (FY
2005 baseline: 54 percent of 572 tribes.)
The strategic plan notes that, "A tribe is counted as having an environmental program for the
purposes of this measure if the tribal government has taken at least one of the following actions,
in combination with having an organizational structure which includes EPA-funded
environmental office or coordinator that has been staffed in the most recent year:
(a) Complete a Tier III TEA, as evidenced by a document signed by the tribal
government and EPA.
(b) Establish environmental laws, codes, regulations, ordinances, resolutions, policies, or
environmental compliance programs, as evidenced by a document signed by the tribal
government.
(c) Complete solid and/or hazardous waste implementation activities.
(d) Complete an inter-governmental environmental agreement (e.g., state-tribe MOA,
federal-tribe MO A, etc)."
19 The 2006 - 2011 Strategic Plan is available online at http://www.epa.gov/ocfo/plan/plan.htm. Last
accessed April 2007.
3-29
-------
While we do not have data to assess progress toward all aspects of these strategic targets,20 we
can say that during the period 2000-2003, 26 percent of the 96 tribes in our sample for which
have GAP activity data developed a code, ordinance, or standard, although these activities may
not be evidenced by a document signed by the tribal government. We also know that during the
same time period, 73 percent of these tribes participated in waste activities, although these may
or may not have included implementation activities.
As part of this evaluation, we asked regional POs about the extent to which they believe that
GAP's goals, objectives, and activities align with or diverge from EPA's strategic goal for tribal
environmental programs.21 Most POs responded that the GAP program is aligned with EPA's
strategic goals for tribes, in that both GAP and EPA's strategic goals seek to build tribal
environmental capacity, and GAP funds activities that are the building blocks of environmental
capacity. GAP supports four common activities that support environmental programs: outreach,
education, enforcement, and training, although the emphasis placed on any of these types of
activities will vary depending on the specific region and the tribe. GAP also enables greater
tribal participation in state and EPA environmental processes that lead to sounder environmental
decisions. For example, in the state of Washington, tribal input and data led EPA to conclude
that state-proposed water quality standards were not sufficiently protective of salmon, and EPA
ultimately required the state to revise its standard.
POs expressed concern about the limitations of GAP that restrict funding to only capacity-
building activities. In addition to building tribal program capacity, many POs believe it is
necessary for EPA to support maintenance of mature tribal environmental programs, otherwise
such programs will cease to operate, and it will be necessary to re-build program capacity. Some
POs pointed out that EPA's strategic goals for Indian country include program implementation,
but such implementation is not supported by GAP, an observation echoed by tribes. One tribe
noted that it can acquire non-GAP grants to develop multi-media programs, but the question of
whether the tribe can sustain these programs with non-GAP funds remains unanswered. Another
tribe identified the apparent contradiction between EPA's strategic performance measures, which
are tied to implementation outputs, and the focus of the GAP guidelines on core program
development and capacity building.
In addition to raising concerns about the issue of program implementation, POs identified other
challenges resulting from using GAP to meet EPA's strategic goals. One PO raised the concern
that it will be difficult for some tribes to ever have delegated authority for environmental
programs, given the frequent turnover of tribal staff. In addition, for tribes that have
"checkerboard" land holdings and non-tribal residents living within the boundaries of
reservations, tribes that seek to develop and enforce codes and ordinances may face fierce
resistance from non-tribal residents that do not accept the tribe's jurisdiction. Many POs report
that tribal needs and concerns vary widely. One PO expressed concern that regional allocations
of GAP funds do not sufficiently align with tribal needs. Since all regions get the same amount
of funding per tribe, a greater share of funding is not being distributed to those tribes with more
20 This evaluation was designed prior to the release of the 2006 - 2011 Strategic Plan, therefore it was not
tailored to measure progress toward strategic targets under this updated plan.
21 During the interview, the interviewer referred to the 2003 - 2008 Strategic Plan Objective 5.3.
3-30
-------
pressing environmental concerns. Another PO pointed to the difficulty of quantifying the impact
of the GAP program for a tribe, particularly the value of having a qualified staff person and
environmental presence, and demonstrating the program's contributions to EPA's strategic goals.
3-31
-------
CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS
The results of this evaluation clearly establish that GAP has been effective in building the
foundation of environmental capacity among tribes, defined as capability in one or more of the
five indicator areas - technical, legal, enforcement, administrative, and communications. This
capability, in turn, has allowed tribes to achieve an environmental presence in Indian country.
Many tribes consider having an environmental presence, i.e., the ability to respond promptly and
effectively to tribal environmental concerns as they arise, as the overarching indicator of
environmental capacity.
To conclude, our review of data from fiscal years 2000 - 2003, along with recent interviews and
discussions with EPA regional project officers (POs) and tribal representatives, suggests that:
• The extent of capacity-building varies across indicator areas for tribes receiving
GAP grants. All but one tribe studied has demonstrated capability for at least one
indicator; the largest proportion of tribes (44 percent) has demonstrated capability for two
indictors. Only one tribe (1 percent) demonstrated capability for all five indicators.
• Tribes receiving GAP have relatively well-developed technical and communications
capabilities. A majority (90 percent) of tribes studied have hired a professional
employee, and most have participated in water and waste activities (73 percent), as well
as internal and external communications activities (71 and 69 percent, respectively).
Tribes use GAP funding to access the training and technical assistance needed to develop
staff expertise and establish an environmental presence. GAP facilitates tribes' ability to
network with other tribes, participate in regional partnerships, and communicate
effectively with tribal Council and the reservation community about tribal environmental
priorities and initiatives.
• Tribes receiving GAP have less developed legal, enforcement, and administrative
capacity compared to the other two indicators. A quarter (26 percent) of tribes have
developed codes, ordinances, or standards with assistance from GAP, but only a few
(seven percent) have gone on to adopt them. An equal number of tribes (26 percent) have
the ability to conduct inspections or other enforcement activities. Among the tribes
studied, only 16 percent demonstrated fiscal administrative capability specific to the
development of property and procurement standards. The results of A133 audits and
regional GAP grant reviews suggest that a few tribes have experienced difficulty with
executing fiscal administrative functions, such as tracking and documenting expenditures
paid for by GAP, and tracking how GAP funds are spent when they are pooled with other
grants. Note that the data that leads to this conclusion is now several years out of date, as
with all of the other indicator data. Interviews with EPA POs suggests that tribal
administrative capacity may have increased in recent years, as described below.
• GAP has done much in recent years to clarify grant expectations and administrative
requirements for tribes, and tribes in turn are increasingly meeting these
expectations and requirements. For example, EPA has developed a standard GAP
work plan format for tribes to use, and has offered GAP management workshops. With
continued outreach and assistance from GAP POs, tribes have significantly improved
their ability to fulfill regional expectations for grants management, such as the timely
4-1
-------
submittal of quality work plans and progress reports. Currently, a majority of tribes in
most regions are submitting their work plans and progress reports on time. Apart from
isolated instances where fiscal problems have been documented, tribes overall almost
always or often meet their region's expectations for the execution of administrative
functions under GAP.
• Tribes report that restrictions on GAP funding that preclude using GAP grants for
program implementation are now hindering tribal environmental program
development. Tribes say that while such restrictions made sense at the outset of GAP,
the situation of tribal programs has evolved. Even though most tribes have more to do to
develop the full complement of capabilities envisioned under GAP (as evidenced by the
fact that only one percent of tribes studied have demonstrated capability for all five
indicators of environmental capacity), some tribes feel that they have already built
sufficient program capacity. These tribes now want to use their GAP funds to address
other pressing needs for program maintenance and implementation. POs agree that tribes
with mature programs have met the limits of capacity building and need funds to sustain
what has been built with GAP.
• Tribes perceive that GAP funding is essential to achieving their environmental
goals, but current levels of funding are insufficient to address tribal priorities.
Consistent, stable, and sufficient funding is a key concern of tribes, since such funding is
viewed as essential for maintaining a trained environmental staff that can respond to
tribal environmental concerns. Tribes have successfully leveraged GAP resources to
acquire additional grant funding from EPA media programs and non-EPA federal
agencies; however, tribes caution that non-GAP grants are extremely competitive and
often have more administrative requirements than GAP. These grants may come with
limits on the type of programs and activities they support and may require substantially
more matching funds than GAP. Tribes expressed concerns about relying on these grants
to implement their environmental programs when their availability and level of funding
can fluctuate from year to year.
Based on the results of our analysis and conversations with tribes, regional POs, and staff at EPA
headquarters, we offer the following recommendations to EPA's American Indian
Environmental Office for ways that EPA can enhance GAP to further support tribes' ability to
establish and sustain their environmental programs.
RECOMMENDATIONS TO EPA
Recommendation 1: Consider developing a mechanism to support Tribal program
implementation. EPA could pursue this in a number of different ways. As an initial step, EPA
headquarters and regions could continue to promote and expand the use of Performance
Partnership Grants (PPGs) by tribes. By combining GAP and other media-specific funds into
one grant, PPGs afford tribes with the flexibility to allocate funding for programs and
implementation activities that most effectively address tribal needs. They also improve
efficiency by streamlining administrative requirements for tribes, such as the preparation of work
plans and progress reports. AIEO has been encouraging use of PPGs in recent years, although
relatively few tribes are currently using them. The relatively infrequent use of PPGs is partially
due to regional preferences, but mainly a result of tribes' perception that PPGs are difficult to put
4-2
-------
in place and administer. Further work with tribes to communicate and possibly simplify the PPG
process could potentially increase the use of PPGs.
Note that since PPGs comprise grants that are subject to variations in funding levels and
availability, this may not be a sustainable option for tribes over the long-term. For this reason,
AIEO could eventually establish a second tier of GAP funding - "GAP plus" - to fund program
implementation for those Tribes that show they have met key indicators of capacity under GAP.
Another approach could be to establish a block grant for tribes similar to those established for
U.S. territories. In the case of a second-tier GAP grant or block grant, the guaranteed availability
of funds for implementation would alleviate the uncertainty currently associated with individual
media grants. This approach would also provide tribes with funding for program activities not
covered under the 20 grants now eligible for incorporation into a PPG. Finally, a second tier
GAP or inclusive block grant would reduce the administrative burden on tribes associated with
applying for and managing numerous grants.
Recommendation 2: Consider working with tribes and regions to enhance administrative,
legal, and enforcement capacity. To help tribes build administrative capacity, EPA could
coordinate more directly with regions to ensure that the frequency and level of programmatic
resources provided, such as GAP training and support for tribal grants management, keep pace
with tribal needs, particularly as new tribal administrative personnel are hired. For tribes that
do not have sufficient funding through GAP or other sources to develop legal services, EPA
regions could offer legal support to help tribes enact their own codes, ordinances, and standards.
In cases where tribes feel that they cannot or do not wish to implement their own environmental
laws and regulations, AIEO should coordinate directly with tribes and clarify roles between tribal
environmental programs and EPA. Although EPA considers tribal delegation of environmental
programs as an important long-term objective, TAS may not be desired or feasible for many
tribes. In these instances, AIEO should consider developing a coherent plan for working with
tribes to protect the environment, while respecting tribal sovereignty.
Recommendation 3: Raise awareness of innovative environmental policy approaches to
complement traditional codes and standards.1 For example, some tribes may benefit from a
greater emphasis on pollution prevention education, self-certification, and compliance assistance
inspections, rather than solely focusing on writing codes, conducting inspections, and taking
enforcement actions. For example, tribes may wish to consider using pollution prevention
materials that have been developed for particular sectors of concern (see, for example, materials
on sectors such as medical facilities, schools, and auto repair shops available from the Pollution
Prevention Resource Exchange2). Tribes may also wish to consider using Environmental Results
Programs (ERP), such as those that have been funded under the EPA State Innovation Grant
program, or adopting elements of ERP.3 To the extent that tribes undertake innovative
1 Innovative policy approaches could potentially used in place of traditional regulatory and enforcement
programs, if such approaches could ensure environmental protection. Innovative policy approaches could also be
used as an interim step, to be used until such time as tribes have traditional regulatory programs in place.
2 Available online at http://www.p2rx.org/P2InfoNexpert/TopicHubs 2.cfm. Last accessed April 2007.
3 Information about State Innovation Grants, including ERPs, is available online at http://www.epa.gov/
innovation/stategrants/. Note that State Innovation Grant funding is not currently available directly for tribes (but
could be available to a tribe in partnership with a state).
4-3
-------
approaches to preventing pollution and encouraging environmental stewardship, it would make
sense for tribes to leverage the considerable experience of EPA and states in developing
innovative policy tools and approaches, as well as specific outreach materials such as pollution
prevention and compliance assistance fact sheets. Tribes may also benefit from working together
to implement innovative approaches. For example, if tribes were interested in exploring ERP as
a potential policy approach, it may make sense for multiple tribes in a region or in neighboring
regions to work together to develop an ERP for a sector of concern. AIEO and regions could
help tribes by raising awareness of innovative policy approaches, readily available materials, and
potential funding sources.
Recommendation 4: Acknowledge cross-cultural differences, and continue working with
tribes to maintain a respectful dialog. A key difference in perspective is that tribes see GAP
funding as a right - an extension of the trust responsibility that EPA has to tribal nations. EPA
views tribes as grantees that must meet certain requirements, particularly administrative
requirements, to show that tribes are accountable for funds that have been spent. Because tribes
vary considerably in their population, land base, wealth, needs, and priorities, they feel that
environmental performance measures should be couched in terms of measuring tribes' progress
toward their goals and not achieving a single "gold standard." There is inherent tension in the
differences between tribal and EPA perspectives, but opportunities to bridge these differences
exist. From the tribes' perspective, one way to strengthen understanding is to hire more Native
Americans to serve as regional POs and tribal coordinators. More frequent site visits to tribes by
AIEO and EPA regional program staff would also help to underscore the diversity of tribal
perspective, priorities, and approaches to environmental protection.
Recommendation 5: Track progress toward achievement of the new 2006-2011 strategic
goals and targets. With the addition of new strategic targets aimed at measuring tribes
implementation of monitoring and assessment activities, EPA needs to develop indicators that
support these goals and targets (e.g., number of tribes that have completed a Tier III TEA,
number of tribes that have completed an inter-governmental environmental agreement (MOAs),
etc.), and regularly track how many tribes are meeting these indicators. At a fundamental level,
AIEO will need to ensure that its data collection systems allow for the effective capture and
tracking of indicators related to these targets. In establishing future strategic goals and targets,
AIEO will also need to consider the degree to which its proposed performance measures align
with tribal priorities and perspectives and the feasibility of tribes' of achieving them. For
example, given the reluctance of many tribes to enter into TEAs and MO As because of concerns
over tribal sovereignty, the selection of these agreements as indicators of tribal performance may
not provide the most accurate estimation of a tribe's environmental capacity or the degree to
which it has successfully implemented specific program components. Further, AIEO needs to
consider that its five-year cycle for setting strategic goals and targets may be too short to
effectively track and measure tribal progress, given that tribal environmental priorities and
abilities may necessitate implementation of programs over a longer time period.
4-4
-------
Appendix A
GAP EVALUATION QUESTIONS
-------
APPENDIX A
EVALUATION QUESTIONS
la. Is GAP being accessed by all federally recognized tribes?
Ib. Why are some tribes not involved in GAP?
Ic. Are there tribes that received GAP grants at one time but which
no longer receive GAP grants? If so, why?
2a. Are tribal governments using the resources (technical, fiscal, and
programmatic) provided as a component of GAP? How often are
GAP resources accessed?
2b. How are tribes using GAP resources?
2c. To what extent have tribes met program expectations for grants
management, execution of administrative functions, and carrying out
proposed activities?
INFORMATION THAT CAN HELP ANSWER QUESTION
• Number and percentage of federally recognized tribes that have ever received GAP funds
• Regional coordinators' perceptions as to why tribes may not seek GAP funding (these
may include having access to other sources of funds, the perception that participation in
GAP is too onerous, etc.)
• Regional coordinators' perceptions as to why tribes may have dropped of GAP grant
rolls.
• Number and type of GAP resources that have been delivered to tribes:
A GAP funding provided to tribes (i.e., fiscal resources)
A Technical assistance and media specific trainings (i.e., technical resources)
A Grants management training (i.e., programmatic resources)
• Tribal access of GAP resources:
A Tribal participation in technical and media-specific trainings
A Tribal participation in grants management training
• Tribal staff and activities funded through GAP (include solid waste implementation)
• Regional coordinators' perceptions about the quality, timeliness, and completeness of
work plans and progress reports received
• Timing of grant end date vs. final close out of the grant - this is an indicator of the degree
to which the grantee met program expectations - the shorter the period of time between
grant end date and final closeout, the more likely that tribes met expectations.
• Results of administrative post award monitoring audits
A-l
-------
EVALUATION QUESTIONS
INFORMATION THAT CAN HELP ANSWER QUESTION
2d. How does participation in GAP increase understanding of how to
develop a tribal environmental program?
• Self reported increase in knowledge and understanding about the necessary steps in
developing a tribal environmental program
• Self-reported increase in skills needed to develop tribal environmental programs
• Self-reported change in awareness and commitment to environmental programs in tribes
3a. What indicators of tribal environmental capacity exist?
3b. To what extent have tribes achieved environmental capacity as
suggested by the presence of these indicators?
Overarching Indicator of Tribal Environmental Capacity:
• Number of GAP recipients that secured ongoing funding from other EPA
sources. [Note that availability of other sources of funding may be a limiting
factor unrelated tribes' environmental capacity.}
• Legal Capability
• Number of GAP recipients that have developed tribal codes, standards, and/or
enforcement programs to control pollution
• Enforcement Capability
• Presence of tribal environmental staff person(s) charged with enforcement duties
• Technical Capability
• Number of GAP recipients with one or more staff specifically tasked with
managing environmental programs (e.g., Environmental Director)
• Size and composition of tribal environmental staff
• Number of environmental programs being carried out in different media annually
by tribes.
• Number of GAP recipients that have taken environmental training
• Communications Capability
• Percent of GAP recipients that have conducted community education and
outreach, based on the grant work plan
• Number of GAP recipients that have executed agreements with other
jurisdictions for management of on- or off-reservation resources
• Tribal participation in EPA or tribal workgroups and/or Task Forces
• Extent of tribal environmental staffs communication of with tribal Council
A-2
-------
EVALUATION QUESTIONS
INFORMATION THAT CAN HELP ANSWER QUESTION
Administrative Capability
• How long has person tasked with managing environmental programs (e.g.
Environmental Director) been in that position
• Number of major findings on A133 audits (which are conducted for any tribe
that spends $500K or more)
• On-site grants management review
3c. What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
Tribal Priorities
Tribal Staffing
Tribal Funding
Commun ication
Degree of tribal Council support for environmental programs
Does tribe have clear environmental priorities?
Rate of change in tribal governments - how often do Council members change?
Turnover rate of tribal Environmental Director and/or staff
Qualifications of tribal Environmental Director and/or staff
Tribal Environmental Staff and/or Director salary levels
GAP funding amounts
GAP funding consistency over time (e.g., number of consecutive GAP grants awarded
over time, range and average number of consecutive GAP grants received by tribes over
time)
Tribe's ability to secure funding beyond GAP (Note, securing funding is both an
indicator of tribal environmental capacity and a factor influencing achievement of
environmental capacity - through interview discussions, try to understand whether
securing other sources of funding is more a cause or effect of success in the GAP
program.)
Degree of information sharing among tribes
Degree to which tribes request information from EPA
Degree to which EPA provides information to tribes ~ specifically extent to which tribes
receiving hands-on technical assistance, oversight, or on-site visits.
A-3
-------
EVALUATION QUESTIONS
INFORMATION THAT CAN HELP ANSWER QUESTION
Regional Activities
• Structure of EPA regional office - specifically, at what level the tribal office located (in
the administrators office or elsewhere - the hypothesis is that the higher the level, the
more likely it is that tribal issues will receive attention and resources).
• Experience and longevity of EPA project officers and technical contacts
• Grantee caseload for EPA project officers
3d. What is the relative contribution of GAP toward achieving
capacity?
• Sequence of grants received by tribes (i.e., do tribes receive GAP grants before other
grants? If so, this would suggest that GAP may contribute to ability to receive other
grants)
• Use of non-GAP funding
(Note - both of these indicators are limited by data on other sources of funds outside of EPA,
e.g. grants from BIA or tribes' internal funds.)
4. Is the GAP providing adequate outputs to achieve tribal goals and
priorities?
• Tribes' perceptions about whether GAP is providing the type and amount of resources
that they need to meet their environmental goals and priorities
• Are there additional resources that tribes feel they would need to have in order to address
their goals and priorities?
• Is environmental capacity building a priority for tribes receiving GAP grants?
5. To what degree does GAP support EPA's strategic goal of
increasing tribes' ability to build environmental program capacity?
• How do GAP's goals currently align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• How do GAP's objectives align with or diverge from EPA's strategic
goal for GAP and other multi-media programs?
• How do tribal GAP activities align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• Summary of GAP goals and objectives
• Summary of EPA's strategic goals and objectives
• Summary of tribal activities funded by GAP, and other outcomes assessed through the
evaluation
A-4
-------
Appendix B
INTERVIEW GUIDE FOR REGIONAL GAP PROJECT OFFICERS
-------
APPENDIX B
Interview Guide for GAP Regional Project Officers
[Introductions] Thank you for agreeing to talk with us today. As you may know,
this interview is part of a broader evaluation of the GAP program that we are conducting
at the request of EPA's American Indian Environmental Office (AIEO). The EPA
General Assistance Program (GAP) as envisioned by Congress includes two key
elements:
1) To provide general assistance grants to build capacity to administer environmental
regulatory programs.
2) To provide technical assistance from EPA to tribal governments and intertribal
consortia in the development of multimedia programs to address environmental issues
on tribal lands.
While tribes and EPA focus on the funding aspects of GAP, technical assistance is
a substantial and significant component of the Program. Examples of assistance include:
• EPA linking tribal staff with the appropriate EPA contacts.
• EPA-sponsored training on administrative or technical skills needed for establishing
tribal multimedia programs.
• EPA review of tribal proposals for establishing programmatic capability, such as
codes, ordinances, and management plans.
• EPA site visits to review and assist tribes with programmatic and administrative
decision making.
The purpose of this interview is to gather your perspective regarding the
effectiveness of the GAP program and how GAP supports EPA's strategic goals of
increasing Tribes' ability to build environmental capacity. Your participation in this
interview, along with input from Tribes and our initial database research, will enhance
our understanding of GAP and will form an important source of information for this
evaluation. Because "environmental capacity" is such a key concept for this evaluation,
we are using a definition based on EPA's 2000 GAP guidelines. For this evaluation,
"environmental capacity" means that a Tribe has established the administrative, legal,
technical and enforcement capability necessary to develop and implement a Tribal
environmental program, as well as the communications capability to work with Federal,
State, Local, Tribal, and other environmental officials. We ask that you base your
answers to questions about environmental capacity on this statutory definition.
We have completed our analysis of data from several EPA databases that contain
information about Tribal grants and will ask you for your insights on these preliminary
findings during this interview. Upon completion of the data collection and analysis phase
of the evaluation, we will compile the results and our conclusions in a report to AIEO,
which will be available for you to review. We anticipate that the results of this
evaluation will help AIEO demonstrate the successes of the Tribal GAP to stakeholders
and identify opportunities for improvement. Finally, we will maintain the confidentiality
B-l
-------
of your responses to the interview questions; any data obtained through this interview
will be analyzed in aggregate with other interview data.
Do you have any questions to ask before I start the interview?
Interview questions
1. To begin, please describe how you are involved with GAP grants in your current
position with EPA.
Evaluation Questions: (la) Is GAP being accessed by all federally recognized
Tribes? (Ib) Why are some Tribes not involved in GAP? (Ic) Are there Tribes that
received GAP grants at one time but which no longer receive GAP grants? If so,
why?
The next two questions in this interview pertain to Tribes' participation in GAP in your
Region:
2. To your knowledge, are there any federally recognized Tribes in your Region that
have not received GAP grants since 1994? If yes, what do you think is the reason
that these Tribes have not received GAP grants (choose all that apply):
a They have not applied for GAP grants. If so, why do you think they have not
applied?
a Their applications have not been accepted. If so, what led them to be not
accepted?
a They have adopted a policy of not accepting Federal grant money.
a Other reasons. (Please explain)
3. To your knowledge, are there any federally recognized Tribes in your Region that
have received GAP grants since 1994, but no longer have a GAP grant? If so,
what do you believe caused the Tribe to drop off of the GAP grant rolls?
4. For Tribes that did not receive GAP grants, do you think the lack of GAP grants
has hindered these Tribes' development of environmental programs? How?
Evaluation Question: (2a) Are Tribal governments using the resources (technical,
fiscal, and programmatic) provided through the GAP? How often are GAP
resources accessed?
According to statute, GAP was established to provide general assistance to Tribes in the
form of monetary support to build environmental capacity as well as technical assistance
for developing multimedia environmental programs on Indian lands. This next question
pertains to Tribes' utilization of all the resources -fiscal, technical, and programmatic -
provided directly and indirectly through the GAP.
B-2
-------
5. Overall, would you say Tribes in your Region utilize the resources (e.g., technical
assistance and training, grants management training) provided as a component of
GAP? If yes, which resources do Tribes most frequently access? Which
resources do Tribes least frequently access?
Evaluation Question: (2c) To what extent have Tribes met program expectations
for grants management, execution of administrative functions, and carrying out
proposed activities?
The next set of questions pertains to the administration of GAP grants by Tribes in your
Region for which you have oversight:
6. To what extent have Tribes in your Region met expectations established by your
region for GAP for each of the following areas?
Grants management
Execution of
administrative
functions
Carrying out
proposed activities
Almost
always
Often
Sometimes
Seldom
Almost
never
Don't
Know
7. To your knowledge, what proportion of Tribes in your Region submits GAP work
plans and progress reports on time?
8. What trends have you seen in the timeliness of GAP work plans and progress
reports submitted by Tribes in your Region over time?
9. To your knowledge, what proportion of Tribes in your Region submits GAP work
plans and progress reports that are complete?
10. What trends have you seen in the completeness of GAP work plans and progress
reports submitted by Tribes in your Region over time?
11. To your knowledge, what proportion of Tribes in your Region submits quality
GAP work plans and progress reports?
12. What trends have you seen in the quality of Tribes' GAP work plans and progress
reports over time?
Evaluation Questions: (3a) What indicators of Tribal environmental capacity exist?
(3b) To what extent have Tribes achieved environmental capacity as suggested by
these indicators? (3c) What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
B-3
-------
13. In your experience, what is the best indicator that a Tribe is achieving
environmental capacity?
14. To your knowledge, to what extent are Tribes in your Region achieving
environmental capacity as defined by this indicator?
a Almost never
a Seldom
a Sometimes
a Often
a Almost always
15. What proportion of Tribes in your Region would you say have received funding
for environmental programs from sources other than GAP?
16. In your experience, when a Tribe secures funding from sources outside of GAP,
does that indicate that the Tribe has been successful in using its GAP grant to
develop environmental capacity?
[Provide summary of findings from database analysis of legal, enforcement, technical
and communications capabilities, and ask for comments.]
The next set of questions pertains to administrative capability achieved by Tribes in your
Region that have received GAP funding. We are considering administrative capability as
one element of environmental capacity.
17. What would you say, historically, is the average length of time for a Tribal
environmental manager or director to remain in his or her position?
18. Do you know if there have been any major findings on audits (e.g., A133, IG,
etc.) conducted for Tribes in your Region? If yes, please describe.
19. What information do you have about the results of on-site GAP grants
management reviews for Tribes in your Region?
The next set of questions pertains to factors that may influence the achievement of
environmental capacity:
20. What factors would you say are the most important contributors to Tribes' ability
to achieve environmental capacity? Please describe how these factors influence
Tribe's achievement of environmental capacity.
21. Next, I am going to read you a list of factors that may influence Tribes ability to
achieve environmental capacity. Based on your knowledge and experience, to
B-4
-------
what degree would you say each factor influences Tribes' ability to achieve
environmental capacity?
For each factor where the interviewee indicates it almost always has an influence on
environmental capacity, follow up by asking how the indicator influences
environmental capacity.
Access to funding outside
of GAP
Tribal Council support
for environmental
programs
Clear Tribal
environmental priorities
Planning documents
prepared by Tribes (e.g.,
five year plans)
Rate of change in Tribal
government leadership
(e.g., Council members)
Turnover rate of Tribal
Environmental Director
and/or staff.
Qualifications of Tribal
Environmental Director
and/or staff
Degree of information
sharing among Tribes
Frequency with which
Tribes request
information from EPA
Cohesiveness of a Tribe's
land base
The status of the Tribal
Office in your Region
(e.g., its location in the
organizational chart)
Experience, knowledge,
and longevity of EPA
project officers
Year-to-year shifts in
funding priorities due to
Almost
always
Often
Sometimes
Seldom
Almost
never
Don't
Know
B-5
-------
changing priorities at
EPA
Different perceptions
about funding priorities
between Tribes and EPA
Extent to which the
Regions consult with the
Tribes
Changes in the GAP
funding process
Almost
always
Often
Sometimes
Seldom
Almost
never
Don't
Know
Evaluation Question: (5) To what degree does GAP support EPA's strategic goal of
increasing Tribes' ability to build environmental program capacity?
The final three questions pertain EPA's strategic goal of building Tribal environmental
capacity under Goal 5.3 of its 2003-2008 Strategic Plan:
22. Based on your understanding of GAP, how do you think GAP's goals and
objectives align with or diverge from EPA's strategic goal for Tribal
environmental programs?
23. Based on your understanding of GAP, how do you think Tribal GAP activities
align with or diverge from EPA's strategic goal for Tribal environmental
programs?
24. Based on your experience, how would you change GAP to improve support for
EPA's strategic goals for Tribal environmental programs? How would you
change GAP to improve support for Tribal goals and priorities?
B-6
-------
Appendix C
DISCUSSION GUIDE FOR TRIBES
-------
Appendix C
Discussion Guide and Proposed Questions for Tribal Panel Discussion
Thank you for joining us today. This panel discussion is part of a broader evaluation of
the General Assistance Program (GAP) that we are conducting at the request of EPA's American
Indian Environmental Office (AIEO). We are conducting this discussion to solicit information
about Tribes' environmental goals and your opinions on how GAP has supported your Tribe's
environmental programming efforts. Your participation on the panel will enhance our
understanding of GAP and will form an important source of information for this evaluation.
Upon completion of the data collection and analysis phase of the evaluation, we will compile the
results and our conclusions in a report to AIEO, which will be available for you to review.
As you may know, from EPA's perspective the primary purpose of the GAP is to help
federally recognized Tribes and intertribal consortia build the basic components of a Tribal
environmental program, which may include planning, developing, and establishing the
administrative, technical, legal, enforcement, communication, and outreach infrastructure. The
primary purpose of this evaluation is to determine how effective GAP has been in building Tribal
environmental capacity among those Tribes receiving funds. Since "environmental capacity" is
such a key concept for this evaluation and we want to very clear about its meaning when we ask
about it in our questions, we are providing a definition based on EPA's 2000 GAP guidelines.
For the purpose of this evaluation, "environmental capacity" means that a Tribe has
established the administrative, legal, technical and enforcement capability necessary to develop
and implement a Tribal environmental program, as well as the communications capability to
work with Federal, State, Local, Tribal, and other environmental officials.
You may also be aware that the EPA General Assistance Program (GAP) as envisioned
by Congress includes two key elements:
1) To provide general assistance grants to build capacity to administer environmental
regulatory programs.
2) To provide technical assistance from EPA to Tribal governments and intertribal consortia
in the development of multimedia programs to address environmental issues on Tribal
lands.
While Tribes and EPA focus on the funding aspects of GAP, technical assistance is a
substantial and significant component of the Program. Examples of assistance include:
• EPA linking Tribal staff with the appropriate EPA contacts.
• EPA-sponsored training on administrative or technical skills needed for establishing
Tribal multimedia programs.
• EPA review of Tribal proposals for establishing programmatic capability, such as codes,
ordinances, and management plans.
C-l
-------
• EPA site visits to review and assist Tribes with programmatic and administrative
decision-making.
Note that this is an evaluation of the GAP program, not the Tribes. Where we ask
questions about Tribes' achievements in building environmental capacity, or barriers to those
achievements, the purpose of our question is to understand whether EPA's GAP program is
working for its intended purpose for the wide array of Tribes that receive GAP grants, and how it
could be improved. This evaluation is not intended as a critique of Tribes' environmental
achievements or a comparison of achievements across Tribes.
We will begin our conversation with introductions and then proceed with a discussion of
Tribal environmental programs and your Tribes' participation in and experiences with GAP, per
the questions below.
Introductory Questions:
Let's begin with questions about Tribes' environmental goals and priorities and their
relationship to GAP and environmental capacity. While Question 2, below, asks for Tribal
definitions of environmental capacity, we ask that you base your answers to all subsequent
questions about environmental capacity on the statutory definition provided.
A. Tribal Environmental Capacity
1. Based on your understanding of GAP, how do the program's goals compare with the
environmental goals and priorities of your Tribe?
2. How does your Tribe define environmental capacity?
3. What is the most important indicator of a Tribe's environmental capacity, as defined by
GAP?
B. Contributors to Environmental Capacity Development
4. Which factors influence environmental capacity attained by Tribes (for example, specific
characteristics of Regional EPA offices or of Tribes)?
5. How have other program areas within EPA, e.g., media programs, helped your Tribe
build environmental capacity?
6. How have non-EPA entities, e.g., other federal agencies, state agencies, and NGOs,
helped your Tribe build environmental capacity?
C-2
-------
Transitional Questions:
Next let's discuss GAP grants and their overall impact on Tribes' level of resources and
environmental programming efforts.
C. GAP Funding Support
7. In thinking about your Tribe's funding over time, how has GAP supported your Tribe's
environmental goals and priorities compared to other funding sources?
8. Which resources provided by GAP (e.g., funds, technical assistance, training) have been
most helpful to your Tribe's environmental programming efforts? Which GAP resources
have not been particularly helpful?
D. Other Sources of Support
9. What other kind of funding has your Tribe received in support of your Tribe's
environmental goals and priorities?
10. What additional resources does your Tribe need to develop your environmental
programs?
Key Questions:
Now let's talk about specific ways in which GAP has influenced Tribes' ability to carry out
environmental programs.
E. GAP Influence on Tribal Approach to Environmental Programming
11. How has GAP influenced your Tribes approach to developing and sustaining Tribal
environmental programs?
12. How has GAP influenced the way you, Tribal members, and particularly members of
your Tribal Council:
a. Establish priorities and plan your environmental programs?
b. Administer your Tribe's environmental programs, (e.g., hiring, training, funding)?
c. Communicate to others within and outside your Tribe about environmental issues of
importance to your Tribe?
C-3
-------
F. Environmental Programs Funded by GAP
13. What kinds of activities or program elements have been funded through your Tribe's
GAP grants? How does this compare to the activities or program elements funded
through other funding sources your Tribe has received?
14. Which of your environmental programs have benefited most from GAP? Which of these
programs have benefited least from GAP?
Final Questions:
15. Considering everything we have discussed so far, how would you change GAP to support
your Tribe's goals and priorities?
16. Have we missed anything?
C-4
-------
Appendix D
LIST OF TRIBES IN EVALUATION SAMPLE
-------
Appendix D
Tribes Selected for GAP Sample
EPA
Region
1
1
2
4
5
5
5
5
5
5
5
6
6
6
6
6
6
6
6
6
6
6
6
6
6
7
7
8
8
8
8
8
8
Tribe Name1
Aroostook Band of Micmac Indians of Maine
Penobscot Tribe of Maine
Seneca Nation of New York2
Miccosukee Tribe of Indians of Florida
Bad River Band of the Lake Superior Tribe of Chippewa Indians of the Bad River
Reservation, Wisconsin
Fond du Lac Band
Grand Portage Band
Grand Traverse Band of Ottawa and Chippewa Indians, Michigan (previously listed as
the Grand Traverse Band of Ottawa & Chippewa Indians of Michigan)
Keweenaw Bay Indian Community, Michigan
Lower Sioux Indian Community in the State of Minnesota
Oneida Tribe of Indians of Wisconsin
Absentee-Shawnee Tribe of Indians of Oklahoma
Cherokee Nation, Oklahoma
Kaw Nation, Oklahoma
Muscogee (Creek) Nation, Oklahoma
Otoe-Missouria Tribe of Indians, Oklahoma
Pueblo of Laguna, New Mexico
Pueblo of Picuris, New Mexico
Pueblo of San Felipe, New Mexico
Pueblo of Taos, New Mexico
Seminole Nation of Oklahoma
Shawnee Tribe, Oklahoma
Tonkawa Tribe of Indians of Oklahoma
Wichita and Affiliated Tribes (Wichita, Keechi, Waco & Tawakonie), Oklahoma
Ysleta Del Sur Pueblo of Texas
Prairie Band of Potawatomi Nation, Kansas
Sac & Fox Tribe of the Mississippi in Iowa
Sisseton-Wahpeton Oyate of the Lake Traverse Reservation, South Dakota (formerly
the Sisseton-Wahpeton Sioux Tribe of the Lake Traverse Reservation)
Skull Valley Band of Goshute Indians of Utah
Southern Ute Indian Tribe of the Southern Ute Reservation, Colorado
Standing Rock Sioux Tribe of North & South Dakota
Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota
Ute Mountain Tribe of the Ute Mountain Reservation, Colorado New Mexico & Utah
In GAP
Database
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Tribes
per
Region
2
1
1
7
14
2
6
1 Per the BIA List published in 70 FR 71194 (11/25/05).
2Although limited data for the Seneca Nation of New York are included in the GAP Activity table for the
years 2000-2003, there are no data for the tribe included in the GAP Position table. For this reason, we did not list
this tribe with the other for which data are available in both the Activity and Position tables. We used data for this
tribe obtained from file reviews to complement the limited data included in the GAP database.
D-l
-------
Tribes Selected for GAP Sample
EPA
Region
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
Tribe Name
Agua Caliente Band of Cahuilla Indians of the Agua Caliente Indian Reservation,
California
Buena Vista Rancheria of Me-Wuk Indians of California
Cabazon Band of Mission Indians, California (previously listed as the Cabazon Band
of Cahuilla Mission Indians of the Cabazon Reservation)
Campo Band of Diegueno Mission Indians of the Campo Indian Reservation,
California
Elem Indian Colony of Porno Indians of the Sulphur Bank Rancheria, California
Enterprise Rancheria of Maidu Indians of California
Ewiiaapaayp Band of Kumeyaay Indians, California (formerly the Cuyapaipe
Community of Diegueno Mission Indians of the Cuyapaipe Reservation)
Hopi Tribe of Arizona
Jamul Indian Village of California
Kashia Band of Porno Indians of the Stewarts Point Rancheria, California
La Jolla Band of Luiseno Mission Indians of the La Jolla Reservation, California
La Posta Band of Diegueno Mission Indians of the La Posta Indian Reservation,
California
Manchester Band of Porno Indians of the Manchester-Point Arena Rancheria,
California
Middletown Rancheria of Porno Indians of California
Paiute-Shoshone Indians of the Lone Pine Community of the Lone Pine Reservation,
California
Paiute-Shoshone Tribe of the Fallen Reservation and Colony, Nevada
Pala Band of Luiseno Mission Indians of the Pala Reservation, California
Paskenta Band of Nomlaki Indians of California
Quartz Valley Indian Community of the Quartz Valley Reservation of California
Quechan Tribe of the Fort Yuma Indian Reservation, California & Arizona
Redding Rancheria, California
Santa Rosa Indian Community of the Santa Rosa Rancheria, California
Santa Ynez Band of Chumash Mission Indians of the Santa Ynez Reservation,
California
Scotts Valley Band of Porno Indians of California
Sherwood Valley Rancheria of Porno Indians of California
Tuolumne Band of Me-Wuk Indians of the Tuolumne Rancheria of California
Habematolel Porno of Upper Lake, California (formerly the Upper Lake Band of Porno
Indians of Upper Lake Rancheria of California)
Walker River Paiute Tribe of the Walker River Reservation, Nevada
White Mountain Apache Tribe of the Fort Apache Reservation, Arizona
Yavapai- Apache Nation of the Camp Verde Indian Reservation, Arizona
In GAP
Database
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Tribes
per
Region
30
D-2
-------
Tribes Selected for GAP Sample
EPA
Region
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Tribe Name
Agdaagux Tribe of King Cove
Arctic Village
Chilkat Indian Village (Klukwan)
Chilkoot Indian Association (Haines)
Chinik Eskimo Community (Golovin)
Circle Native Community
CoeurD'Alene Tribe of the CoeurD'Alene Reservation, Idaho
Confederated Tribes of the Grand Ronde Community of Oregon
Confederated Tribes of the Siletz Reservation, Oregon
Egegik Village
Eklutna Native Village
Emmonak Village
Gulkana Village
Hoonah Indian Association
Inupiat Community of the Arctic Slope
Kenaitze Indian Tribe
Kootenai Tribe of Idaho
McGrath Native Village
Naknek Native Village
Native Village of Ambler
Native Village of Eagle
Native Village of Eyak (Cordova)
Native Village of Kivalina
Native Village of Kongiganak
Native Village of Kotzebue
Native Village of Kwigillingok
Native Village of Napaimute
Native Village of Nightmute
Native Village of Nunapitchuk
Native Village of Point Lay
Native Village of Ruby
Native Village of Selawik
Native Village of Tanacross
Native Village of Tatitlek
Native Village of Tazlina
Newtek Village
Nondalton Village
Northway Village
Organized Village of Kwethluk
Pedro Bay Village
Petersburg Indian Association
Sauk-Suiattle Indian Tribe of Washington
In GAP
Database
Y
Y
Y
No. Tribes
per Region
48
D-3
-------
Tribes Selected for GAP Sample
EPA
Region
10
10
10
10
10
10
Sample
Size
Tribe Name
Shoshone-Bannock Tribes of the Fort Hall Reservation of Idaho
Skokomish Indian Tribe of the Skokomish Reservation, Washington
Twin Hills Village
Village of Aniak
Village of Salamatoff
Yakutat Tlingit Tribe
In GAP
Database
No. Tribes
per Region
111
D-4
-------
Appendix E
GPRA CODES AND ACTIVITY CATEGORIES
-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 383 AIR ACTIVITIES
n 109 AIR ACTIVITIES
O 100 AIR ACTIVITIES
n 101 AIR ACTIVITIES
EH
102 AIR ACTIVITIES
EH
103 AIR ACTIVITIES
EH
104 AIR ACTIVITIES
EH 105 AIR ACTIVITIES
EH 384 AIR ACTIVITIES
EH 106 AIR ACTIVITIES
EH 107 AIR ACTIVITIES
EH
108 AIR ACTIVITIES
EH 119 AIR ACTIVITIES
EH 110 AIR ACTIVITIES
EH 111 AIR ACTIVITIES
EH
112 AIR ACTIVITIES
EH
113 AIR ACTIVITIES
EH
114 AIR ACTIVITIES
EH 115 AIR ACTIVITIES
EH 385 AIR ACTIVITIES
EH 116 AIR ACTIVITIES
EH 117 AIR ACTIVITIES
EH
118 AIR ACTIVITIES
EH GENERAL MANAGEMENT
236 AND ADMINISTRATION
EH GENERAL MANAGEMENT
428 AND ADMINISTRATION
EH
GENERAL MANAGEMENT
421 AND ADMINISTRATION
GENERAL MANAGEMENT
420 AND ADMINISTRATION
SUB1
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Communication
Communication
Communication
Grant Number(s):
SUB2 SUB3 Note
Air Grants Administration
Building Survey
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Source Inventory
Source Inventory-T1 4(9005)
Staff Program Capacity
Development
Air Grants Administration
Building Survey
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Source Inventory
Source Inventory-T1 4(9005)
Staff Program Capacity
Development
External Communication (e.g.,
with regulated community or other
governments)
Internal Communication (e.g., with
Tribal Executive, community, K-
12, or adult education)
GENERAL MANAGEMENT
382 AND ADMINISTRATION
GENERAL MANAGEMENT
225 AND ADMINISTRATION
Program Establishment
Activities
Program Establishment
Activities
Baseline Environmental
Assessment
Page 1 of 10
-------
Tribe Name:
ACT GPRA I
CATEGORY
SUB1
Grant Number(s):
SUB2
SUB3
Note
GENERAL MANAGEMENT
228 AND ADMINISTRATION
GENERAL MANAGEMENT
229 AND ADMINISTRATION
GENERAL MANAGEMENT
230 AND ADMINISTRATION
Program Establishment
Activities
Program Establishment
Activities
Program Establishment
Activities
Enforcement Capability
Fiscal Administration Capacity
Fiscal Administration Capacity
Standards in
Place
Standards for
Property
Management
GENERAL MANAGEMENT
231 AND ADMINISTRATION
GENERAL MANAGEMENT
232 AND ADMINISTRATION
GENERAL MANAGEMENT
235 AND ADMINISTRATION
GENERAL MANAGEMENT
233 AND ADMINISTRATION
GENERAL MANAGEMENT
234 AND ADMINISTRATION
436 GRANT WRITING
429 GRANT WRITING
430 GRANT WRITING
431 GRANT WRITING
432 GRANT WRITING
433 GRANT WRITING
434 GRANT WRITING
435 GRANT WRITING
381 LAND ACTIVITIES
344 LAND ACTIVITIES
333 LAND ACTIVITIES
339 LAND ACTIVITIES
338 LAND ACTIVITIES
340 LAND ACTIVITIES
341 LAND ACTIVITIES
391 LAND ACTIVITIES
342 LAND ACTIVITIES
343 LAND ACTIVITIES
Program Establishment
Activities
Program Establishment
Activities
Staff
Staff
Staff
Clean Air Quality
Clean Water Activities
Cross Media Activities
Drinking Water Quality
General Management and
Administration
Solid and Hazardous Waste
Activities
Toxic Substances Activities
Asbestos
Asbestos
Asbestos
Asbestos
Asbestos
Asbestos
Asbestos
Asbestos
Asbestos
Standards for
Fiscal Administration Capacity Procurement
Legal Capability
Develop Position Descriptions
Training Program
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Page 2 of 10
-------
Tribe Name:
ACT GPRA I
CATEGORY
SUB1
Grant Number(s):
SUB2
SUB3
Note
252 LAND ACTIVITIES
237 LAND ACTIVITIES
247 LAND ACTIVITIES
246 LAND ACTIVITIES
248 LAND ACTIVITIES
249 LAND ACTIVITIES
395 LAND ACTIVITIES
250 LAND ACTIVITIES
251 LAND ACTIVITIES
268 LAND ACTIVITIES
253 LAND ACTIVITIES
263 LAND ACTIVITIES
262 LAND ACTIVITIES
264 LAND ACTIVITIES
265 LAND ACTIVITIES
396 LAND ACTIVITIES
266 LAND ACTIVITIES
267 LAND ACTIVITIES
356 LAND ACTIVITIES
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Emergency Response
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste
Lead
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
345 LAND ACTIVITIES
351 LAND ACTIVITIES
350 LAND ACTIVITIES
352 LAND ACTIVITIES
353 LAND ACTIVITIES
392 LAND ACTIVITIES
Lead
Lead
Lead
Lead
Lead
Lead
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Page 3 of 10
-------
Tribe Name:
ACT GPRA I
CATEGORY
SUB1
Grant Number(s):
SUB2
SUB3
Note
354 LAND ACTIVITIES
355 LAND ACTIVITIES
368 LAND ACTIVITIES
357 LAND ACTIVITIES
363 LAND ACTIVITIES
362 LAND ACTIVITIES
364 LAND ACTIVITIES
365 LAND ACTIVITIES
393 LAND ACTIVITIES
366 LAND ACTIVITIES
367 LAND ACTIVITIES
380 LAND ACTIVITIES
369 LAND ACTIVITIES
375 LAND ACTIVITIES
374 LAND ACTIVITIES
376 LAND ACTIVITIES
377 LAND ACTIVITIES
394 LAND ACTIVITIES
378 LAND ACTIVITIES
379 LAND ACTIVITIES
284 LAND ACTIVITIES
269 LAND ACTIVITIES
279 LAND ACTIVITIES
278 LAND ACTIVITIES
280 LAND ACTIVITIES
Lead
Lead
Pesticides
Pesticides
Pesticides
Pesticides
Pesticides
Pesticides
Pesticides
Pesticides
Pesticides
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
Recycling
Recycling
Recycling
Recycling
Recycling
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Page 4 of 10
-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 281 LAND ACTIVITIES
n 397 LAND ACTIVITIES
O 282 LAND ACTIVITIES
EH
283 LAND ACTIVITIES
n 300 LAND ACTIVITIES
285 LAND ACTIVITIES
CH
295 LAND ACTIVITIES
O 294 LAND ACTIVITIES
CH
296 LAND ACTIVITIES
n 297 LAND ACTIVITIES
O 398 LAND ACTIVITIES
n 298 LAND ACTIVITIES
n
299 LAND ACTIVITIES
O 316 LAND ACTIVITIES
301 LAND ACTIVITIES
EH
311 LAND ACTIVITIES
n 310 LAND ACTIVITIES
CH
31 2 LAND ACTIVITIES
O 313 LAND ACTIVITIES
n 399 LAND ACTIVITIES
O 314 LAND ACTIVITIES
EH
315 LAND ACTIVITIES
n 332 LAND ACTIVITIES
SUB1
Recycling
Recycling
Recycling
Recycling
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Solid Waste
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
UST
Grant Number(s):
SUB2 SUB3 Note
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
317 LAND ACTIVITIES
UST
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Page 5 of 10
-------
Tribe Name:
ACT GPRA 1 CATEGORY
n
327 LAND ACTIVITIES
O 326 LAND ACTIVITIES
EH
328 LAND ACTIVITIES
n 329 LAND ACTIVITIES
O 400 LAND ACTIVITIES
n 330 LAND ACTIVITIES
n
331 LAND ACTIVITIES
O SPECIAL EMPHASIS
205 ACTIVITIES
O SPECIAL EMPHASIS
164 ACTIVITIES
O SPECIAL EMPHASIS
155 ACTIVITIES
O SPECIAL EMPHASIS
158 ACTIVITIES
O SPECIAL EMPHASIS
159 ACTIVITIES
O SPECIAL EMPHASIS
160 ACTIVITIES
O SPECIAL EMPHASIS
161 ACTIVITIES
O SPECIAL EMPHASIS
162 ACTIVITIES
O SPECIAL EMPHASIS
163 ACTIVITIES
O SPECIAL EMPHASIS
174 ACTIVITIES
O SPECIAL EMPHASIS
165 ACTIVITIES
O SPECIAL EMPHASIS
168 ACTIVITIES
O SPECIAL EMPHASIS
169 ACTIVITIES
O SPECIAL EMPHASIS
170 ACTIVITIES
O SPECIAL EMPHASIS
171 ACTIVITIES
O SPECIAL EMPHASIS
172 ACTIVITIES
O SPECIAL EMPHASIS
173 ACTIVITIES
O SPECIAL EMPHASIS
184 ACTIVITIES
SUB^
UST
UST
UST
UST
UST
UST
UST
Endangered Species
Endangered Species
Endangered Species
Endangered Species
Endangered Species
Endangered Species
Endangered Species
Endangered Species
Environmental Justice
Environmental Justice
Environmental Justice
Environmental Justice
Environmental Justice
Environmental Justice
Environmental Justice
Environmental Justice
NEPA/TEPA/Cultural
Resources
Grant Number(s):
SUB2 SUB3 Note
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
Grants Administration
Wetland Identification and
Delineation
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
Grants Administration
Wetland Identification and
Delineation
Page 6 of 10
-------
Tribe Name:
ACT GPRA 1
CATEGORY
SUB1
Grant Number(s):
SUB2
SUB3
Note
SPECIAL EMPHASIS
175 ACTIVITIES
SPECIAL EMPHASIS
178 ACTIVITIES
SPECIAL EMPHASIS
179 ACTIVITIES
SPECIAL EMPHASIS
180 ACTIVITIES
SPECIAL EMPHASIS
181 ACTIVITIES
SPECIAL EMPHASIS
182 ACTIVITIES
SPECIAL EMPHASIS
183 ACTIVITIES
SPECIAL EMPHASIS
194 ACTIVITIES
SPECIAL EMPHASIS
185 ACTIVITIES
SPECIAL EMPHASIS
188 ACTIVITIES
SPECIAL EMPHASIS
189 ACTIVITIES
SPECIAL EMPHASIS
190 ACTIVITIES
SPECIAL EMPHASIS
191 ACTIVITIES
SPECIAL EMPHASIS
192 ACTIVITIES
SPECIAL EMPHASIS
193 ACTIVITIES
SPECIAL EMPHASIS
204 ACTIVITIES
SPECIAL EMPHASIS
195 ACTIVITIES
SPECIAL EMPHASIS
198 ACTIVITIES
SPECIAL EMPHASIS
199 ACTIVITIES
SPECIAL EMPHASIS
200 ACTIVITIES
SPECIAL EMPHASIS
201 ACTIVITIES
SPECIAL EMPHASIS
202 ACTIVITIES
SPECIAL EMPHASIS
203 ACTIVITIES
154 WATER ACTIVITIES
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
Pollution Prevention
Pollution Prevention
Pollution Prevention
Pollution Prevention
Pollution Prevention
Pollution Prevention
Pollution Prevention
Pollution Prevention
Sustainable Development
Sustainable Development
Sustainable Development
Sustainable Development
Sustainable Development
Sustainable Development
Sustainable Development
Sustainable Development
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
Grants Administration
Wetland Identification and
Delineation
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
Grants Administration
Wetland Identification and
Delineation
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
Grants Administration
Wetland Identification and
Delineation
Page 7 of 10
-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 224 WATER ACTIVITIES
n 401 WATER ACTIVITIES
O 403 WATER ACTIVITIES
n 404 WATER ACTIVITIES
406 WATER ACTIVITIES
407 WATER ACTIVITIES
408 WATER ACTIVITIES
O 409 WATER ACTIVITIES
n 405 WATER ACTIVITIES
410 WATER ACTIVITIES
O 131 WATER ACTIVITIES
n 121 WATER ACTIVITIES
SUB1
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Non-Point Sources
Non-Point Sources
Grant Number(s):
SUB2 SUB3 Note
Administering EPA Grant
Baseline Assessment
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Administering EPA Grant
122 WATER ACTIVITIES
126 WATER ACTIVITIES
127 WATER ACTIVITIES
128 WATER ACTIVITIES
129 WATER ACTIVITIES
386 WATER ACTIVITIES
130 WATER ACTIVITIES
402 WATER ACTIVITIES
412 WATER ACTIVITIES
41 3 WATER ACTIVITIES
41 5 WATER ACTIVITIES
41 6 WATER ACTIVITIES
41 7 WATER ACTIVITIES
418 WATER ACTIVITIES
41 4 WATER ACTIVITIES
41 9 WATER ACTIVITIES
21 4 WATER ACTIVITIES
206 WATER ACTIVITIES
207 WATER ACTIVITIES
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Source Water Protection
Source Water Protection
Source Water Protection
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Administering EPA Grant
Baseline Assessment
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Page 8 of 10
-------
Tribe Name:
ACT GPRA 1 CATEGORY
n
208 WATER ACTIVITIES
O 209 WATER ACTIVITIES
n 389 WATER ACTIVITIES
O 210 WATER ACTIVITIES
n 211 WATER ACTIVITIES
CH
21 2 WATER ACTIVITIES
CH
213 WATER ACTIVITIES
n 223 WATER ACTIVITIES
CH
21 5 WATER ACTIVITIES
CH
21 6 WATER ACTIVITIES
CH
21 7 WATER ACTIVITIES
O 218 WATER ACTIVITIES
n 390 WATER ACTIVITIES
O 219 WATER ACTIVITIES
n 220 WATER ACTIVITIES
CH
221 WATER ACTIVITIES
CH
222 WATER ACTIVITIES
SUB1
Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Grant Number(s):
SUB2 SUB3 Note
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grants Administration
Source Water Inventory
Staff Program Capacity
Development
Wetland Identification and
Delineation. NOT IN USE.
PLEASE RECLASSIFY
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grants Administration
Source Water Inventory
Staff Program Capacity
Development
Wetland Identification and
Delineation. NOT IN USE.
PLEASE RECLASSIFY
142 WATER ACTIVITIES Watershed
132 WATER ACTIVITIES Watershed
133 WATER ACTIVITIES Watershed
137 WATER ACTIVITIES Watershed
138 WATER ACTIVITIES Watershed
139 WATER ACTIVITIES Watershed
140 WATER ACTIVITIES Watershed
387 WATER ACTIVITIES Watershed
141 WATER ACTIVITIES Watershed
153 WATER ACTIVITIES Wetlands
143 WATER ACTIVITIES Wetlands
144 WATER ACTIVITIES Wetlands
Administering EPA Grant
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Administering EPA Grant
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
Page 9 of 10
-------
Tribe Name:
\ ACT GPRA 1
CATEGORY
SUB1
Grant Number(s):
SUB2
SUB3
Wore
p
p
p
n
148 WATER ACTIVITIES Wetlands
149 WATER ACTIVITIES Wetlands
150 WATER ACTIVITIES Wetlands
151 WATER ACTIVITIES Wetlands
388 WATER ACTIVITIES Wetlands
152 WATER ACTIVITIES Wetlands
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Page 10 of 10
-------
Appendix F
TRIBAL POPULATION STATISTICS
-------
APPENDIX F: POPULATION INFERENCES BASED ON SAMPLE DATA
Exhibit 1 below shows confidence intervals for the population proportions of various indicators.
Exhibit 1: Summary of Population Proportions
Indicator*3'
Percent of tribes that participated in activities to
increase legal capacity
Percent of tribes that developed codes,
ordinances, or standards
Percent of tribes that adopted/implemented
codes, ordinances, or standards
Percent of tribes that participated in enforcement
activities
Percent of tribes that have at least one
professional staff member
Percent of tribes participating in water activities
Percent of tribes participating in waste activities
Percent of tribes participating in air activities
Percent of tribes participating in internal
communication activities
Percent of tribes participating in external
communication activities
Percent of tribes participating in general
communication activities
Percent of tribes that took advantage of technical
resources
Percent of tribes that took advantage of
programmatic resources
Percent of non-GAP grants received
concurrently with GAP funding
Percent of tribes receiving non-GAP funding
Sample Proportion
25%
26%
7%
26%
90%
73%
73%
49%
71%
69%
29%
76%
23%
90%
62%
Sample Size(b)
96
96
96
96
96
96
96
96
96
96
96
96
96
1242(c)
111
95 Percent
Confidence Interval
Low
16%
17%
2%
17%
83%
64%
64%
39%
62%
59%
20%
68%
15%
88%
53%
High
34%
35%
12%
35%
96%
82%
82%
59%
80%
78%
38%
85%
31%
91%
71%
(a) We could not calculate population proportions for the percentage of tribes with unfavorable audit results, because
results do not meet the criteria of approximately normal distribution.
(b) Unless otherwise noted, sample size refers to the number of tribes.
(c) In this case, sample size refers to number of grants, rather than number of tribes.
F-l
-------
Exhibit 2 below shows confidence intervals for the population means for various indicators.
Exhibit 2: Summary of Population Means
Indicator
Amount of GAP grant
Number of technical resources accessed
Number of programmatic resources accessed
Number of months between project end and closeout date
Number of professional and technical FTEs hired
Sample Mean
$102,472
4.4
1.2
12.7
1.2
Sample Size(a)
754
96
40
175 (b)
66
95 Percent
Confidence Interval
Low
$ 97,957
3.4
0.8
11.4
1.1
High
$106,986
5.3
1.6
14.0
1.4
(a) Unless otherwise noted, sample size refers to the number of tribes.
(b) In this case, sample size refers to number of grants, rather than number of tribes.
F-2
-------
Appendix G
ACTIVITY COMPARISON ACROSS TRIBE SAMPLE GROUPS
-------
Appendix G
Percent of Tribes with Activities in each Category
100%
0%
General Land Activities Water Grant Writing Air Activities Special
Management Activities Emphasis
and Activities
Administration
D Tribes in Database • Tribes with File Reviews
Percent of Tribes Participating in each Activity Type
100%
O Tribes in Database • Tribes in File Reviews
G-l
-------
Appendix H
REGIONAL GAP PROJECT OFFICERS INTERVIEWED
-------
Appendix H
INTERVIEWEES: REGIONAL PROJECT OFFICERS FOR GAP
EPA Region
Region 1
Region 2
Region 4
Region 5
Region 6
Region 7
Region 8
Region 9
Region 10
Cross-Region/Region 4
Primary Interviewee
Jean Crocker
Christine Yost
Cynthia Nolan
Michael Nishi
Dale Roy
Wolfgang Brandner
Judith Hervig (not completed)
Timothy Wilhite
Alan Moomaw
Dan Clone
H-l
-------
Appendix I
TRIBES PARTICIPATING IN PANEL DISCUSSIONS
-------
Appendix I
United South and Eastern Tribes (USET) 2007 Impact Week Meeting, Arlington, VA1
• Miccosukee Tribe of Indians of Florida (Panelist)
• Poarch Band of Creek Indians of Alabama (Panelist)
• Narragansett Indian Tribe of Rhode Island (Panelist)
EPA Region 5 2007 Indian GAP Conference Week, Chicago, IL
• Forest County Potawatomi Community, Wisconsin (Panelist)
• Little Traverse Bay Bands of Odawa Indians, Michigan (Panelist)
• Grand Traverse Band of Ottawa and Chippewa Indians, Michigan
• Huron Potawatomi, Inc., Michigan
• Lac Courte Oreilles Band of Lake Superior Chippewa Indians of Wisconsin (Panelist)
• Leech Lake Band of Minnesota Chippewa
• Little River Band of Ottawa Indians, Michigan
• Mille Lacs Band of Minnesota Chippewa DNR/E
• Pokagon Band of Potawatomi Indians, Michigan and Indiana
• Quinault Tribe of the Quinault Reservation, Washington
• Saginaw Chippewa Indian Tribe of Michigan
EPA Region 8 Tribal Operations Committee Meeting, Denver, CO
• Ute Mountain Tribe of the Ute Mountain Reservation, Colorado, New Mexico & Utah
(Panelist)
• Turtle Mountain Band of Chippewa Indians of North Dakota (Panelist)
• Confederated Salish & Kootenai Tribes of the Flathead Reservation, Montana (Panelist)
• Assiniboine and Sioux Tribes of the Fort Peck Indian Reservation, Montana
• Blackfeet Tribe of the Blackfeet Indian Reservation of Montana
• Flandreau Santee Sioux Tribe of South Dakota
• Northern Cheyenne Tribe of the Northern Cheyenne Indian Reservation, Montana
• Shoshone Tribe of the Wind River Reservation, Wyoming
• Sisseton-Wahpeton Oyate of the Lake Traverse Reservation, South Dakota
• Southern Ute Indian Tribe of the Southern Ute Reservation, Colorado
• Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota
• Ute Indian Tribe of the Uintah & Ouray Reservation, Utah
1 Due to inclement weather, we were not able to conduct a panel discussion with tribes at this event. We
instead conducted separate interviews with each of the tribal representatives on the panel.
1-1
-------
Appendix J
SOURCES OF NON-GAP FUNDING AND PROGRAM SUPPORT
-------
Appendix J
Sources of Non-GAP Funding and Program Support Received by Tribes
EPA Programs
• Clean Water Act (CWA) Section 106 - water program infrastructure, staffing, water quality standards
development, well-protection plans, water quality lab, and training.
• CWA Section 106b (since cut) - wetland management planning
• CWA Section 319 - non point source pollution, water quality standards development
• Solid waste - solid waste planning, code development, staffing, recycling
• Clean Air Act Section 103 - air program, mercury deposition
• Resource Conservation and Recovery Act (RCRA) Support Agency Cooperative Agreement
• Underground Storage Tanks (UST) and Underground Injection Control (UIC) - Direct Implementation
Tribal Cooperative Agreement (DITCA)
• Public Water Systems Safety (since cut) - water treatment/facilities
• Brownfields
• Pesticides
• Asthma Program
• Environmental Justice
Non-EPA Agencies and Programs
• U.S. Natural Resources Conservation Service (NRCS) - sedimentation issues, road issues, technical data,
forest management, erosion control, soil protection
• U.S. Department of Agriculture (USDA) (Rural development) - solid waste programs, equipment, buildings
• U.S. Fish and Wildlife Service (FWS) - technical assistance, travel funding, forestry, water testing, burning,
wildlife habitat
• U.S. Bureau of Indian Affairs (BIA) - surface water cleaning, land resources, water resources, fish and
wildlife compliance, land use enforcement, forestry, road maintenance, parks, wetlands permits
• U.S. Bureau of Reclamation (BOR) - water activities; septic issues
• U.S. Indian Health Service (HS) - solid waste programs, equipment, buildings
• U.S. Department of Health and Human Services (DHHS) - Administration for Native Americans (ANA)
• U.S. Army Corps of Engineers - land use/wetlands permitting
• State entities - WI Department of Natural Resources
• Non-profits - National Groundwater Assn., North American Waste Management Society, Tip-of-the-Mitt
Watershed Council, WI Assn. Of Lakes, Little Traverse Bay Organization, West Virginia University
J-l
------- |