SERA
May 2007
        Evaluation of the
        Tribal General
        Assistance
        Program
        Promoting Environmental Results
        •4	
        Through Evaluation

-------
This evaluation was performed by Industrial Economics, Incorporated
(lEc) for EPA's Office of Environmental Policy Innovation under
Contract EP-W-04-023 between EPA and lEc. The lEc evaluation
team included Tracy Dyke-Redmond and Heather Posner and Allison
Barasz.  Rodges Ankrah, Teresa Kuklinski and Candi Schadel of
EPA's Office of Water, American Indian Environmental Office and
Yvonne M. Watson of EPA's Office of Environmental Policy
Innovation as the technical advisor.

This report was developed under the Program Evaluation Competition,
co-sponsored by EPA's Office of Policy, Economics and Innovation
and the  Office of the Chief Financial Officer.  To access copies of this
or other EPA program evaluations, please go to EPA's Evaluation
Support Division's website at http:www.epa.gov/evaluate.

-------
                           TABLE OF CONTENTS
EXECUTIVE SUMMARY	ES-1

CHAPTER 1: INTRODUCTION                                                  1-1
      I.    Overview of The Tribal General Assistance Program	1-1
      II.   Purpose/Objectives of The Evaluation	1-5
      III.  Evaluation Questions	1-6
      IV.  Structure of The Report	1-8

CHAPTER 2: GAP EVALUATION METHODOLOGY                             2-1
      I.   Evaluation Design	2-1
      II.  Steps for Conducting the Evaluation	2-2
      III. Quality Assurance Procedures	2-18
      IV.  Strengths and Weaknesses of The Evaluation Design	2-18

CHAPTERS: GAP EVALUATION FINDINGS                                     3-1
      I.   Federally Recognized Tribes' Access to GAP	3-2
      II.  Tribal Utilization of Resources Provided by GAP	3-3
      III. Indicators of Tribal Environmental Capacity	3-12
      IV.  GAP Process Outputs to Achieve Tribal Goals and Priorities	3-26
      V.  GAP Support for EPA's Strategic Goal of Increasing Tribes'
          Ability to Build Environmental Program Capacity	3-28

CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS                       4-1
Appendix A: GAP Evaluation Questions
Appendix B:  Interview Guide for Regional GAP Project Officers
Appendix C:  Discussion Guide for Tribes
Appendix D: List of Tribes in Evaluation Sample
Appendix E:  GPRA Codes and Activity Categories
Appendix F:  Tribal Population Statistics
Appendix G: Activity Comparison Across Tribe Sample Groups
Appendix H: Regional GAP Project Officers Interviewed
Appendix I: Tribes Participating in Panel Discussions
Appendix J: Sources of Non-GAP Funding and Program Support

-------
EXECUTIVE SUMMARY

Introduction

The EPA is responsible for administering Federal environmental statutes on all U.S.
lands, including Indian country. The EPA recognizes tribal governments as the primary
parties for making environmental policy decisions  and implementing environmental
programs that affect Indian communities. GAP was established under the authority of the
Indian Environmental General Assistance Program Act of 1992. The primary purpose  of
GAP  is to help federally recognized tribes and intertribal  consortia build the  basic
components of a tribal environmental program, which may include planning, developing,
and establishing the administrative, technical, legal, enforcement, communication, and
outreach infrastructure.

In 2004, EPA's American Indian Environmental Office, which manages GAP, applied for
funding assistance  from  EPA's Office of Policy, Economics, and  Innovation  for a
program  evaluation to determine how effective GAP has  been in building  Tribal
environmental capacity.   For the  purpose  of this  evaluation,  Tribal environmental
capacity  is defined as administrative,  legal, technical  and enforcement  capability  of
Tribes to develop and implement a Tribal environmental program, and communications
capability to work with Federal, State, Local, Tribal,  and other environmental officials.
This evaluation is the result of that request and is designed to answer the following five
groups of questions:

   1)  Is the GAP being accessed by all federally-recognized tribes?  If not, why are
       some tribes not involved in GAP? Are there tribes that received GAP grants  at
       one time but which no longer receive GAP grants? If so, why?
   2)  Are tribal governments using the resources (technical, fiscal, and programmatic)
       provided as a component of GAP?
       a)  How often are they accessed?
       b)  How are tribes using these resources?
       c)  To what extent have tribes met program expectations for grants  management,
          execution of administrative functions, and carrying out proposed  activities?
       d)  How does participation in GAP increase understanding of the process required
          to develop a tribal environmental program?

   3)  What indicators of tribal environmental capacity exist?
       a)  To what extent have tribes achieved environmental capacity as  suggested by
          the presence of these indicators?
       b)  What factors contribute to the achievement  of environmental capacity, and
          what is the impact of these factors?
       c)  What is the relative contribution of GAP toward achieving capacity?

   4)  Is  the GAP process providing adequate outputs to achieve tribal goals and
       priorities?
                                      ES-1

-------
   5)  To what degree does GAP  support EPA's strategic goal  of increasing tribes'
       ability to build environmental program capacity?

Methodology

To  address  these questions,  the evaluation drew  on several  sources of information,
including existing databases file reviews for  a  sample of 111 tribes in  the nine EPA
regions with federally recognized tribes. Databases reviewed included:

       1.  GAP Accountability Tracking System, which contains records  from  a sample
          of GAP recipients who had received GAP funding prior to 2003 and provides
          documentation of their programming efforts under GAP;

       2.  Grants Information and  Control  System, which contains records of all EPA
          grants, including all GAP grants;

       3.  Audit Database, which contains information on audits of government grants to
          tribes and states, including audit findings and dates for EPA grants awarded to
          tribes; and

       4.  Strategic Goals Reporting System, which contains records on how GAP grants
          support  EPA's  Strategic Goal 5,  Objective  5.3, which is to  build  tribal
          environmental capacity.

The database reviews were  supplemented with reviews of regional  files containing grant
documents  (e.g., quarterly  reports  provided by  tribes funded under GAP) in order to
ensure adequate representation of tribal grants reviewed across  EPA regions.  Note that
GAP database  and file review data represents GAP grantee activity from October 2000 -
September 2004, and is therefore somewhat out of date.

In addition to database and file reviews, the evaluation was based on discussions with key
stakeholders, including  panel discussions with  tribal representatives at  three regional
tribal  meetings and interviews with  GAP  project officers in eight  regions.  These
discussions provided a more recent perspective on the extent to which GAP is supporting
development of tribal environmental capacity.

The draft evaluation methodology was peer reviewed by EPA, tribal representatives, and
academic evaluation experts.  The draft methodology was modified to address comments
from the peer reviewers, including changes such as:

   •   Inviting greater tribal input during the interviews of Tribal representatives;

   •   Asking tribal representatives about their definition of environmental capacity; and

   •   Assessing the extent to which organizations other than EPA, as well as program
       areas within EPA other than GAP,  have helped Tribes develop  environmental
       capacity.
                                      ES-2

-------
Findings

Federally Recognized Tribes' Access to GAP:

During  1994-2004, 89 percent the  561  federally recognized tribal governments in the
United States received at least one GAP grant. The tribes that did not receive GAP grants
had various reasons for not accessing this funding, such as:

   •   A policy of not accepting federal grant money

   •   Very small tribes without basic infrastructure to apply

   •   Non-reporting or fiscal mismanagement made some tribes ineligible

   •   In one instance, lack of regional staff to process applications.

   •   Lack  of GAP funding  hindered the development of these tribes' environmental
       programs (except for tribes with significant financial resources of their own).

Tribes' Use of GAP Resources

Between 1994 and 2004, the 111 tribes in the sample received an average of seven GAP
grants; on average the value of each  grant was  $102,472.  A majority (76 percent) of
tribes in the  sample accessed technical resources, such as workshops or training. GAP
facilitates  contact and  networking with regional tribal  staff, other tribes,  EPA media
program offices, and non-EPA agencies and organizations.   A minority (23 percent) of
tribes in the  sample accessed  programmatic resources, defined as GAP-specific grants
management  or fiscal administrative training.

How Tribes Are Using GAP Resources

Tribes use GAP funds  and technical and programmatic resources primarily to establish
and maintain  a tribal environmental presence in Indian country, which many tribes define
as having  a qualified staff  person available on  the  reservation to  respond  to
environmental issues of concern to their tribal council and members.  In addition, tribes
use GAP resources to participate  in  a variety of activities that  help  build their
environmental capacity and expand their environmental presence.   We examined the
types  of activities conducted by the 96 tribes in  our sample for which we were able to
obtain activity data from the either the GAP database or file reviews. Nearly all tribes
(98  percent) participated   in  activities  related  to the  general  management  and
administration of their environmental programs.  A majority of tribes also participated in
land activities (84 percent), water activities (73 percent), and grant writing activities (65
percent). A  smaller proportion of tribes conducted air  activities and special emphasis
activities.

A further analysis of tribal activities funded by GAP by activity  type shows  that more
than  80 percent of tribes  in the sample participated in program development  or
establishment, staffing, and communication activities.   Approximately  two-thirds  of
                                       ES-3

-------
tribes engaged in baseline assessment and grant writing activities.  Considerably fewer
tribes  conducted  activities  associated  with  media-specific  programs,  such  as the
development of Quality Assurance  Project  Plans  and  monitoring  capacity; the
development of legal tools such as codes, ordinances, standards, and permitting authority;
and the administration of grants received in support of these programs.  Only two tribes
participated in database development activities.

How Tribes Are Meeting EPA Expectations

The award of GAP grants brings with it  EPA's expectation that tribes will fulfill the
requirements of GAP for demonstrating accountability in the utilization of funds as well
as for  grants management  and  performance  reporting,  detailed in  the  2000  GAP
Guidelines.  Based on our interviews with regional POs, we found that, overall, tribes are
meeting regional expectations for grants management, the execution of  administrative
functions,  and  carrying out  proposed activities.   Tribes continue to improve the
timeliness, quality, and completeness of their  GAP  work  plans and progress reports.
Currently,  most tribes  in a  majority  of  regions are submitting their work plans and
progress reports on time.

We also examined the results of A-133 audits conducted for the 111 tribes in the sample
to assess tribes' ability to execute administrative functions.  Approximately 25 percent of
the  111 tribes in our sample had been audited during the period 1997-2004.  Of these 27
tribes, the audits for 24 resulted in at least one reportable condition, material weakness, or
material noncompliance outcome.  Audit findings mainly  cited problems  with tribes'
ability  to correctly track and document expenditures.  Note that because participation in
an A-133  audit is required only when a tribe's total annual expenditures of federal funds
exceed  a  high  threshold,  most of the tribes in the  sample would  not likely  have to
undergo such an audit.   As a result, the tribes in the sample that were audited and cited
with a reportable condition, material weakness, or a material non-compliance may not be
representative of the ability of the tribes not audited to execute administrative functions
pertaining to GAP grants.  It  may also be the case  that since A-133 audits include a
review of all federal expenditures for a tribe, the findings recorded in the Audit Database
for  tribes  in our sample may  not  be related to tribal  fiscal  performance under GAP.
Regional Project Officers for GAP identified only a few tribes that had received a major
finding on an Al33 audit.

Influence of GAP on Tribes'  Understanding of Environmental Program Development

In addition to assessing the direct resource outputs  provided by GAP, the evaluation
seeks to discern how tribal participation in GAP and utilization of GAP resources has
influenced, 1) tribes' understanding of the  process required to develop an  environmental
program, and, 2) the way tribes approach the various administrative  and  programmatic
functions associated with the development process.  Tribal representatives emphasize that
instead  of changing tribal understanding of how to develop an environmental program,
GAP facilitates tribes'  ability  to develop  a program that is responsive to  each tribe's
unique environmental conditions and priorities.  GAP  resources enable tribes to establish
an environmental presence, which in turn provides the foundation upon which each tribe
                                       ES-4

-------
can build an environmental program tailored to meet its needs. From EPA's perspective,
however, GAP may influence and clarify tribal priorities  as tribal environmental staff
acquire training,  learn  about specific  environmental conditions  on tribal lands, and
become more aware of concrete program opportunities through their interactions with
EPA regional tribal and media program contacts.

Indicators of Tribal Environmental Capacity

Many tribal representatives offered definitions of capacity and indicators that fall within
the GAP category of technical  capability, such  as:  hiring and  training of qualified
environmental professionals and expansion of tribal environmental programming efforts
to include media-specific components. A few tribes linked environmental capacity with
legal or enforcement capability.

Regional project officers identified many of the same key indicators of capacity as tribes,
such  as tribes'  ability  to  establish  an  environmental  presence;  retain qualified,
knowledgeable staff over the long-term; and diversify their environmental programming.

Achievement of Tribal Environmental Capacity

In  order  to determine  the  extent  to  which tribes  in  the  sample  have  achieved
environmental capacity as defined by  GAP, we examined tribal capability in each of the
five indicator areas - technical, legal, enforcement, administrative, and communication.
We identified a set of coded activity types for each indicator and equated tribal capability
in that area with a tribe's participation in one or more related activity.  Exhibit ES-1 lists
the activity types selected to demonstrate tribal capability for each indicator  and the
proportion  of the 96 tribes that participated in activities within each  type during 2000-
2004.
Exhibit ES-1: Tribal Achievement of Environmental Capacity, 2000-2004 (n = 96)
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental Capacity
Developed a Code, Ordinance, or Standard
Participated in an Activity to Increase Legal Capacity
Adopted a Code, Ordinance, or Standard
Participated in an Activity to Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to Increase Fiscal Administration Capacity
Participated in Internal Communication Activities
Participated in External Communication Activities
Participated in General Communication Activities
Number
of Tribes
25
24
7
25
86
70
70
47
15
68
66
28
Percent of
Tribes
26%
25%
7%
26%
90%
73%
73%
49%
16%
71%
69%
29%
                                       ES-5

-------
Factors that Contribute to the Achievement of Environmental Capacity

In addition to requesting tribal  and regional input on the most important indicators of
tribal environmental  capacity, we asked tribes and POs to  identify factors that impact
environmental capacity and  describe how they  influence tribal efforts.  As ranked by
project officers,  the following factors almost always or often influence environmental
capacity:

   •   Tribal council support for environmental programs
   •   Qualifications of tribal environmental director and/or staff
   •   Turnover rate of tribal environmental director and/or staff
   •   Clear tribal environmental priorities
   •   Degree of information sharing among tribes
   •   Access to funding outside of GAP.

Factors tribal representatives consider most influential include:
   •   Stability of knowledgeable tribal environmental staff
   •   Effective communication between tribes and EPA regions (and between the EPA
       regional tribal offices and media programs)
   •   Support of tribal council for planning and funding  environmental  programs
       relative to other tribal priorities.

Relative Contribution of GAP Toward Achieving Environmental Capacity

There are many  factors that  can potentially affect tribes' achievement of environmental
capacity such as GAP funding and technical assistance, the  stability of tribal leadership
and staffing,  and the degree to which Tribal council members focus  on environmental
concerns.  Another potential factor is tribal access to  other sources of funding (e.g., EPA
media programs, other federal and state agencies, and tribes themselves).

Interviews with  tribal representatives  and POs make clear that they perceive  GAP
funding as essential to achieving environmental  capacity.  Many tribes say that without
GAP funding, they would be able to do very little environmental work.  They stress that
GAP is the foundation for their environmental programs, and GAP resources enable them
to establish a basic program infrastructure, through which they can apply for other types
of environmental funding.  This view  supports a basic premise  of the GAP program,
namely, that as GAP helps tribes build their environmental capacity, tribes will be able to
access other sources of funding to support their environmental programs.

A comparison of tribes that first accessed GAP early in the program's existence (1994 -
1999) to those that first accessed GAP in later years (2000 - 2004),  shows that tribes that
accessed GAP earlier have acquired a greater proportion of non-GAP EPA funding
relative to GAP funding. This supports the view that GAP is helping tribes expand their
sources of environmental funding, which suggests that tribes have increased their
environmental capacity accordingly. However, tribes that accessed GAP earlier have not
achieved a greater number of indicators of environmental capacity with GAP funds
                                      ES-6

-------
compared to tribes that adopted GAP later. The results of this analysis run counter to the
hypothesis that tribes that have had GAP funding for a longer period of time would be
further along in the process of developing environmental capacity, compared to tribes
that had received GAP funding for a shorter period of time.
Sufficiency of GAP Outputs to Achieve Tribal Goals

Many of the tribal representatives interviewed stated that a key goal for tribes is having
an environmental presence on tribal land, i.e., a qualified staff person who can coordinate
the tribe's  environmental  programs, maintain a cohesive program, and be a point of
contact  for members of the tribal  community  and neighboring communities.   GAP
enables  tribes to establish this environmental  presence  by providing the funds to hire,
train, and retain professional and technical environmental staff.  Tribes emphasize that
GAP provides a foundation for tribal environmental programs.

While  tribal  representatives state  that GAP  funding  is  vital  for  establishing  and
maintaining an environmental presence, many perceive current levels of GAP funding as
insufficient.  When  asked  about additional  resources tribes need  to develop their
environmental programs, several tribes indicate that above all else, they need sustained,
consistent funding over time to  enable them to hire and retain sufficient qualified staff,
and thereby  retain institutional  knowledge.  GAP provides an important  source of
sustained funding, although  some tribes note that the requirement to re-apply for GAP
funding every year takes away from the stability of the  GAP program and the staff that
GAP supports.

Tribes sometimes have goals and priorities that GAP does  not address, either because
GAP  funding is  insufficient to meet  these goals,  or because these goals involve
implementation of environmental programs.  Most tribes interviewed say that in order to
meet tribal  goals,  they need to be able to use GAP funding for program implementation
and maintenance,  and they need additional funding to support this additional effort. In
addition, while  most  tribes perceive overall consistency between  GAP goals and tribal
priorities, they also pointed  out that it is difficult to mesh  the cultural and traditional
values  of the tribes  with the bureaucratic and regulatory  guidelines and definitions
established  by GAP.
GAP Support of EPA7 s Strategic Goals

EPA's 2003 - 2008 Strategic Plan includes Objective 5.3, the percent of tribes that "had
access to an environmental presence." This indicator increased from 36 percent in 1996
to an estimated  90.4 percent in FY 2006, with a peak of 97 percent in FY 2004. Access
to an environmental presence  is defined as the annual dollar value of GAP funding that
AIEO determines is needed to  establish an environmental presence.

EPA's  updated  2006 - 2011  Strategic Plan includes new targets for building program
capacity,  including:  increasing  the percent  of tribes  conducting  EPA-approved
                                      ES-7

-------
environmental monitoring and assessment activities and increasing the percent of tribes
with an environmental program.  Further data is needed to conclusively assess progress
toward these new targets.


Conclusions

Based on the findings from this evaluation, the evaluators conclude that:

   •   The extent of capacity-building  varies across indicator areas  for tribes in the
       sample that received GAP grants.  These tribes have relatively well-developed
       technical and communications capabilities, but less developed legal, enforcement,
       and administrative capacity.

   •   GAP  has  done  much in  recent  years  to  clarify  grant  expectations  and
       administrative requirements for tribes, and tribes in turn are increasingly meeting
       these expectations and requirements.

   •   Tribes report that restrictions on  GAP grants that preclude using GAP funds for
       program  implementation  are  now  hindering  tribal  environmental program
       development.

   •   Tribes emphasize that GAP funding is essential to achieving their environmental
       goals, but perceive that current levels of funding are insufficient to address tribal
       priorities.


Recommendations

The  evaluation team  makes the  following  recommendations for  EPA based  on the
findings and conclusions from the evaluation:

Consider developing a mechanism to support tribal program implementation.
   •   EPA HQ and Regions could  continue  to  promote and  expand the  use of
       Performance Partnership Grants (PPGs) by tribes.
   •   AIEO could eventually establish a second tier of GAP funding - "GAP plus"  - to
       fund program  implementation for those Tribes that show  they  have met  key
       indicators of capacity under GAP.
   •   Another approach could be to establish a block grant for tribes similar to those
       established for U.S. territories.

Consider working more directly with tribes and regions to enhance administrative, legal,
and enforcement capacity.
   •   To help build  administrative capacity, AIEO could coordinate with  regions to
       ensure that programmatic resources provided keep pace with tribal needs.
   •   EPA regions  could offer  legal support to  help  tribes enact their own codes,
       ordinances, and standards.
                                       ES-8

-------
    •   In cases where tribes feel that they cannot or do not wish to implement their own
       environmental laws and regulations, AIEO should consider developing a coherent
       plan for working with tribes to protect the environment, while respecting  tribal
       sovereignty.

Raise  awareness  of innovative  environmental policy  approaches that complement
traditional codes and standards.
    •   Tribes may benefit from a greater emphasis on pollution prevention education,
       self-certification, and compliance assistance inspections.
    •   Tribes could  leverage the  considerable  experience  of  EPA  and  states in
       developing innovative policy tools and approaches, as well as specific outreach
       materials.
    •   AIEO  and  regions  could help tribes by raising awareness of innovative policy
       approaches, readily available materials, and potential funding sources.

Acknowledge cross-cultural differences, and continue working with tribes to maintain a
respectful dialog.
    •   A key difference in perspective is that Tribes see GAP funding as an extension of
       EPA's trust responsibility.
    •   EPA views tribes as grantees  that must meet certain requirements to show that
       they are accountable for funds.
    •   To foster greater understanding, Tribes suggest hiring more Native Americans to
       serve as regional POs and tribal coordinators.
    •   More frequent site visits to tribes by AIEO and EPA regional program staff  could
       help underscore the diversity of tribal perspective,  priorities, and  approaches to
       environmental protection.

Track progress toward achievement of the new 2006-2011 strategic goals and targets.
    •   AIEO  needs to ensure that its data collection systems allow for the effective
       capture and tracking of indicators related to the updated strategic targets.
    •   AIEO  should consider the degree to which its proposed performance measures
       align with  tribal priorities  and perspectives, and  the  feasibility of tribes' of
       achieving them.
    •   The five-year cycle for setting strategic goals and targets may  be too short to
       effectively  track and  measure tribal progress.  AIEO  should consider keeping
       consistent goals for a longer period of time.
                                       ES-9

-------
CHAPTER 1:  INTRODUCTION

This  report summarizes  the  evaluation  of the tribal  General  Assistance Program  (GAP)
conducted between 2005 and 2007 by Industrial Economics, Inc (ffic).  The first section in this
chapter begins with an introduction and overview of the tribal GAP, including a logic model that
describes the program's design.  The next section describes the purpose and objectives of the
evaluation, and provides a review of the specific questions the evaluation is designed to answer
and the relationship  between the evaluation questions and elements of the logic model.  The
chapter concludes with a description of the structure of the evaluation report.

I.  OVERVIEW OF THE TRIBAL GENERAL ASSISTANCE PROGRAM

The  EPA is responsible for administering Federal environmental statutes on all U.S. lands,
including Indian country.   The EPA recognizes tribal governments as the primary parties for
making environmental policy decisions and implementing  environmental programs that affect
Indian communities.  GAP was established under the authority  of the  Indian Environmental
General Assistance Program Act of 1992.  The primary purpose of the GAP is to help federally
recognized tribes and intertribal consortia  build the basic components of a tribal environmental
program, which may  include planning, developing, and establishing the administrative, technical,
legal,  enforcement,  communication,  and outreach  infrastructure. Total  GAP  funding  has
increased from initiation of the  program,  starting at a total of $5.4 million  in  1992 to $60.4
million in 2005, although funding for 2006 was projected to decline slightly to $56.9 million.1
As shown in Exhibit  1-1, average funding per tribe has ranged from a low of $26,738 in 1996 to
a high of $106,623 in 2004. Note that in  recent years (2005 and 2006), the average amount of
funding per tribe has declined.  Note also that not all eligible  tribes receive GAP funding, and
funding is not distributed evenly between tribes, so these figures provide only a general idea of
the level funding  available to tribes over time.   Chapter 3 provides further details about the
amount of funding provided to GAP grantees.
       1  Data is drawn  from the Goal  5,  Objective  5.3  Reporting System,  available  online  at
https://iasint.rtpnc.epa.gov/TATS/tats_prv/tats securitv.login form?p mode=reports.   under  Target 1  Program
Performance Report. Last accessed April 2007.
                                           1-1

-------
Exhibit 1-1: History of GAP Funding*
Year
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
Total GAP
Funding
$15,000,000
$28,000,000
$38,500,000
$42,000,000
$42,000,000
$52,000,000
$52,000,000
$56,150,000
$60,991,000
$60,404,000
$56,900,000
Number of
Federally
Recognized Tribes
561
562
565
565
567
572
570
572
572
572
572
Average Amount of
Funding per Tribe
$26,738
$49,822
$68,142
$74,336
$74,074
$90,909
$91,228
$98,164
$106,628
$105,601
$99,476
Percent Increase (Decrease) from Prior
Year in Average Amount of Funding
Per Tribe
—
86%
37%
9%
0%
23%
0%
8%
9%
(1%)
(6%)
*Data on number of eligible entities, and therefore amount of funding per tribe, is not available for 1992 to 1995.
The GAP provides annual grant funding to federally recognized tribes and intertribal consortia
through a negotiated process administered by each EPA region. The grant funds may be used by
tribes to plan and carry out any number of capacity-building activities including the development
of administrative procedures; quality assurance/quality control systems; sampling and laboratory
capabilities; baseline  environmental assessments;  enforcement programs;  legal procedures;
communications plans; computer information systems; and staff qualifications and expertise.
GAP may not be used for the ongoing implementation of media-specific environmental programs
once established, with the exception of solid waste program implementation activities.  Since
GAP funds may generally not be used for implementation activities, the GAP program defines its
outcomes as changes in knowledge  and behavior (i.e., short-term and intermediate outcomes, as
described below).

To illustrate the various components of the GAP, EPA and lEc developed a logic model (i.e., a
graphical  representation of the  relationships between program inputs,  outputs, and intended
outcomes), presented in Exhibit 1-2.  Key components include the following:

•  Resources are the basic inputs of funds, staffing, and knowledge dedicated to the program.

•  Activities/Outputs are the specific  actions taken  to  achieve  program  goals and the
   immediate products that result.  Under the GAP, these products include grant funds, technical
   assistance, training, and grant oversight.

•  Customers are the users of the activities and  outputs (fiscal, technical, administrative)
   provided.  They are the tribal governments that receive GAP grants  and the environmental
   employees hired with GAP funds.
                                          1-2

-------
•   Short-Term Outcomes are changes in awareness, attitudes, understanding, knowledge, and
    skills  resulting  from  program  outputs.    Technical  and  grant  management  training
    opportunities  provided to  tribal  environmental  employees  through the  GAP  increase
    understanding of the processes required in developing a tribal environmental program.  Note
    that outcomes listed in italics are intended as illustrative examples of the intended effects of
    tribes' increased understanding for how to develop a tribal environmental program.

•   Intermediate Outcomes involve changes in behavior that  are broader in scope than short-
    term outcomes.  Intermediate outcomes often build upon the progress achieved in the short-
    term. Under the GAP, changes in tribal  awareness, understanding, and skill level pave the
    way for planning,  development, and initiation of capacity-building activities.  The  logic
    model includes  examples of activities  that represent  increased  capability in the legal,
    enforcement, technical, communications, and administrative  arenas.  Note that outcomes
    listed in italics are intended as illustrative examples of the  effects of tribes' increased legal,
    enforcement, technical, communications, and administrative capability.

•   Long-Term Outcomes  parallel  the overarching  goals  of the  program  and  are the
    environmental  improvements  and public health benefits  that flow from the behavioral,
    procedural,  and operational changes.

•   Contextual/External Variables are factors, not directly controlled  by the program or its
    entities, which may affect program performance.  For example, changes in tribal policy and
    budgetary priorities may influence the ability of tribes to sustain environmental efforts.

Elements of the logic model, noted by the circled letter codes, are referenced in section III in the
discussion of the evaluation questions.
                                           1-3

-------
Exhibit 1-2.  GAP Grant Program Evaluation Logic Model
Legend: Letter codes are used to connect elements of the logic
model to the evaluation questions, as shown in Exhibit 1-3.
i Resources
i Activities ! j Outputs ! j Customers

i Outcomes i
Short-Term I
Intermediate
Long-Term
Provide
Funding to
Tribes

.
©
GAP Grants


.
k.
^
Tribal
Executives
Inter-tribal
Consortia
Executives

Agency
Technical
Expertise
A
Dedica
Staffin
FTE-
Regior
Nation
Progra
Manag

k
ted
g;65
AIEO,
is, and
al
m
ers


Prov
Tecr
Assi
Trib
ide
stance to /D^N
3S \ 	 J
r
^
w
Pro's
Rec
Gra
Ove
Trib


External Factors:/ s
tribal leadership, (*-'
vision, continuity,
priority of
environmental issues,
education levels, staff
turnover, and
resource levels.
Assumptions: GAP
funding facilitates
ability of tribes to
acquire other EPA
grants that also
contribute to
capacity-building.

/ide
pient/
it ^^^
rsight to £ '
es ^~—S
L
Technical and Environmental
Media-Specific \~W\ Employees
Trainings fn\ Funded by GAP

Site Visits

Grant
Management
Trainings

GAP Data
Collected and
Performance
Tracking
Performed

J



k
_fe
Increased
understanding of the
process required in
the development of a
tribal environmental
program.
Increased
awareness of
multiple media
program areas.
Increased
knowledge of grants
management
requirements and
approaches.
                                                        Tribes develop legal and
                                                        enforcement infrastructure, i.e.,
                                                        codes, regulations, ordinances, and
                                                        standards that can be used to
                                                        implement management policies
                                                        and guidelines.
                                                        Tribes demonstrate ability to
                                                        perform the inventories,
                                                        monitoring, and inspected needed
                                                        to ensure compliance with
                                                        environmental policies and
                                                        guidelines.
                                                        Tribes develop technical skills for
                                                        environmental management such as
                                                        monitoring and analysis, baseline
                                                        assessment, data management,
                                                        quality assurance procedures, and
                                                        emergency response systems.
                                                        Assumption or creation of media-
                                                        specific programs.
                                                        Tribes demonstrate ability to
                                                        communicate about environmental
                                                        issues with the community, tribal
                                                        executives, the regulated
                                                        community, and other government
                                                        entities.
                                                        Tribes   establish   ability   and
                                                        procedures   for   managing   and
                                                        accounting   for  program  funds,
                                                        including procedures for  staffing
                                                        and training, management of office
                                                        resources   and  personnel,   and
                                                        communication  with other tribal
                                                        government agencies.
                                                                                           Planning for tribal codes
                                                                                           and ordinances
                                                                                           Ratification of tribal
                                                                                           codes and ordinances
                                                                                           Certification of tribal
                                                                                           programs
                                                                                          Approval of tribal grant
                                                                                          work plans
                                                                                          :::,
                                                                                           Public education and
                                                                                           outreach programs
                                                                                           Established
                                                                                           communication with other
                                                                                           players
                                                                                           Established  staffing  for
                                                                                           media-specific
                                                                                           Acquisition of equipment
                                                                                          Reductions in
                                                                                          questionable audits
                                                                                                                              Established capacity to plan,
                                                                                                                              develop, implement, and
                                                                                                                              manage environmental
                                                                                                                              programs.
                                                                                                                                   Achievement of EPA's
                                                                                                                                   Strategic Plan Section 5.3
                                                                                                                                   Goals
                                                                                                                              Compliance with federal
                                                                                                                              statutes and regulations

                                                                                                                              Sustainability of tribal
                                                                                                                              environmental programs
                                                                                                                              Improved environmental
                                                                                                                              conditions in Indian Country
  ( K J
     Consistent Communication between AIEO, regions and tribes about Challenges and Performance

-------
II.  PURPOSE/OBJECTIVES OF THE EVALUATION

Since the passage of the Government Performance and Results Act of 1993, federal agencies are
expected to regularly conduct evaluations of their programs as part of a larger effort to promote
results-oriented government.2 EPA's 2003-2008 Strategic Plan reflects a "sharpened focus on
achieving measurable results" and notes that program evaluations  are  used, "to identify areas
needing improvement, more effective strategies for achieving established goals, and ways to
improve data collection or better measure program results."3  One area where EPA is seeking to
measure its results is in its tribal program.

In 2004, EPA's American Indian Environmental Office (AIEO), which manages GAP, applied
for funding assistance from EPA's Office  of Policy, Economics, and Innovation (OPEI) for a
program evaluation to be conducted by an independent evaluator. This evaluation is the result of
that request.  AIEO's  goal in seeking this evaluation support was to, "... determine how effective
GAP has been in building tribal environmental capacity with those tribes receiving funds[,]...to
see if we are reaching the tribes to change their knowledge and behavior, and to determine how
permanent those changes are."4  The 2000  GAP guidelines  establish elements of a core tribal
environmental protection program.   These include  "establishing]  the administrative,  legal,
technical and enforcement capability of tribes to develop  and implement a tribal  environmental
program...[and] [establishing a Tribal communications capability to work with Federal, State,
Local, Tribal, and  other environmental  officials."5  Having capability in each of these areas
constitutes having environmental capacity for the purpose of this evaluation.

AIEO  is also considering whether to expand the GAP beyond its current mandate of building
tribal environmental  capacity.  Before proceeding with an expansion  of the GAP, AIEO has
decided to  evaluate the existing program with regard to its  impact on tribal capacity development
and its relevance to tribal needs and to Section 5.3 of EPA's 2003-2008 Strategic Plan.

Industrial  Economics, Inc.  (ffic) was  selected to assess the  impact  of the GAP on tribal
environmental capacity and  help  AIEO  understand which elements  of GAP contribute to the
establishment of multi-media  environmental  programs.   The  results  of this  evaluation are
intended to help the tribal  program demonstrate its successes to  stakeholders and identify
opportunities for improvement.  The evaluation results are expected to be of particular interest to
GAP stakeholders involved in awarding grants, providing technical assistance and oversight, and
planning and executing tribal environmental  programs.   Participation by tribal  environmental
         National Research Council, Decision Making for the Environment:  Social and Behavioral Science
Research Priorities, Washington, D.C.: National Academies Press, 2005.

       3 U.S. Environmental Protection Agency (EPA), 2003-2008 EPA Strategic Plan: Direction for the Future.
Washington, D.C., EPA, 2005.

       4 American Indian Environmental Office (AIEO), Improving Results: Program Evaluation Competition
Application, Unpublished document provided by AIEO, 2004.

       5 U.S. Governmental Protection Agency (EPA), Indian Environmental General Assistance Program:
Guidelines on the Award and Management of General Assistance Agreements for Indian Tribes. Washington, D.C.,
EPA, 2000.
                                            1-5

-------
staff in the evaluation process is intended help build EPA's awareness of tribal priorities and
progress toward  achieving environmental goals.   The results  are  expected help  AIEO and
Regional GAP project officers more effectively target outreach to tribes and identify the level of
resources (administrative, financial, and technical) necessary to promote and sustain  tribal
environmental initiatives. Tribal environmental managers may find the evaluation results useful
in influencing community  decision-makers  and  further raising the  profile of environmental
protection  programs among tribal  members.  More broadly, the evaluation results will  be  of
interest to policy planners at  EPA Headquarters tracking  progress  on the objectives EPA's
strategic plan.6

III. EVALUATION QUESTIONS

The evaluation is designed to answer the following five groups of questions:

1) Is the GAP being accessed by all federally-recognized tribes? If not, why are some tribes not
   involved in GAP? Are there tribes that received GAP grants at one time but which no longer
   receive GAP grants? If so, why?
2) Are tribal governments using the resources (technical, fiscal, and programmatic) provided as
   a component of GAP?
   a)  How often are they accessed?
   b)  How are tribes using these resources?
   c)  To what extent have tribes met program expectations for grants management, execution
       of administrative functions, and carrying out proposed activities?
   d)  How does participation in GAP increase understanding of the process required to develop
       a tribal environmental program?

3) What indicators of tribal environmental capacity exist?7
   a)  To what extent have tribes achieved environmental capacity as  suggested by the presence
       of these indicators?
   b)  What factors contribute to the achievement of environmental  capacity, and what  is the
       impact of these factors?
   c)  What is the relative contribution of GAP toward achieving capacity?

4) Is the GAP process providing adequate outputs to achieve tribal goals and priorities?
5) To what degree does GAP support EPA 's strategic goal of increasing tribes' ability to build
   environmental program capacity?
        6 This evaluation was specifically designed to assess progress on Section 5.3 of EPA's 2003-2008
strategic plan. After this evaluation was designed, EPA issued an updated 2008-2011 Strategic Plan, with updated
goals and targets for AIEO. While the evaluation was not specifically designed to address progress toward these
updated goals and targets, Chapter 3 does describe findings from this  evaluation in relation to the 2008-2011
Strategic Plan.

       7 The order of the three sub-questions listed for this evaluation question has been changed in Chapter 3 to
facilitate the presentation of the evaluation findings.
                                            1-6

-------
Exhibit 1-3 lists the final evaluation questions and the components of the logic model to which
they correspond.
Exhibit 1-3:
Relationship Between Evaluation Questions and Logic Model
Evaluation Question
la. Is GAP being accessed by all federally recognized tribes?
Ib. Why are some tribes not involved in GAP?
Ic. Are there tribes that received GAP grants at one time but
which no longer receive GAP grants? If so, why?
2a. Are tribal governments using the resources (technical, fiscal,
and programmatic) provided as a component of GAP? How
often are GAP resources accessed?
2b. How are tribes using GAP resources?
2c. To what extent have tribes met program expectations for
grants management, execution of administrative functions,
and carrying out proposed activities?
2d. How does participation in GAP increase understanding of
how to develop a tribal environmental program?
3 a. What indicators of tribal environmental capacity exist?
3b. To what extent have tribes achieved environmental capacity
as suggested by the presence of these indicators?
3c. What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
Tribal Priorities
Tribal Staffing
Tribal Funding
Communication
Regional Activities
3d. What is the relative contribution of GAP toward achieving
capacity?
4. Is the GAP providing adequate outputs to achieve tribal goals
and priorities?
5. To what degree does GAP support EPA's strategic goal of
increasing tribes' ability to build environmental program
capacity?
Component of the Logic Model
©
(A)
©
© © ©
© © © ® © ©
©
©
® © ® ® ® ®

©
®
® ©
®
(M)
Overarching question
Not directly shown in logic model
®
                                           1-7

-------
IV. STRUCTURE OF THE REPORT

Following this introduction, Chapter 2 of the report presents the methodology used in conducting
the evaluation, including  study design, data sources, a plan for data analysis, quality assurance
procedures, and strengths  and weaknesses of the methodology. Chapter 3 presents the evaluation
findings, organized by the five evaluation  questions described in section III above. Chapter 4
presents the conclusions resulting from the evaluation findings and recommendations  to AIEO
for future improvements to GAP.
                                           1-8

-------
CHAPTER 2:  GAP EVALUATION METHODOLOGY

This  chapter  summarizes  key  aspects  of the methodology  used to evaluate the  General
Assistance Program (GAP).  The methodology begins with an overarching evaluation design,
followed by a series of tasks undertaken in conducting the evaluation, including gathering and
analyzing data, interpreting findings, and reporting results.  The methodology also addresses
quality  assurance  procedures used,  and comments  on the strengths  and weaknesses of the
evaluation design.

I.  EVALUATION DESIGN

Most program evaluations,  including the present study, are designed to address two overarching
questions: 1) what are the program's outcomes, effectiveness, and impacts; and 2) how or why is
a  program  effective  or ineffective.1  It  is typically difficult to answer the first question
definitively because it is often not possible to infer causal relationships between programs and
measured  outcomes  and  long-term impacts.   Although  randomized,  controlled  trials  or
experiments can isolate causal  effects, it  is very rare to be able to conduct a randomized,
controlled study of government environmental programs. For example, in the present evaluation,
in order to develop a randomized, controlled trial, it would be necessary to award GAP grants to
only a subset of tribes chosen on a random basis, and then compare the outcomes of tribes that
did receive grants to those that did not. Such a design would be impractical and questionable on
legal and ethical grounds.

Since it is not possible  to conduct a randomized,  controlled experiment for this evaluation, we
chose an alternate evaluation design.  In this case, our evaluation design options were limited.
For example, we could not conduct direct  controlled trials, since we had no ability to control
variables (e.g.,  the turnover rate and qualifications of tribal environmental  staff, or  the grantee
caseload for EPA project officers) that might affect program outcomes.  Moreover, we could not
choose  a quasi-experimental design, since that methodology  relies  on selecting comparison
groups (i.e., comparing tribes that received  a GAP grant to those that did not).  It is not feasible
to make this comparison since the  vast majority  of eligible tribes have received GAP grants.
Those that have not received such funding  are atypical and therefore would not present a valid
basis of comparison (e.g.,  they  are very small tribes, and thus may  have inherently different
abilities to achieve environmental capacity with or without GAP grants).

In light of these limitations, we conducted a non-experimental direct analysis.  This type of
evaluation examines only the subject group receiving the "intervention" (in this case, a GAP
grant).  Our evaluation incorporated aspects of a longitudinal study, which examines conditions
of the study group over time. Specifically, we compared tribes that had GAP grants for a longer
period of time compared to  tribes that had GAP for a shorter period of time, to see whether tribes
       1  Office  of Management  and  Budget (OMB),  What  Constitutes Strong  Evidence  of a  Program's
Effectiveness?  Washington, D.C., OMB, 2004.   Available from http://www.whitehouse.gov/omb/part/2004
program eval.pdf. Accessed July 2005.
                                           2-1

-------
with more years of GAP funding had more effectively developed environmental capacity.  For
tribes that  had GAP  grants for a shorter period of time, we also  compared  an indicator of
capacity  (percentage of EPA funding derived from sources other than  GAP) before  and after
GAP grants  were awarded, to  see  if access to GAP grants led to  an increase in funding
diversification.

This evaluation addresses short term and intermediate  outcomes achieved by GAP.  However,
GAP grants  are  limited  to  capacity-building  and  are  not intended  to  cover  program
implementation (with the exception of solid and hazardous waste programs.)2  Therefore, this
evaluation focuses on short-term and intermediate outcomes (i.e.,  changes in tribal knowledge
and  behavior), rather than long-term outcomes,  such  as environmental  and public health
improvements, that would be achieved only through the implementation of tribal environmental
programs.
II.  STEPS FOR CONDUCTING THE EVALUATION

The four major steps taken to conduct this evaluation include:  1) identifying the information
needed  to  answer the evaluation questions, 2) collecting  and  analyzing data from  existing
databases and files, 3) collecting and analyzing data from interviews panel discussions, and 4)
reporting results and conclusions.  Exhibit 2-1 lists the detailed tasks completed under each of
these steps.  The four steps are then discussed in more detail in the following sections.
        The purpose of GAP grants, which are defined by statue, are explained in the Indian General Assistance
Program 2006 Grant Administration Guidance, available at http://www.epa.gov/indian/pdfs/gap2006.pdf.  Accessed
April 2007.
                                           2-2

-------
                       Exhibit 2-1: Steps for Conducting the Evaluation
      A. Identify Information Needed and Prepare for Data Collection:
          1)  Establish data indicators.
          2)  Identify data sources.
          3)  Develop data collection tools (guides for interviews with EPA regional staff and
             focus groups with tribal representatives).
      B. Collect and Analyze Data from Existing Databases and Files:
          1)  Select a sample of tribal GAP recipients.
          2)  Develop an evaluation database.
          3)  Collect quantitative and qualitative data from databases and grantee file reviews.
          4)  Categorize and code qualitative data.
          5)  Summarize data through descriptive statistical analysis.
      C.  Collect and Analyze Data from Interviews and Panel Discussions:
          1)  Identify regional GAP coordinators to interview.
          2)  Schedule and conduct interviews.
          3)  Select a sample of tribes for participation in group interviews.
          4)  Schedule and conduct group interviews.
          5)  Code responses from regional GAP interviews.
          6)  Analyze trends and patterns in data from interviews and group interviews.
      D.  Prepare Final Evaluation Report, in Accordance with EPA Guidelines:
          1)  Introduce GAP program and the purpose of the evaluation.
          2)  Describe methods for data collection and analysis.
          3)  Summarize key findings from quantitative and qualitative data analyses, and consider
             relationship between quantitative  and qualitative findings.
          4)  Develop conclusions and identify lessons learned.
          5)  Propose recommendations.
A. Identify Information Needed and Prepare for Data Collection

In preparing this evaluation methodology, ffic identified specific types of information that could
help answer each evaluation question (Appendix A).  lEc developed this list of data needs in
consultation with staff from AIEO and regional GAP Project Officers  (POs).  Indicators  of
environmental capacity were initially selected based on the statutory definition of tribal capacity,
which was informed by tribal perspectives at the time that the statute was written (although tribal
perspectives may have changed since that time). AIEO and the regional  POs offered feedback
on key indicators of tribal environmental capacity, guidance in defining the overarching indicator
of capacity for tribes, and insight into the factors that may potentially impact tribes' achievement
of environmental capacity.  Tribal representatives had an opportunity to  review and comment on
the evaluation  methodology,  which  included a  description  of the  proposed indicators  of
environmental capacity. The final indicators of capacity were  selected  with all of this input in
mind.
                                             20
                                            -3

-------
Regional  POs  also identified several databases that could  supply needed data  on tribal
environmental programming efforts, including:

       1.  GAP Accountability Tracking System (GAP database) - This database, maintained by
          AIEO, contains records from a sample of GAP recipients who had received GAP
          funding  prior to 2003  and provided  documentation of their  programming efforts
          under GAP.  The database  records include  details on the grants awarded, activities
          conducted with GAP funds, and positions funded by GAP during the period October
          2000 through September 2004.3 A total of 92 tribes  are included in one or more of
          the tables in the GAP database, representing at least  20 percent of tribes in eight of
          EPA's ten regions.4

       2.  Grants Information and Control System (GICS) - This database, maintained by EPA's
          Office of Administration and Resources Management, contains records of all EPA
          grants, including all GAP grants. We received a pull of all GICS records of GAP and
          other EPA grants awarded to tribes from  1994 - 2004.  The GICS  data includes for
          each grant, the tribe name;  region; award amount and date project end date, budget
          end  date, and closeout date;  and a description.5

       3.  Audit Database - (http://harvester.census.gov/sac/dissem/entity.html).  This database,
          maintained by the U.S. Census Bureau, contains information on audits of government
          grants to tribes and States, including audit findings and dates for EPA grants awarded
          to tribes.

       4.  Strategic  Goals   Reporting   System   (5.3S-D)   -  (https://oasint.rtpnc.epa.gov/
          TATS/tats_prv/entry_page). This database, maintained by AIEO, contains records on
          how GAP grants support EPA's Strategic Goal 5, Objective 5.3, which is to build
          tribal environmental capacity. The database contains  records on the number of tribes
          per  region that have achieved Treatment as a State  (TAS), Direct Implementation
          Tribal  Cooperative Agreements (DITCA), GAP  Grants, Quality Assurance Project
          Plans (QAPPs), Tier III Tribal-EPA Environmental Agreements (TEAs), Performance
          Partnership  Grants (PPGs),  and  other agreements   (such   as  Memoranda  of
          Understanding (MOUs), Memoranda of Agreement  (MOAs),  and  Tier I & Tier II
          TEAs).
       3 This time period corresponds with GAP work plan fiscal years 2000, 2001, 2002, and 2003. Unlike a
federal fiscal year, which is named for the year ending in September, a GAP work plan year is named for the year
beginning in October. For example, GAP work plan year 2000 began on October 1, 2000 and ended on September
30, 2001.

       4 According to AIEO, 20 percent of tribes in most EPA regions were selected for inclusion in the GAP
database, however, where needed, additional tribes were selected to make sure that at least two tribes were included
in the GAP database from each Region.

       5 In addition, POs recommended that we also review the IGMS  database, which is a subset of the GICS
database. We received and reviewed a data pull from IGMS of grants made to tribes from 1992 to 1999.  However,
since GICS contains more complete records, we used the data pull from GICS in lieu of the IGMS data.
                                           2-4

-------
In addition, AIEO and POs provided suggestions for other sources of information to supplement
the data found in existing databases, including:

   •   Reviews  of regional and/or tribal files,  such as grant documents, correspondence, and
       related materials;

   •   Discussions with AIEO staff;

   •   Interviews with regional EPA staff that oversee GAP grants;

   •   Interviews with tribal representatives; and

   •   Review of EPA's strategic plan as it relates to GAP.

For each type of information needed to answer the evaluation questions, lEc worked with AIEO
and the regional  POs to identify the data source(s) most likely to provide pertinent information.
Finally, lEc developed data collection tools to use in gathering data during interviews with POs
and panel discussions with tribal representatives.  These interview and discussion guides were
designed to collect the types of information that could not be gathered through an analysis of
existing databases, and are attached to this methodology as Appendices B and C.

B. Collect and Analyze Data from Existing Databases and Files

The existing databases provided by AIEO include many of the types of information needed to
answer the evaluation questions.  The databases include both quantitative information (e.g., the
dollar  amount of each grant), which can be summarized mathematically, as  well as narrative
descriptions (e.g., types of activities conducted with  GAP funds).  The GAP  database represents
a sample of tribes, thus, we  used  this existing  sample and augmented it with information on
additional tribes, as described below.

Sample Selection

The GAP database provided the  starting  point for our sample selection,  since this database
provides information specifically on GAP-funded activities conducted by tribes, without which
we could not address the evaluation questions.  AIEO initially populated the database from a
random selection of tribes in Regions 1, 2, 4, 5, 6,  7, 8, and 9 that  had received GAP funding
prior to 2003  and provided documentation  of their programming efforts under GAP.   AIEO
chose their sample for the GAP database by sampling at an equal rate within  each EPA region to
ensure that the representation of tribes in the database is consistent with the distribution of tribes
in different parts of the country.  (In Region 10, only a few tribes  were included in the GAP
database, representing only one percent of the  tribes in the GICS database in the region that
received GAP grants.  There are  no federally  recognized tribes in Region  3, so none were
included in the  GAP database from this region.)   We felt  it was  important to maintain this
geographic representation in the sample selected  for this evaluation, since tribes in different parts
of the  country  vary in size, resources,   and  other characteristics that  may  affect tribal
environmental capacity.
                                           2-5

-------
Two tables within the GAP database - the Activity and Position tables - contain information
needed to help answer the evaluation questions pertaining to use of GAP  funds.  As shown in
Exhibit 2-2, a total of 66 tribes that are included in the GICS database are listed in both of these
tables.6  These 66 tribes represent eight EPA regions and  13 percent of the 500 tribes that have
received  GAP grants between 1994 and 2004,  according  to the GICS database.7  Note that
Regions 2 and 10 are considerably underrepresented in  the Activity and Position tables.  Were
these regions excluded from the analysis, the records in the GAP database would represent 22
percent of tribes that received GAP grants between 1994 and 2004.  In order to provide each
region with approximately equal representation in the evaluation sample, we set the final number
of tribes to be selected from each region as close to 22 percent as possible.  For some  regions (2,
9, and 10) with fewer tribes in the Activity and Position tables, we randomly selected additional
tribes not included in the GAP database to include in the final sample. We used grant files and
related documents for these additional tribes obtained from the POs to fill in the types of data
that would otherwise be included in the GAP  database.  For other regions (5, 6, 7,  and 8), we
randomly selected from among tribes in the Activity and Position tables to ensure approximately
22 percent of each region's tribes would be included in the sample.  Appendix D lists the tribes
included in our sample.

Where records in  the GAP database included in  our sample turned  out  to be incomplete  or
inadequate for a  particular question, we did not analyze the data for that tribe for that question.
The same held true with file reviews:  if a file had incomplete data, we analyzed the available
data,  rather than select a different record for  inclusion in the sample.  We characterized any
significant data gaps as part of our analysis.
       6 There is one tribe included in the GAP activity and position tables that is not included in the GICS
database. Since information from all three of these sources is important to the analysis, we did not include this tribe
in our sampling pool, and it is not reflected in the totals in Exhibit 2-2. Also note that we did not included intertribal
consortia in our sample (although they are  represented in the GICS and GAP databases), since including these
consortia could result in double-counting of tribes that received GAP both independent of and as part of a consortia.

         Note that Region 10 tribes eligible for GAP grants, and therefore included in this analysis, do not include
Alaska Native regional or village corporations.  The reason for this exclusion is explained in 2000 GAP  Guidelines,
Indian Environmental  General Assistance  Program Guidelines  on  the Award and Management  of General
Assistance Agreements for Indian Tribes, available at http://www.epa.gov/indian/pdfs/gap2000.pdf last accessed
May 2007.
                                             2-6

-------
EXHIBIT 2-2: COUNT OF TRIBES RECEIVING GAP GRANTS
COMPARED TO COUNT OF TRIBES IN THE GAP DATABASE, BY REGION
EPA Region
1
2
4
5
6
7
8
9
10
Total
Total excluding
Regions 2 and 1 0
Number of
Tribes
Receiving GAP
Grants 1994-
2004
-------
from existing databases and that gathered from file reviews.8 A list of the codes used is included
in Appendix E.

We used statistical methods to summarize and analyze the data in the existing databases.  Most
of the evaluation  questions could be  answered by descriptive statistics - including the range,
mean,  median, standard deviation,  and/or proportion - for key  indicators.   For example,  to
answer the question, "How often are GAP resources accessed?" we considered the percentage of
tribes in our sample that participated in technical and programmatic training opportunities. We
also used  statistics to draw  inferences  about the total  population of tribes receiving GAP
grants.9'10 For  qualitative  data,  such  as EPA media program  categories  or staff titles,  we
calculated counts and percentages to describe patterns and identify trends.

The one evaluation question  that cannot be adequately addressed  by  descriptive statistics is,
"What is the relative contribution of GAP toward the achieving  environmental capacity?"   In
order to answer this question, we compared tribes that had GAP grants for a longer period of
time ("early adopters") to tribes that had GAP for a shorter period of time ("late adopters"), to
see  whether  tribes  with   more  years  of GAP  funding had  more  effectively  developed
environmental capacity, as  measured  both by the percent of EPA funding attributable to GAP
and achievement of specific indicators of environmental  capacity.  For late adopters, we also
compared the percentage of EPA funding derived from sources other than GAP before and after
GAP grants were awarded, to  see  if access to GAP grants led  to  an increase in funding
diversification.

Exhibit 2-3 summarizes each type of information that we gathered from the  existing databases
and the type of analysis we  conducted for each type of data. The primary unit of analysis in this
evaluation is the tribe.  We used descriptive  statistics to summarize the status of tribes with
regard to  evaluation questions 1, 2, and  3a.   For the sake of brevity in the table, we refer to
calculating "averages" and "percentages."  Where it is noted that we calculated an average, we in
fact calculated the sample range, mean, median, and standard deviation, in order to provide a full
       8 The coding categories can be found in Appendix A of the report EPA American Indian Environmental
Office Gap Accountability Tracking System, Developed for AIEO, draft  October 6, 2003.  The document was
provided by AIEO to lEc.
       9 Our statistical calculations assumed that we had obtained a large or approximately random sample of the
population. In cases where these conditions were not met, we did not develop inferences about the whole population
of tribes.
       10 We analyzed all tribes across Regions in a single set of descriptive statistics. Where it seemed relevant,
we also calculated separate sets of descriptive statistics for tribes in each Region.

-------
description of the data.  We also estimated the population mean, which  is reported with the
confidence level and confidence interval.  Where it is noted that we calculated a percentage, we
calculated the sample proportion as well as an estimate of the population  proportion, which is
reported with the confidence level  and  confidence  interval.   The results  of this inferential
analysis are included in Appendix F.
                                           2-9

-------
                                 EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
       Evaluation Question
                 Analytic Approach'1
                                                                                Analytical Method
la. Is GAP being accessed by all
  federally recognized tribes?
•   Number and percentage of federally recognized tribes
    that have received GAP funds between 1994 and 2004.
                                                        •    Calculate the number and percentage of tribes based on the total
                                                            population of GAP grants in the GICS database, rather than a sample.
2a. Are tribal governments using the
   resources (technical, fiscal, and
   programmatic) provided as a
   component of GAP? How often
   are GAP resources accessed?
    Amount and type of GAP resources that have been
    delivered to and accessed by tribes:

    •  Fiscal resources: GAP funding provided to tribes

    •  Technical resources: Technical assistance and
       media specific trainings

    •  Programmatic resources: Grants management
       training
                                                       •   Calculate average amount of grant award(s).

                                                       •   Calculate percentage of tribes that have received technical assistance
                                                           and media specific trainings among GAP-funded activities.

                                                       •   Calculate percentage of tribes that have received programmatic
                                                           resources (e.g. grants management training) and calculate the average
                                                           number of grants management trainings described among each tribe's
                                                           GAP-funded activities.
2b. How are tribes using GAP
    Tribal staff and activities funded through GAP
   resources >.
                                                            Categorize and code activities, and then calculate percent of tribes
                                                            conducting each type of activity.  Include solid waste implementation
                                                            in GAP-funded activities.
2c. To what extent have tribes met
  program expectations for grants
  management, execution of
  administrative functions, and
  carrying out proposed activities?
•   Timing of grant end date vs. final close out of the grant
    - according to regional POs, the shorter the period of
    time between grant end date and final closeout, the
    more likely that tribes met expectations.   Note that
    other factors such as EPA project officer turnover, lack
    of FTE, or lack of EPA's emphasis on closeouts could
    affect this indicator.

•   Results of administrative post award monitoring audits.
                                                            Calculate the average number of months between grant end data and
                                                            final closeout data for those grants with an action code "FC" in the
                                                            GICS database, meaning the grant has been closed out.

                                                            Calculate the percent of tribes that were audited and had a reportable
                                                            condition, material weakness, and/or material non-compliance finding
                                                            as a result of the audit.
                                                                             2-10

-------
                                 EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
       Evaluation Question
                 Analytic Approach'3
                    Analytical Method
3b. To what extent have tribes
   achieved environmental capacity
   as suggested by the presence of
   indicators of environmental
   capacity?

Note, the indicators of
   environmental capacity were
   selected based on the purpose of
   GAP as described in the GAP
   program guidelines as well as
   input from regional POs early in
   the evaluation process and
   feedback from tribal
   representatives on the draft
   evaluation methodology.
Overarching Indicator of Environmental Capability

•   The sequence of GAP and non-GAP grant funding
    secured by GAP recipients over time
Describe the sequence of GAP funding relative to non-GAP funding
received by tribes; calculate the number and percentage of tribes that
received non-GAP funding before, during, and after receiving GAP
funding.(b)	
Legal Capability
•   GAP recipients that have developed tribal codes,
    standards, and/or enforcement programs to control
    pollution

•   GAP recipients that have adopted or implemented
    tribal codes, standards, and/or enforcement programs
    to control pollution
Query the activities field in the database to identify developing codes,
standards, and similar activities in the database, and then calculate the
number and percentage of tribes that mention these among their GAP
activities. To the extent possible, distinguish between codes,
standards, etc. that have been developed versus those that have been
adopted or implemented.
Enforcement Capability
•   Presence of tribal environmental staff person(s)
    charged with enforcement duties
                                                                                              Calculate the number and percentage of tribes for which records show
                                                                                              enforcement or inspection among the activities funded by GAP, or
                                                                                              that have a position description that mentions enforcement or
                                                                                              inspection.	
                                   Technical Capability

                                   •   GAP recipients with one or more staff specifically
                                       tasked with managing environmental programs

                                   •   Size and composition of tribal environmental staff
                                   •   Environmental programs being carried out in different
                                       media annually by tribes
                                   •   GAP recipients that have taken environmental training
                                                           Calculate the percentage of tribes that have a position of
                                                           "Environmental Director" or an equivalent term, or that have a
                                                           position with a "Professional" category code.

                                                           Calculate average number of staff members listed by each tribe in the
                                                           sample, and the percentage of overall positions that are classified as
                                                           Professional, Administrative, Technical, Legal,  Clerical, or Other.
                                                           Query the database to identify activities associated with air, water,
                                                           and waste programs, and then calculate the number and percentage of
                                                           tribes conducting each type of activity.
                                                           Calculate the number and percentage of tribes that list environmental
                                                           training as among their activities.	
                                                                            2-11

-------
                                EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
       Evaluation Question
             Analytic Approach'3
                                                                              Analytical Method
                                  Communications Capability
                                  •   GAP recipients that have conducted community
                                      education and outreach, based on the grant work plan

                                  •   GAP recipients that have executed agreements with
                                      other jurisdictions for management of on- or off-
                                      reservation resources

                                  •   Tribal participation in EPA or tribal workgroups and/or
                                      Task Forces
                                                       Calculate the number and percent of tribes that list community
                                                       education, outreach, or similar terms among their activities.

                                                       Calculate the number and percent of tribes that list inter-governmental
                                                       agreements among their activities.

                                                       Calculate the number and percent of tribes that list participation in
                                                       workgroups or task forces among their activities.
3c. What factors contribute to the
  achievement of environmental
  capacity, and what is the impact
  of each factor?
Tribal Funding
•   GAP funding amounts
                                      GAP funding consistency over time
                                                       Calculate average amount of GAP award.
                                                   •    Calculate the average number of consecutive GAP grants received per
                                                       tribe over time.
3d. What is the relative contribution
  of GAP toward achieving
  capacity?
Non-GAP grant funding amounts
                                      Sequence of EPA grants received by tribes
                                                       •  Calculate average amount of funding from non-GAP, EPA grants.
                                                          Compare percentage of non-GAP EPA funding between early and late
                                                          adopters, and for late adopters, before and after receiving a GAP
                                                          award.

                                                       •  Calculate the percent of tribes that received GAP funding before
                                                          securing other funding. Note, we do not have data to assess funding
                                                          from outside of EPA, e.g. grants from BIA or tribes' internal funds,
                                                          which limits the conclusions that we can draw.
                                                                          2-12

-------
                                EXHIBIT 2-3: ANALYSIS OF INFORMATION GATHERED FROM EXISTING DATABASES
       Evaluation Question
            Analytic Approach'3
                    Analytical Method
5. To what degree does GAP support
   EPA's strategic goal of increasing
   tribes' ability to build
   environmental program capacity?
Summary of GAP goals and objectives

Summary of EPA's strategic goal 5, objective 5.3, and
related targets for building tribal capacity

Summary of progress towards EPA's strategic targets
under objective 5.3, as reported in the Strategic Goals
Reporting System

Summary of findings on the extent to which GAP
appears to support tribes' development of
environmental capacity.
Review GAP goals and objectives contained in EPA's Indian
Environmental General Assistance Program: Guidelines on the
Award and Management of General Assistance Agreements for Indian
Tribes (U.S. EPA, 2000) snA Report to Congress: The Indian
Environmental General Assistance Program (U.S. EPA, 2001)

Compare Goal 5.3, Build Tribal Capacity, and other strategic goals
with relevance to tribes that are included in EPA's 2003-2008
Strategic Plan (U.S. EPA, 2003) with the range of tribal
environmental programs documented in the EPA 5.3S-D.
(a) Information to be collected from GAP database, GICS, Audit Database, and 5.3 S-D, supplemented by File Reviews for Regions 2, 9, and 10.
(b) Securing funding in addition to GAP has been identified as both an indicator of tribal environmental capacity and a factor influencing achievement of environmental capacity
by regional POs. Through interview discussions and focus groups, we will try to understand whether securing other sources of funding is more a cause or effect of success in the
GAP program.
                                                                          2-13

-------
A potential source of bias in our analysis is the selection of tribes for our sample based on the
availability of comprehensive tribal records. If the tribes with data in the Activity and Position
tables tend to differ systematically from tribes that are not represented in these tables, then our
sample would not be truly representative.  This could also be  true if tribes in the GAP database
are  not  represented in the Activity and Position tables because their records are incomplete,
suggesting that  perhaps these tribes have less  administrative  capacity  than  those that are
included.   On the other hand, if the Activity and Position tables are incomplete for reasons
unrelated to tribal characteristics (e.g., the time and resources available to EPA  to populate the
database) there might not be a bias in our sample.  To assess this potential bias, we compared
tribes in our sample that are not included in the Activity and Position tables to tribes that are
included in those tables to see  if they  differ significantly for the analysis of activities funded by
GAP. The results of this analysis are included in Appendix G.  Overall, the tribal participation
rates for different types of activities funded by GAP seem similar for tribes included in the GAP
database tables and those for which we conducted file reviews. The only substantial difference is
that tribes assessed through the file reviews appear to conduct  more communication and baseline
assessment activities than  tribes included in the database analysis.  We do not believe this
difference is sufficiently large  to limit our ability to draw inferences  about the whole population
of GAP grants. Due to the scope of the evaluation, it was not possible to conduct further tests on
the GAP database to rule out potential sources of bias.  However, we reviewed the description of
the methodology used in preparing the database, and we did not find any notable  sources of bias.
We are aware that AIEO is making an effort to improve its data tracking over time, and the GAP
database was developed as  a  prototype  of a more comprehensive  database that AIEO is now
compiling.

C. Collect and Analyze Data from Interviews with Regions and Tribes

To complement the analysis of data from the EPA databases, we conducted individual interviews
with EPA staff from eight regions and panel discussions with representatives from an array of
tribes.   As discussed above, we  used the data  from existing databases to  develop an initial
analysis of the extent of tribes' use of GAP, how  tribes are using GAP resources, the degree to
which tribes are meeting GAP grants management expectations, the  extent to which  tribes have
achieved environmental capacity as demonstrated by  key indicators, and the degree to which
different factors may be associated with key indicators  of environmental capacity. However, our
ability to accurately answer the evaluation questions was greatly enhanced by soliciting feedback
on and interpretation of this initial analysis by regional POs and tribal representatives who have
personal experience issuing and  implementing  GAP grants.  We  presented POs and  tribal
representatives with an  abbreviated  summary of our  analysis and asked  for their  help in
understanding the data, and identifying any situations where they felt the data may be misleading
or incomplete. In addition, we  relied on POs and tribal  representatives to provide  information for
a number  of questions that are not addressed at all in the existing databases.  Our approach to
sample selection, data collection, and data analysis are described in detail below.

Sample Selection

We conducted one-hour telephone interviews with EPA POs  located in  eight regions, and  one
cross-regional representative.  (The names of specific interviewees are included in Appendix H.)
In order to be sure that the interviewees had sufficient  experience with GAP grants to be able to
                                           2-14

-------
answer  our questions, we interviewed  current POs if they  had a minimum of two years of
experience working with GAP grants in the region they represent.  If a current PO did not have
this minimum level of experience, we interviewed a former PO for that region, if available.

We also conducted 2-3 hour panel discussions with representatives from a subset of nine tribes
that have received GAP grants in the past five years.  The interviews were held in conjunction
with existing gatherings of GAP grantees, in order to facilitate participation of tribes.   Each
session was scheduled for a different part of the country, to provide geographic diversity among
the tribes participating.  The tribal panelists were selected by the regional EPA staff organizing
each meeting to achieve two  objectives: to include different types of tribes and ensure that the
representatives had detailed knowledge of GAP grants and tribal environmental programming.

In addition to tribal representatives on the panel, we invited other tribes attending the gathering
to attend the session if they  wished.   For each  set of interview questions answered by the
panelists,  ffic opened  the discussion  to  include comments  and  questions from the tribal
representatives assembled in the audience.  This approach  allowed for a greater range of tribal
input on the interview questions for those tribes that  wished to provide comments.  We also
provided each invited tribal representative with the set of interview  questions in advance, so they
could solicit feedback from other tribes in their region if they wished.  The names of tribes
participating in the panel discussions are included in Appendix I.11

We recognize that relying on EPA to select tribal representatives from among attendees  at an
EPA-sponsored GAP event did not provide a sample representative of the population of GAP
grantees; we assumed that the tribes that participate in such an event are more likely to be active
partners in GAP  and other EPA programs than tribes  that eschew them.  Tribes that actively
collaborate with EPA and other tribes  on GAP  projects may have different experiences and
perspectives than those that do not.  Despite this likely source of bias from the group interviews,
we believe this  approach was  the most feasible way  to gather information  from tribal
representatives.  Moreover, while recognizing the limitations of this proposed sample of tribal
representatives, we believe that this group was able to provide insights and illustrative examples
about how GAP grants work  for tribes that are especially engaged with EPA  on  the  GAP
program.

Data Collection and Analysis

The  interviews and panel  discussions address aspects  of the evaluation questions that are not
covered or cannot be fully  addressed in  our analysis of the EPA  databases (see Exhibits 2-4 and
2-5).  In addition, these forums provided POs and tribal representatives with the opportunity to
offer their interpretations of and feedback on the results  of the  preliminary analysis we conducted
with information  from the  EPA databases.  Interviews  with POs included structured and open-
ended questions;  Appendix B includes the interview guide for POs.   Appendix C includes the
tribal discussion guide.
       11 Due to inclement weather, we were unable to conduct a session with tribal representatives in the Eastern
region of the U.S. We instead conducted individual telephone interviews with each of the three panelists selected
for that session.
                                           2-15

-------
EXHIBIT 2-4:
INFORMATION GATHERED IN INTERVIEWS WITH REGIONAL GAP PROJECT OFFICERS
Evaluation Question
Ib. Why are some tribes not involved
in GAP?
Ic. Are there tribes that received
GAP grants at one time but which
no longer receive GAP grants? If
so, why?
2c. To what extent have tribes met
program expectations for grants
management, execution of
administrative functions, and
carrying out proposed activities?
3 a. What indicators of tribal
environmental capacity exist?
3b. To what extent have tribes
achieved environmental capacity
as suggested by the presence of
these indicators?
3c. What factors contribute to the
achievement of environmental
capacity, and what is the impact
of each factor?
5. To what degree does GAP support
EPA's strategic goal of increasing
tribes' ability to build
environmental program capacity?
Information Sought from Regional Project Officers to Help Answer
Evaluation Questions
• Perceptions as to why tribes may not seek GAP funding;
perceptions on ability of tribes to initiate environmental programs
without it
• Perceptions as to why tribes may have dropped off the GAP grant
rolls
• Regional POs' perceptions about the quality, timeliness, and
completeness of work plans and progress reports received
Administrative Capability:
• How long has the tribal staff person tasked with managing
environmental programs (e.g. Environmental Director) been in that
position?
• Major findings on A133 audits
• Results of on-site grants management review
• Degree of tribal council support for environmental programs
• Does tribe have clear environmental priorities?
• Rate of change in tribal governments - how often do Council
members change?
• Turnover rate of tribal Environmental Director and/or staff
• Qualifications of tribal Environmental Director and/or staff
• Degree of information sharing among tribes
• Degree to which tribes request information from EPA
• Structure of EPA regional office - specifically, at what level the
tribal office located (in the administrators office or elsewhere)
• Experience and longevity of POs
• How do GAP's goals currently align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• How do GAP's objectives align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
• How do tribal GAP activities align with or diverge from EPA's
strategic goal for GAP and other multi-media programs?
2-16

-------
                                           EXHIBIT 2-5:
     INFORMATION GATHERED IN PANEL DISCUSSIONS WITH TRIBAL REPRESENTATIVES
       Evaluation Question
 Information Sought from Tribal Representatives to Help Answer
	Evaluation Questions	
 Ib. Why are some tribes not involved
   in GAP?
    Perceptions as to why tribes may not seek GAP funding;
    perceptions on ability of tribes to initiate environmental programs
    without GAP
2d. How does participation in GAP
   increase understanding of how to
   develop a tribal environmental
   program?
     Serf reported increase in knowledge and understanding about the
     necessary steps in developing a tribal environmental program
     Serf-reported increase in skills needed to develop tribal
     environmental programs
     Serf-reported change in awareness and commitment to
     environmental programs in tribes	
3 a. What indicators of tribal
   environmental capacity exist?


3b. To what extent have tribes
   achieved environmental capacity
   as suggested by the presence of
   these indicators?
  •  Tribes' perceptions of key indications of environmental capacity.

 Communications Capability:
 •    Extent of tribal environmental staffs communication of with tribal
      Council

 Other capabilities (legal, enforcement, technical, administrative) not
 fully addressed in the analysis of EPA databases.
3c. What factors contribute to the
   achievement of environmental
   capacity, and what is the impact
   of each factor?
      Degree of tribal Councils' support for environmental programs
      Do tribes have clear environmental priorities?
      Rates of change in tribal governments - how often do Council
      members change?
      Turnover rates of tribal Environmental Director and/or staff
      Qualifications of tribal Environmental Director and/or staff
      Degree of information sharing among tribes
      Availability of non-grant tribal funding	
3d. What is the relative contribution
   of GAP toward achieving
   capacity?
      Sequence of grants received by tribes (i.e., do tribes receive GAP
      grants before other grants? If so, this would suggest that GAP
      may contribute to ability to receive other grants)
      Availability of non-grant tribal funding	
4. Is the GAP providing adequate
   outputs to achieve tribal goals and
   priorities?
     Tribes' perceptions about whether GAP is providing the type and
      amount of resources that they need to meet their environmental
      goals and priorities
     Are there additional resources that tribes feel they would need to
      have in order to address their goals and priorities?
     Is environmental capacity building a priority for tribes receiving
      GAP grants?
The first  step  in analyzing the  data collected through the interviews  and  panel discussions
involved broadly categorizing and summarizing responses and using them to  qualify the results
of our quantitative analysis.  We also captured comments that, in our judgment, synthesize the
views expressed by interviewees and panel discussion participants. In this way, our analyses and
conclusions reflected the insights and perspectives provided by POs and tribal representatives.
                                                2-17

-------
D. Prepare Final Evaluation Report

This report constitutes the final evaluation report, which has been prepared in accordance with
EPA guidelines.

III. QUALITY ASSURANCE PROCEDURES

Two parts of this evaluation required a quality assurance review.  We first needed to ensure that
the way we  coded data during our review of tribal GAP files was consistent with the coding of
data already in the GAP database. To meet this requirement, we used the same coding categories
used by EPA to create the GAP database, and we reviewed these categories with AIEO staff to
be sure we understood how to interpret them.  We created a standardized code sheet to use for all
file reviews, and  coordinated  between lEc  staff conducing the reviews to  ensure coding
consistency  (Appendix E).  Finally, in cases where we had questions about interpretation, we
referred the questions to AIEO staff.

The second part of this evaluation requiring a quality assurance review is the analysis itself (e.g.,
calculation of descriptive statistics, populations  proportions, and population means, as well as
characterization of qualitative information gathered in the interviews and panel discussions).  All
quantitative  computations and analyses have been reviewed by at least one manager to ensure
accuracy.  With regard to the qualitative information from interviews  and panel discussions, lEc
used interview guides to ensure consistency in the way we asked questions during discussions
with EPA staff and tribal representatives. lEc also had a staff person present to type notes during
each interview (this recorder was usually not the  same person who conducted the interviews). In
most cases the notes recorded were comparable  to a transcript (although some information was
summarized to keep pace with the flow of the conversation).  We compiled the  interview notes
into summary documents, and grouped together responses to each interview question to facilitate
their  characterization.   Where interviews produced quantitative information (e.g., ranked
responses to  questions  posed along  a  Leichart   scale),   we  analyzed  these  responses
mathematically.   We made the draft summary of our results  available to POs and tribal
representatives  that  we  interviewed to  allow them to correct  any  inaccuracies in   our
interpretation of their comments, however no corrections were submitted.
IV. STRENGTHS AND WEAKNESSES OF THE EVALUATION DESIGN

The strengths  of this analysis are that it answers the specific evaluation questions posed by
AIEO, and it draws on both quantitative and qualitative data to answer them.  The methodology
underwent extensive  peer review (with comments provided by EPA staff, tribal representatives,
and academic  reviewers)  and we modified  the  evaluation design in light of the comments
received.  Academic  reviewers noted that while the evaluation would not meet the requirements
of a formal  academic evaluation, the analytical  rigor is  appropriate for the budget and time
available to conduct it.

The final evaluation methodology varies  only slightly from the  proposed  methodology that
underwent peer review.  The most significant change is that the original methodology proposed
conducting a  regression analysis to assess  the  correlation  between index of environmental
                                          2-18

-------
capacity and key indicators, such as the amount of GAP funding.  In retrospect, pursuing this
analysis would have been ill advised.  Based on what we heard in interviews, we believe there
are many confounding factors (e.g., tribal population and land base, degree of cohesiveness of
tribal land holdings, and tribal socio-economic status) that could influence a tribe's achievement
of environmental capacity, in addition to the indicators that we can measure, such as the amount
of GAP funding. Accounting for all these factors was beyond the scope of this investigation.
We opted instead to compare early vs. late adopters of GAP, to see if tribes' access to GAP for a
longer period  increased the degree  to which they had  achieved  capacity. If true, this would
suggest that access to GAP contributes to tribes' ability to develop environmental capacity.

A key limit of this methodology, both in its original and final form, is that it does not prove that
GAP caused the changes in tribal environmental capacity described in  Chapter 3.  However,
qualitative  interviews  suggest that GAP has indeed been an essential  resource in helping tribes
build their  environmental programs. Another limitation of our analysis is that since the tribes
included in the  panel discussions are not necessarily representative  of  all GAP grantees, we
cannot rely on information from these sessions to make inferences about GAP grantees as a
whole. However, given  the time and budget limitations of the analysis, and U.S. Information
Collection Request (ICR) restrictions, we were not able to interview a larger sample of tribes that
would be representative of GAP grantees as a whole.

Although this  evaluation design significantly limits the types of conclusions and our ability to
generalize  them  to the larger  population of GAP grantees, in  our judgment, it  is the best
methodology available given the characteristics of the program under evaluation.  Moreover, this
evaluation design is well adapted for understanding how and why a program is effective, which
can provide useful information for program management.
                                          2-19

-------
CHAPTER 3: GAP EVALUATION FINDINGS

This chapter presents  the  findings from  EC'S evaluation of the  General Assistance Program
(GAP). The findings are based on a review of records for a sample of 111 tribes awarded GAP
grants, stored in four federal databases:  the  EPA GAP  Accountability Tracking System  and
Strategic Goals  Reporting  System,  both maintained by EPA's American Indian Environmental
Office (AIEO); the Grants  Information and Control System (GICS) maintained by EPA's Office
of Administration and Resources Management; and the Audit Database maintained by the U.S.
Census Bureau.  Where needed, records in the databases were supplemented by file reviews of
GAP progress reports maintained by EPA  regions.  Statistics presented in the narrative are based
on the sample of  111 tribes.   Appendix F  includes inferences  based on this sample  about
characteristics for the entire population  of GAP grantees.   Information  obtained from these
databases, with the exception of the GAP  database, span the years 1994-2004,  unless otherwise
noted.  Due to the limitations of the version of the GAP database used for this evaluation, tribal
activity and position data only cover the  period October  2000 through September 2004.l   The
activity table in the database contains records of discrete activities conducted by tribes during
this period but does not track the overall  effort and time expended by  tribes  to carry out each
activity.2

In addition to information  from the databases and  file reviews,  these findings are informed by
interviews with EPA project officers (POs), and a series of panel  discussions and interviews with
tribal representatives.  Information  from  these  interviews and  discussions is drawn from the
experience of the interviewee, which encompasses recent years and may date back to  2000 or
earlier.

The findings are organized  around the five evaluation questions, as follows:

    I.  Is the  GAP being accessed by all federally recognized  Tribes?  If not, why are some
       Tribes not involved in GAP? Are  there Tribes that received GAP grants at one time but
       which no longer receive GAP grants? If so, why?
   II.  Are  Tribal  governments using the  resources (technical, fiscal,  and programmatic)
       provided as a component of GAP?
          A.  How often are they accessed?
          B.  How are Tribes using these  resources?
          C.  To what extent have Tribes met program  expectations for grants  management,
              execution of administrative  functions, and carrying out proposed activities?
       1 This time period corresponds with GAP work plan fiscal years 2000, 2001, 2002, and 2003.  Unlike a
federal fiscal year, which is named for the year ending in September, a GAP work plan year is named for the year
beginning in October. For example, GAP work plan year 2000 began on October 1, 2000 and ended on September
30, 2001. Thus, the time period for which GAP data are available extends from October 2000 through September
2004.
       2 This data limitation may create the impression that tribes which reported fewer activities in a given
program area were less active in building capacity than tribes which reported more activities in that area, however,
certain capacity building activities are ongoing, and thus may represent a substantial effort for the tribe even if they
are only counted in the database once for any given reporting period.
                                            3-1

-------
          D.  How does participation in GAP increase understanding of the process required to
              develop a Tribal environmental program?

  III.  What indicators of Tribal environmental capacity exist?
          A.  To what extent have Tribes achieved environmental capacity as suggested by the
              presence of these indicators?
          B.  What factors contribute to the achievement of environmental capacity, and what is
              the impact of these factors?
          C.  What is the relative contribution of GAP toward achieving capacity?

  IV.  Is the GAP process providing adequate outputs to achieve Tribal goals and priorities?
   V.  To what degree does GAP support EPA's strategic goal of increasing Tribes' ability to
       build environmental program capacity?

Based  on these findings, the next chapter discusses recommendations and conclusions of the
evaluation.
I.  FEDERALLY RECOGNIZED TRIBES' ACCESS TO GAP

GAP is utilized by a significant majority of federally recognized tribes. From 1994-2004, 500 of
the 5613 federally recognized tribal governments in the United States received at least one GAP
grant.  This means that approximately 89 percent of tribes received a GAP grant during this time
period.4

lEc interviewed EPA POs from eight regions, who work  directly with tribes to administer their
GAP  grants.   We asked POs whether  they  knew of any federally recognized tribes  in their
regions that had not received GAP grants since 1994, and if so, why these tribes did not receive
GAP funding.  The POs indicated that only a few tribes in their regions had not received GAP
grants.  Tribes in four regions had adopted a policy of not accepting federal grant money.  In
addition, POs in two regions shared their perspective that some tribes have not applied for GAP
funding because they are too small and  do understand how to use GAP, or because they do not
have the basic infrastructure to apply for the  funding. POs in two other regions explained that
tribal performance issues, such as non-reporting or fiscal mismanagement, had made some tribes
ineligible for EPA funds.  Lastly, in one  region, there was  a time when the regional office did not
have sufficient staff to process all incoming GAP applications, and the region ended up denying
grants to tribes whose proposals it deemed insubstantial.

We also asked POs whether any tribes in their regions had at one time received GAP grants, but
no longer receive GAP funding, and if  so, what caused them to drop off the GAP grant rolls.
       3 Bureau of Indian Affairs (BIA). 2007. Bureau of Indian Affairs.  Washington, D.C., BIA. Available
from http://www.doi.gov/bureau-indian-affairs.html. Accessed April 2007.
       4 These results were obtained through an analysis of EPA's Grants Information and Control System (GICS)
database records between 1994 and 2004, and are based on an analysis of GAP funding provided to all 561 federally
recognized tribes.
                                           3-2

-------
Many POs responded that one or more tribes in their region had received a GAP grant between
1994 and 2004, but had since dropped off the GAP grant rolls. At least two tribes in one region
had their own financial resources  from successful casinos and  decided not to apply for GAP
funding in order to free up that funding  for other tribes. In another region, staff turnover left
some small tribes without staff to manage their GAP grants. In a few cases, performance issues
or fiscal mismanagement left tribes temporarily ineligible for GAP funding. Once tribes resolved
these  issues they became eligible for GAP grants again. Finally, one tribe lost its federal
recognition, and thus was no longer eligible for GAP funding.

We then asked POs  if the lack of GAP grants  hindered the development of environmental
programs for tribes that did not receive GAP grants.  The POs explained that  except for those
tribes with significant financial resources of their own, the lack of GAP funding did hinder the
development  of  tribal environmental programs.  Tribes without GAP funding or substantial
revenue streams of their own have not been able to devote sufficient effort to their environmental
programs. These tribes missed networking opportunities  and had to rely on EPA for permitting
and enforcement. One PO explained,  however, that the temporary loss  of GAP grants due to
fiscal mismanagement ultimately benefited certain tribes, because it led them to reevaluate their
efforts and make administrative improvements to access GAP in the future.
II. TRIBAL UTILIZATION OF RESOURCES PROVIDED BY GAP

GAP provides funding as well as technical and programmatic assistance to tribes.  Technical
assistance may include: EPA-sponsored training, linking tribal staff with EPA media program
contacts, EPA review of tribal proposals for establishing programmatic capability, and site visits.
For the purpose of this evaluation, we  have defined programmatic assistance as any EPA-
sponsored training or outreach  directed toward improving tribal administration of GAP grants.
One  objective of the evaluation  is to determine whether tribes  are accessing the array of
resources (fiscal, technical, and programmatic) provided as a component of GAP, and how often
they access these resources.  The evaluation also seeks to assess how tribes' participation in GAP
and their use of GAP resources  influences the way tribes approach developing and administering
their environmental programs.

A. Tribes Access of GAP Resources

GAP funds received by tribes constitute the most critical resource provided by GAP.  The 111
tribes in the sample received a  total of 754 GAP grants between 1994 and 2004. The funding
amounts awarded for these grants ranged from $1,918 to $423,000; the mean GAP award for the
sample of tribes was $102,472.  These funds have supported various components of a core tribal
environmental  program, including salaries  for environmental  program staff, access to training
and networking opportunities, equipment, outreach and education, and contract services.  GAP
funds have remained important to tribal environmental programming efforts over time, even after
tribes secured additional funds from other EPA and non-EPA sources. Exhibit 3-1 illustrates the
number of GAP  grants received by tribes.   Just over three quarters of the tribes (79 percent)
received nine or  fewer GAP grants during this 10-year period.  The maximum number of GAP
grants received by a tribe was 19; the average across tribes was seven.
                                          3-3

-------
              Exhibit 3-1: Number of GAP Grants Received by Tribes, 1994-2004 (n=lll)

Qno/, _
JU
C" 9Wn -
&
£
P&
H

-------
and outreach available to tribes. Some regions hold an annual GAP conference at which GAP
administrative training may or may not be provided, while  other regions routinely offer GAP-
specific workshops.  One region indicated that it has had more success with assisting tribes with
the fiscal administration of GAP on a one-on-one basis than through formal training.

B. Tribes' Use of GAP Resources

Tribes use GAP  funds and  technical and programmatic  resources primarily to establish  and
maintain a tribal environmental presence in Indian country, which many tribes define as having a
qualified staff person available on the reservation to respond to environmental issues of concern
to their tribal council and  members.  In addition, tribes use GAP  resources  to participate in a
variety of activities that help build their environmental capacity and expand their environmental
presence.  We examined the types  of activities conducted by the  96 tribes  in our  sample for
which we were able to obtain activity data from the  either the GAP  database or file reviews.
Exhibit 3-2 presents the percentage of tribes that participated in different categories of activities.5
Nearly all tribes (98  percent) participated in activities related to the general management and
administration of their environmental programs.  A majority of tribes also participated in land
activities (84 percent), water activities (73 percent), and grant writing activities (65 percent). A
smaller proportion of tribes conducted air activities and special emphasis activities.
       5 Activities listed in the GAP database are organized into categories that correspond to six main program
areas. Within each category, activities are grouped into more narrow subcategories. For example, land activities
may include activities associated with asbestos, emergency response, Superfund, hazardous waste, lead, pesticides,
solid waste and recycling, or underground storage tanks (UST).  Activities conducted within each subcategory may
be further classified by type (in Exhibit 3-3), such as general program development,  staffing, communication,
baseline  assessment,  development of monitoring capacity, development of codes, ordinances, or standards,
developing permitting/licensing authority, development of QAPPs, grant administration, and database development.
For a complete list of categories and subcategories, see Appendix E.
                                              3-5

-------
            Exhibit 3-2: Activities funded by GAP, Organized by Major Category, 2000-2004 (n = 96)
g, 100% -,
&
s
_ 80%
fj
«
a
.=
& ^»0%
•e
u
^
•C
SS 4no/,
s
.Q
L^ 90%
o
1
V
PH 0%








































General Land Activities Water Activities Grant Writing Air Activities Special Emphasis
Management and Activities
Administration Category of Activity
Exhibit 3-3 shows tribal participation in activities further classified by activity type. More than
80  percent  of tribes  participated in program  development  or establishment,  staffing,  and
communication activities.  Approximately two-thirds of tribes engaged in baseline assessment
and grant writing activities.  Considerably fewer tribes conducted activities  associated with
media-specific programs, such as the development of Quality Assurance Project Plans (QAPPs)
and monitoring capacity; the  development of legal tools such as codes, ordinances, standards,
and permitting authority; and the administration of grants received in support of these programs.
Only two tribes participated in database development activities.
                                            3-6

-------
         Exhibit 3-3: Activities funded by GAP, Organized by Activity Type, 2000-2004 (n = 96)
100% -,
13
u
-S 60% -
w
^g
^
•-C
u
£
*£ 9n%

H
=g 0% J











































































	












	






















	
















1


n n n n
II II II II 1 1


1 |5l 1 1 | 1  ^ S * °si^c^cg a gg
£ £s| | i i £ is |i| |& |l.^ i it
ill I j a 1 ft ill P 111 1 ^
S | Q§Q^° ig

Activity Type
We also asked tribes directly for details about the activities they conduct with GAP funding.  The
activities tribes mentioned are consistent with the categories and activity types described above,
although many also reflect the specific interests and environmental priorities of individual tribes.
For example, one tribe performed an assessment of medicinal plant species on the reservation.
Another  has conducted environmental  education  integrated with tribal  culture and  language.
Noting the  subsistence  diet of its members, another tribe  used GAP funds to address water
quality and increased mortality rates  potentially linked to the consumption of contaminated fish.
A number of tribes have also used  GAP to participate  in training  and planning  activities  for
emergency  response programs  that include  natural disaster  assistance,  such as forest  fire
prevention.

The  range of activities  conducted by tribes  suggests both the depth and breadth of capacity-
building  within and across program areas.   One  example is water quality programs.  Tribes
recalled using GAP funds for wetlands surveys, riparian zone protection activities, surface water
and  well monitoring,  and obtaining staff  certifications for drinking water and wastewater
treatment systems.  A few tribes mentioned using GAP to hire contractors to conduct specialized
activities  including hazardous waste cleanups,  water sampling and analysis, program planning
for solid waste and emergency management, and electronics recycling.

Under the general  program management  and administration category, tribes have engaged in
activities to  further establish their legal and enforcement capability.  These activities include the
                                           3-7

-------
development of ordinances for solid waste and recycling, open burning, zoning, and underground
storage tanks, as well as the establishment of water quality standards.  GAP has enabled at least
one tribe to issue wetlands permits and conduct its own inspections to ensure compliance with
tribal regulations. For tribes that do not have permitting or enforcement authority for programs
such as  UST,  Underground  Injection   Control  (UIC),  or the National  Permit Discharge
Elimination System (NPDES),  GAP provides funds for tribal environmental staff to accompany
federal inspectors and gain knowledge and expertise.

C. Expectations for  Grants  Management, Execution  of Administrative Functions,  and
   Carrying Out Proposed Activities

The  award of GAP  grants  brings  with it EPA's expectation  that  tribes  will  fulfill the
requirements of GAP for demonstrating accountability in the utilization of funds as well as for
grants management and performance reporting, detailed in the 2000 GAP Guidelines.6  The
Guidelines also specify  requirements for  the preparation of work plans, financial reports, and
performance reports that demonstrate progress toward the achievement of deliverables stated  in
the grant work plans.

Based on  our interviews with regional POs, we found that, overall, tribes are meeting regional
expectations for grants management, the execution of administrative functions, and carrying out
proposed activities.  When asked to rate tribal fulfillment of regional expectations along a five-
point Lei chart scale, POs said that tribes almost always or often meet their region's expectations
for grants management  and  the  execution  of administrative functions, and  almost always
complete  the activities proposed  in their work plans.  According  to POs, tribes continue  to
improve the timeliness of their GAP work plans and progress reports.  Currently, most tribes in a
majority of regions are submitting their work plans and progress reports on time.  In one region,
however, only 20 percent of tribes are submitting timely work plans and progress reports.  POs
attribute  significant improvements in the timeliness of tribal submittals  in  part to efforts
undertaken by regional  staff to improve their ability to track submittals and to raise tribes'
understanding  of reporting expectations.   They  also acknowledge  that timeliness  tends  to
improve as tribes gain experience  preparing work plans and progress reports, although setbacks
can occur when there is high turnover among  tribal environmental directors.  Delays in work
plan submittal may also occur when  EPA does not announce award amounts until late in the
year.  Because  GAP has traditionally received its appropriation toward the end of the second
quarter, the time period  tribes have for preparing and submitting work plans is often extended.
One region addressed this problem by establishing a timetable for tribes that specifies when their
work plans would  be  due and when  regional staff would  respond to them.  This region
emphasized to tribes that their GAP awards  could be jeopardized if they did not submit their
work plan by the deadline.

The  quality  and  completeness  of GAP work  plans and  progress  reports  has  improved
concurrently with submission timeliness.  POs indicated that while most tribes submit quality
work plans initially, regions must still enter into  subsequent negotiations with some tribes  to
improve both the quality  and  completeness  of their work plans.   One PO said that only 10
percent of the tribes in the region  submit initial work plans of  sufficient quality; however,
       6 Available online at www.epa.gov/indian/pdfs/gap2000.pdf. Last accessed May 2007.
                                          3-8

-------
approximately  90 percent show  quality improvements in subsequent submittals.  In addition,
work plans and progress reports may  fulfill the basic criteria for completeness but still lack
adequate detail.  Work plan and report templates have contributed to this problem in one region.
A few POs view the use of the standard work plan format recently implemented nationally by
EPA as having positively influenced quality, but others report that the  new format has caused
some declines in quality  due to the need for time to become familiar with the new format, as well
as tribal concern over changing expectations.  Negotiations conducted with  tribes  during the
development of earlier standard formats in two regions have facilitated tribal acceptance of the
new formats, and have helped ease the transition to using them.  As with timeliness,  the quality
and  completeness  of  work  plans  and  progress  reports has  continued to improve with the
increased experience and stability of tribal environmental staff.

To further assess tribal execution of administrative functions, we  examined the length  of time
tribes took to close out their GAP grants after the date the grants ended.  Our assumption, based
on prior interviews with EPA staff, is that tribes  that close out their GAP grants relatively
quickly after the end of the grant period have fewer administrative requirements to address  at the
end of the  grant,  and demonstrate greater capability to execute administrative functions.  Of the
754 GAP grants awarded to the 111 tribes in our sample from  1994 through 2004, EPA's GICS
database indicates that tribes closed out 175 grants (23 percent) during this period. The  amount
of time that passed between the end date and final closeout  date for each grant ranged from
approximately minus 3 months (i.e., the grant was closed out prior to the grant's  end  date) to 51
months. On average, it took tribes about a year (13 months) to close out their GAP grants.

We also examined the results of A-133 audits conducted  for the 111  tribes  in the sample to
assess  tribes'  ability  to execute  administrative functions.7   As  illustrated in Exhibit 3-4,
approximately  25 percent of the 111 tribes in our sample  had been audited during the period
1997-2004. Of these 27 tribes,  the audits for 24 resulted in at  least one reportable condition,
material weakness, or material noncompliance outcome.8  Exhibit 3-5 presents the proportion of
tribes with any of these three major audit findings organized by region.  The greatest number of
tribes audited was  in Region 10, followed by Regions  9, 8, and 6.  Audits that resulted in  a
Reportable Condition also resulted  in a Material Weakness.  All the tribes audited in  Regions 7,
8, and 9 had audit findings in these categories. Fewer tribes had audits that resulted in Material
Non-Compliance.   Note  that because participation in an A-133  audit is required only  when  a
tribe's total annual expenditures  of federal funds exceed a high threshold, most  of the tribes in
our sample would not likely have to undergo such an audit.  As a result,  the tribes in our sample
that were audited and  cited with a reportable condition, material weakness, or a material  non-
compliance may not  be representative of  the  ability  of the tribes  not audited to  execute
administrative  functions  pertaining to GAP  grants.   It may also be the case  that since A-133
       7 In accordance with Office of Management and Budget (OMB) Circular No. A-133 (68 FR 38401), non-
federal entities that expended $300,000 (or $500,000 for fiscal years ending after December 31, 2003) or more in a
year of federal awards, such as grants, are required to have a single or program-specific audit conducted on an
annual basis. Available from: http://www.whitehouse.gov/omb/circulars/al33/al33.html.  Accessed April 2007.
       8 Reportable conditions, as defined in the OMB Circular No. A-133 (68 FR 38401), constitute deficiencies
in internal control over major programs.  Reportable conditions  may  be individually  or cumulatively material
weaknesses.  Material noncompliance means noncompliance with the "provisions of laws, regulations, contracts, or
grant agreements related to a major program."
                                            3-9

-------
audits include a review of all federal expenditures for a tribe, the findings recorded in the Audit
Database for tribes in our sample may not be related to tribal fiscal performance under GAP.

                        Exhibit 3-4: A-133 Audit Findings, 1997-2004 (n = 111)
                      Tribes Audited but no
                       Problems Identified
                             3%
           Tribes Not Audited
                 76%
                                                                        Tribes Reporting a Material
                                                                     Weakness, Reportable Condition, or
                                                                         Material Non-Compliance
                                                                                 22%
Exhibit 3-5: A-133 Audit Findings by Region, 1997-2004 (n = 111)
EPA
Region
1
2
4
5
6
7
8
9
10
Number of
Tribes in
Sample
2
1
1
7
14
2
6
30
48
Percent of
Tribes in the
Sample that
were Audited
0%
0%
0%
29%
29%
50%
83%
20%
19%
Percent of Tribes
Audited with a
Reportable
Condition
-
-
-
50%
75%
100%
100%
100%
78%
Percent of Tribes
Audited with a
Material
Weakness
-
-
-
50%
75%
100%
100%
100%
78%
Percent of Tribes
Audited with a
Material Non-
compliance
-
-
-
50%
50%
0%
60%
50%
33%
Percent of Tribes
Audited with no
problems
reported
-
-
-
50%
0%
0%
0%
0%
22%
                                               3-10

-------
Regional POs identified only a few tribes that had received a major finding on an A133 audit.
POs said  these  audit findings mainly cited  problems with tribes' practices in tracking and
documenting expenditures, such  as incomplete time and attendance records for environmental
staff on the GAP payroll.   In addition, audits revealed some concerns about tribes' ability to
track the non-Federal share of their funding and the use of GAP funds when those funds were
combined with  other grants (e.g., when GAP funds and other grants were pooled to purchase
equipment).  At the regional level, the grants administration office often assumes responsibility
for reviewing tribes'  management  of GAP grants while POs focus their reviews  of grantee
progress on activities funded by GAP.  For regional on-site GAP  grant management reviews,
POs report that a majority of tribes receive positive reviews, but in the small number of instances
where problems are identified, the regions work directly with the tribes to correct them.  In one
region where GAP grants are incorporated into tribal PPGs, all regional program  staff involved
in the PPG, including senior managers, participate in the reviews and the development of follow-
up action items with the tribes.

D.  How Participation  in  GAP  Influences  Understanding of  the Process  Required to
    Develop a Tribal Environmental Program

In addition to assessing the direct resource outputs provided  by  GAP, the evaluation seeks to
discern  how  tribal participation  in  GAP and utilization of GAP resources has influenced, 1)
tribes' understanding of the process required to develop an environmental program, and, 2) the
way tribes approach the various  administrative and programmatic functions associated with the
development process.    Tribal  representatives  emphasize  that  instead  of changing tribal
understanding of how to develop an environmental program, GAP facilitates tribes' ability to
develop a  program that is  responsive to  each tribe's unique environmental conditions and
priorities.   GAP resources enable tribes to  establish an environmental presence,  which in turn
provides the foundation upon which each tribe can build an environmental program tailored to
meet its needs. From EPA's perspective, however,  GAP  may  influence and clarify tribal
priorities as  tribal environmental staff acquire  training,  learn about specific  environmental
conditions on tribal lands, and become more aware of concrete program opportunities through
their interactions with EPA regional tribal and media program contacts.

Although  tribes' maintain the view that GAP has not directly changed  their environmental
priorities or their understanding of what is needed to achieve them, they acknowledge that on a
practical level, GAP has helped them develop  a planning perspective,  specifically, establishing
and refining the annual goals in their work plans to support their priorities.  One tribe noted that
the progress made toward the goals outlined in its GAP work plan in a given year influences the
priorities it establishes for the next year.  As another tribe observed, however, the goals EPA
establishes for GAP in a given year might target an emerging EPA priority, such as children's
health,  which  may not correspond  with tribal  environmental  priorities.9   Under these
circumstances, tribes may enter  into negotiations  with EPA to resolve differences but  end up
modifying their approach to align with the GAP goals in order to secure needed funding.
       9 The divergence between tribal and EPA goals may be more of an issue for tribes seeking media-specific
funds from EPA, rather than GAP funding, since GAP funds may be used to address tribal priorities.
                                          3-11

-------
With regard to the influence of GAP on the administration of tribal  environmental programs,
tribes stress that the continuity of GAP funding provides them with the means to hire, train, and
keep qualified, environmental professionals.  Retaining a qualified, knowledgeable staff person
allows tribes to effectively manage their GAP funds, carry out activities proposed in their work
plans, and seek opportunities to further expand and diversify their environmental programming
in response to tribal priorities.  GAP facilitates tribes' awareness of the broader  environmental
resources and infrastructure available to assist them.  Through contact with GAP POs and other
personnel in each region's tribal office, tribal environmental staff learn about the media-specific
programs within EPA and external to EPA that can provide funding and technical assistance to
complement GAP activities.

GAP also facilitates the creation of partnerships among tribes, and between tribes, and outside
agencies and organizations, that can give tribes a voice in addressing environmental concerns on
and off the reservation. Most tribal representatives assert that in addition to promoting external
communication, GAP has influenced how tribes communicate internally to tribal members and
council about  environmental concerns.   Tribes use GAP funds to develop  environmental and
cultural education programs for tribal youth and prepare outreach materials, such as monthly
newsletters and radio announcements.    The education and  outreach  conducted  by  tribal
environmental staff helps to build community  recognition for  environmental activities on the
reservation  and  raises  the  credibility  of  the  tribal  environmental  program  and  its
accomplishments. For example, one tribal representative mentioned that GAP assistance helped
the tribe gain recognition in the community for its environmental leadership.  The  environmental
director and staff formed committees to raise awareness among tribal members and involve them
in establishing environmental priorities on the reservation.
III.  INDICATORS OF TRIBAL ENVIRONMENTAL CAPACITY

An important objective of the evaluation is to determine how effective GAP has been in building
tribal environmental capacity  among tribes receiving GAP funds.   For the purpose of this
evaluation, we are using a definition based on EPA's 2000 GAP Guidelines: "environmental
capacity" means that a tribe has established the administrative, legal, technical, and enforcement
capability necessary to develop and implement a tribal environmental program,  as well as the
communications capability to  work with federal,  state,  local, tribal,  and other  environmental
officials. This section attempts to answer this question in detail by: 1) identifying the indicators
of environmental capacity, as defined by GAP, tribes, and regional POs; 2) assessing the extent
to which tribes have "achieved" environmental capacity given the presence of these indicators;
3) identifying other  factors that may influence  the development  of tribal  environmental
programs; and 4) determining how GAP has contributed to tribal environmental capacity relative
to these other factors.

A. Tribal and Regional Perspectives on Indicators of Environmental Capacity

Not  withstanding  the GAP definition of  environmental  capacity  and the five  indicators
incorporated  within,  we  asked  tribal  representatives  and regional POs  to  provide their
perspective on what it means for a tribe to have attained environmental capacity.  Tribes tend to
define environmental  capacity more  generally than GAP and equate it with a variety of key
                                          3-12

-------
indicators, as shown in Exhibit 3-6.  Many of the tribal definitions of capacity and indicators fall
within the  GAP  category of technical  capability, e.g., the  hiring  and training of  qualified
environmental professionals  and the expansion  of tribal environmental programming efforts to
include media-specific components.  A few tribes linked environmental capacity with legal or
enforcement capability.  One tribal  representative equated environmental  capacity with three
broad indicators:  1) the ability of tribes to recognize environmental problems, 2) the ability of
tribes to address the problem and take immediate  action to protect health and the environment,
and, 3)  the ability (i.e.,  knowledge and  capability) of tribes to prevent the problem from
happening again.  By that person's estimation, 90 percent of tribes have met the first indicator,
35-50 percent have met the second, and only 10 percent have met the third.
              Exhibit 3-6: Tribal Definitions of Environmental Capacity and Key Indicators
                  Definitions
             Key Indicators
    Having an environmental presence - the ability to
    respond quickly to environmental issues and
    implement environmental initiatives in response to
    tribal concerns as they arise.
    The ability to attract, hire, train, and sustain
    qualified environmental staff to regulate tribal
    environmental programs across all media areas.
    The ability to build tribal awareness and
    understanding of environmental conditions on tribal
    lands.
    The ability to carry out environmental protection via
    the development of codes and ordinances.
    The ability to enforce environmental permits and
    conduct compliance actions on reservation lands.
Progression over time from a single multi-media
program with one staff person to multiple program
areas with support staff.
Trained staff and growth of programs with media
grants.
The degree of institutionalization of environmental
programs and the reliance of the tribe on the
technical expertise of its environmental director and
staff.
A functioning environmental program with all five
individual components - administrative, legal,
technical, enforcement, and communications.
Establishing a water quality system and standards,
receiving Treatment As a State (TAS), and being
able to run the program efficiently for the
betterment of the tribe.
Convergence of the tribe's environmental program
with its economic development strategy.
Protection of tribes' sovereign right to protect water,
land, air.	
Regional POs identified many of the same key indicators of capacity as tribes, such as tribes'
ability to establish an environmental presence; retain qualified, knowledgeable  staff over the
long-term; and diversify their environmental programming.  These POs see the  longevity  and
expertise of staff as the primary means by which tribes achieve success.  Seasoned staff are adept
at finding  solutions to  environmental  problems, they  participate  more  fully  in regional
networking opportunities (e.g., RTOC calls), they provide mentoring to other tribes, and they are
likely  to have comprehensive programs  funded by multiple grants.  POs also pointed to the
development   of  legal  and  enforcement  capability as  an   important  indicator; one  PO
acknowledged, however, that while many tribes have developed codes and ordinances, few have
the capability to enforce them.  Another PO cited direct positive results, such as when a tribe's
efforts lead to a demonstrable  change in behavior or environmental outcomes for tribal members,
as an important indicator.
                                             3-13

-------
B.  The Extent to Which Tribes Have Achieved Environmental Capacity
In order to determine the extent to which tribes  in  our sample have  achieved environmental
capacity as defined by GAP, we examined tribal capability in each of the five indicator areas -
technical, legal, enforcement, administrative, and communication.  We identified a set of coded
activity  types for  each  indicator  and  equated tribal capability in  that  area with  a  tribe's
participation  in one or more related activity.  Exhibit 3-7 lists the activity types  selected  to
demonstrate  tribal  capability for  each indicator  and the proportion of the  96  tribes  that
participated in activities within each type during 2000-2004.  For legal  capability,  we identified
three activity types: activities to increase legal capability and the development and/or adoption  of
codes, ordinances, and standards.10  For enforcement and administrative capability,  we identified
only  one activity type for each: activities to increase enforcement  capability and activities  to
increase fiscal administrative capability, respectively.11  For technical  capability we identified
four activity types:  the hiring of a professional  employee,  and participation in water, waste,  or
air activities.12  For communications capability we identified three activity categories:  general,
internal, and external communication activities.
Exhibit 3-7: Tribal Achievement of Environmental Capacity, 2000-2004 (n = 96)
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental Capacity
Developed a Code, Ordinance, or Standard
Participated in an Activity to Increase Legal Capacity
Adopted a Code, Ordinance, or Standard
Participated in an Activity to Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to Increase Fiscal Administration Capacity
Participated in Internal Communication Activities
Participated in External Communication Activities
Participated in General Communication Activities
Number
of Tribes
25
24
7
25
86
70
70
47
15
68
66
28
Percent of
Tribes
26%
25%
7%
26%
90%
73%
73%
49%
16%
71%
69%
29%
       10 The development of codes, ordinances, and standards includes the development of permitting/licensing
authority and water quality standards.
       11 Enforcement activities include all activity descriptions containing the keywords "enforce" or "inspect."
Activities to increase fiscal administration capability comprise activities related to the development of standards for
property management, procurement, and general fiscal administration.
       12 Water activities include ground water, non-point source, point source, source water protection, UIC,
watershed, and wetlands.  Waste activities include  hazardous waste,  recycling, and  solid waste.  Air activities
include external and indoor air.
                                              3-14

-------
Exhibit 3-8 presents the number and percent of tribes that achieved at least one indicator for each
category of environmental capacity during the period 2000-2004. For example, in Region 1, one
of the two tribes in  the region achieved at  least one indicator of technical capacity, while in
Regions  10, 33 of the 34 tribes in the region achieved at least one indicator of technical capacity.
This exhibit shows that, overall, the vast majority tribes have achieved at least one measure of
capacity  in the areas of technical and communications capability.  Approximately one-third of
tribes have achieved at least one measure of legal capability, while fewer tribes have achieved
enforcement and administrative capacity.

Exhibit 3-9 shows the  number and proportion of tribes that demonstrated capability in multiple
categories of capacity  during the same time  period.  For example, in Region 1, one of the two
tribes did not demonstrate capability in any of the  indicator  categories (legal, enforcement,
technical, administrative,  or communications  capability).   The  second tribe  in the  region
demonstrated  capability  in  only   one  of  these categories.    Forty-four  percent  of tribes
demonstrated  capability in two categories of capacity, 24 percent demonstrated capability in
three categories,  and 21 percent demonstrated capability in four categories of capacity.
                                           3-15

-------
Exhibit 3-8: Number and Percent of Tribes that Achieved at Least One Indicator for Each Category of Environmental Capacity,
by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Total
No. Tribes in
Sample with
Data
2
1
1
7
14
2
6
29
34
96
Legal
Number
0
0
1
2
2
0
2
12
14
33
Percent
0%
0%
100%
29%
14%
0%
33%
41%
41%
34%
Enforcement
Number
0
1
1
4
3
0
2
10
4
25
Percent
0%
100%
100%
57%
21%
0%
33%
35%
11%
26%
Technical
Number
1
1
1
7
14
2
6
29
33
94
Percent
50%
100%
100%
100%
100%
100%
100%
100%
97%
98%
Administrative
Number
0
0
0
0
0
1
1
9
4
15
Percent
0%
0%
0%
0%
0%
50%
17%
31%
12%
16%
Communications
Number
0
1
1
4
10
1
4
27
32
80
Percent
0%
100%
100%
57%
71%
50%
67%
93%
94%
83%
Exhibit 3-9: Number of Categories of Environmental Capacity for which Tribes Achieved at Least One Indicator, by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Total
No. Tribes in
Region with
Data
2
1
1
7
14
2
6
29
34
96
Zero
Number
1
0
0
0
0
0
0
0
0
1
Percent
50%
0%
0%
0%
0%
0%
0%
0%
0%
1%
One
Number
1
0
0
0
2
1
1
1
3
9
Percent
50%
0%
0%
0%
14%
50%
17%
3%
9%
9%
Two
Number
0
0
0
4
10
0
3
9
16
42
Percent
0%
0%
0%
57%
71%
0%
50%
31%
47%
44%
Three
Number
0
1
0
3
1
1
0
8
9
23
Percent
0%
100%
0%
43%
7%
50%
0%
28%
27%
24%
Four
Number
0
0
1
0
1
0
2
11
5
20
Percent
0%
0%
100%
0%
7%
0%
33%
38%
15%
21%
Five
Number
0
0
0
0
0
0
0
0
1
1
Percent
0%
0%
0%
0%
0%
0%
0%
0%
3%
1%
3-16

-------
Technical Capability
Ninety-eight percent of the 96 tribes in our sample participated in activity types associated with
technical capability.  Within this indicator category, 90 percent of the 66 tribes for which we had
GAP position data hired  at least one professional or technical employee with GAP funding.
Exhibit 3-10 presents the  number of professional and technical full-time equivalent (FTE) staff
supported by  GAP.   Tribes hired  more professional staff (an average  of  0.8 FTEs), which
included environmental program directors or managers, than technical  staff (an average of 0.4
FTEs).  These results suggest that tribes depend primarily on GAP funding to hire full-time
environmental directors and managers but may rely on GAP in combination with other funding
sources to hire technical personnel for specific programs or to perform discrete tasks.
Exhibit 3-10: Number of Full Time Equivalent Positions (FTEs) funded by GAP,
2000-2004 (n = 66)
Category
Professional
Technical
Total Professional and
Technical FTEs
Total FTEs provided by GAP
53.4
27.7
81.2
FTEs per Tribe
0.8
0.4
1.2
To assess  the  scope of tribal technical capability  supported by GAP,  we  examined  tribes'
participation in activities in three media-specific  program areas.  As shown in Exhibit 3-11,
nearly three-quarters (73 percent) of the 96 tribes participated in either a water or waste activity.
Approximately half participated in an air activity.   Exhibit 3-12 displays the number and
proportion of tribes in each region that participated in these media-specific areas. Regions 2, 5,
6, 8, 9 and 10 had tribes that participated in activities in all three media areas; Regions 2 and 5
had the highest rates of tribal participation across the three areas. The rates of tribal participation
in Regions 1, 2, 4, and 7 may be a function of the small number of tribes included in the sample
for each region.
Exhibit 3-11: Number and Percent of Tribes with Media-Specific Activities,
2000-2004 (n = 96)
Media Area
Water
Waste
Air
Number of Tribes
70
70
47
Percent of Tribes
73%
73%
49%
                                           3-17

-------
Exhibit 3-12: Number and Percent of Tribes Participating in Activities in Each Media Area,
by Region, 2000-2004 (n = 96)
Region
1
2
4
5
6
7
8
9
10
Number of Tribes
with Activity
Information
2
1
1
7
14
2
6
29
34
Water Activities
Number
1
1
1
6
10
2
6
21
22
Percent
50%
100%
100%
86%
71%
100%
100%
72%
65%
Waste Activities
Number
0
1
0
7
8
0
4
22
28
Percent
0%
100%
0%
100%
57%
0%
67%
76%
82%
Air Activities
Number
1
1
0
6
7
1
4
12
15
Percent
50%
100%
0%
86%
50%
50%
67%
41%
44%
Tribes engage in many activities specific to the media program areas, although there appears to
be a  greater  diversity of activities in the water  and waste categories than in the air activity
category.  Exhibit  3-13 lists examples  of the kinds of activities conducted in  each area, as
described by tribal representatives.
   Exhibit 3-13: Examples of Media-Specific Program Activities Conducted by Tribes with GAP Funding
        Water Activities
     Waste Activities
     Air Activities
    Water quality planning
    Development of water quality
    standards
    Surface water monitoring
    Well monitoring and
    compliance
    Water sample analysis
    Riparian zone protection
    Sewage treatment permitting
    Wetlands - survey, permitting
    Mercury testing in fish
    Certification of staff
    QA/QC for macroinvertebrate
    identification
Development of solid waste,
recycling, and burn barrel
ordinances
Solid waste - open dump
cleanup; waste inventory
Recycling - electronics, motor
oil
School chemical cleanup
Hazardous spill cleanup
Indoor air quality monitoring
Communications Capability

Eighty-three percent of tribes in our sample participated in one or more activity types associated
with communication capability (Exhibit  3-9). As shown  in Exhibit 3-8,  71 percent of tribes
participated in internal communication activities, (e.g., with the tribal council, tribal members,
schools, and other tribal  offices). A nearly equal proportion, 69 percent, participated in external
communication activities (e.g. with other tribes, EPA, other Federal  and non-federal agencies,
and non-governmental organizations).  Only about 30 percent of tribes reported participating in
general  communication  activities  that could  not be  specified as either internal  or external
communication.  Regions 1,  4, 9, and 10 had the highest rates of tribal participation in activities
                                              3-18

-------
associated  with communications  capability.   Neither tribe  in  our sample from  Region  1
participated in these types of activities. As discussed in Section II.D., GAP supports a range of
internal and external communications  activities,  from developing partnerships with  federal
agencies to preparing outreach materials for tribal youth.

Legal and Enforcement Capability

Approximately a third of tribes (34  percent) in our sample met at least one indicator of legal
capability (see Exhibit 3-8).  While approximately a  quarter of tribes participated  in general
activities to increase legal  capacity or developed a code, ordinance,  or standard, far fewer (7
percent) actually adopted a code, ordinance, or standard (see Exhibit 3-7). Regions 4, 9, and 10
had the highest proportion of tribes in our sample that demonstrated legal capability.  Regions 1,
2, and 7 had no tribes that  demonstrated this capability.  That fewer tribes in our sample had
demonstrated legal capability compared  to  other indicators runs counter to  the experience of
tribal representatives  who  participated in the panel discussions.   Many tribes said they had
developed codes and ordinances for solid waste and water quality, among other programs, with
assistance from GAP. The difference may be a function of the limited time period, 2000-2004,
for which data were available on tribal activities.  Depending on when tribes  in the sample first
received GAP funding, they  may have  conducted  activities  related to  legal   capability,
particularly  the development of tribal codes and standards, either  prior to the start or after the
conclusion of this  period.  Tribes may have also participated in legal activities, including code
and standards development with support provided from  non-GAP funding sources.  According to
one  tribal representative, the tribe  provides its  own  legal  staff to  assist the environmental
program with its legal activities.

Fewer tribes in our sample demonstrated enforcement capability than legal capability.  As shown
in Exhibit 3-7, just over a  quarter of tribes participated in inspections or other enforcement-
related activities.  Regions 2,  4, and 5 had the highest proportion of tribes that participated in
these activities.  No  tribes in Regions  1 or 7 participated in activities  associated with the
development of enforcement  capability  from 2000-2004.    Section II.B. describes some of
enforcement activities conducted by  tribes with  support from  GAP.   Tribal enforcement
capability may be  a function of tribes' ability to develop and adopt their own  codes, ordinances,
and  standards.  While the  proportion  of  tribes  shown to have demonstrated enforcement
capability is consistent with the assessments of POs, it is possible that tribes may have conducted
enforcement activities or inspections outside the  period for which  GAP activity  data  were
available or may have funded these activities with non-GAP support.

Administrative Capability

Only 16 percent  of tribes  in  our sample participated in activities  related to administrative
capability (Exhibit 3-8).  Fewer tribes have demonstrated capability in the administrative  area
than any other indicator.   As mentioned above, for the purposes  of this analysis we defined
administrative capacity as tribal participation in activities related  only  to fiscal administrative
capability,  including  the development of procurement and property management  standards.
Our  narrow focus  on  fiscal  administrative capability likely omitted tribal participation in other
administrative activities,  such as grants administration, which tribes may have conducted  as a
component of their media-specific  programs.  The observations of the regional POs discussed in
                                           3-19

-------
Section II.C.  regarding the  extent  to  which  tribes  are  meeting  expectations  for  grants
management and execution of administrative functions suggest that  contrary to this finding,
tribes have increased their administrative capability over time.

C. Factors  that Contribute to the Achievement of Environmental Capacity

In addition  to requesting tribal and  regional input  on the most important indicators of tribal
environmental capacity, we asked tribes and POs to identify factors that impact environmental
capacity and describe how they influence tribal efforts. We found considerable overlap between
the key indicators of capacity identified by tribes and POs and the factors they cited, which
influence  capacity.  For example, the ability of a  tribe to hire  and retain knowledgeable
environmental staff is considered to  be both a factor influencing capacity  and an indicator of
tribal environmental  capacity.   In other words, tribes  need committed, trained professionals to
develop their environmental programs, and tribes with established environmental programs are
more likely to have a stable, qualified staff.

We presented POs with a list of potential factors and asked them to rank each on a Liechart scale
according to the degree to which it influences the ability of tribes in their respective regions to
attain environmental  capacity.  Exhibit 3-14 lists the factors that a majority of POs identified as
more (almost always or often) and less (sometimes., seldom, or almost never) influential.  The
rankings suggest that the tribal  council's relationship with a tribe's environmental program and
its director, demonstrated by the level of support and clarity of direction it provides, significantly
impacts a tribe's ability to attain environmental capacity. POs reported that stable funding is the
most critical factor affecting the longevity of tribal environmental staff, but emphasized that the
actions of tribal council can also lead to staff turnover when there is a lack of communication
between a tribe's  environmental director and council members, or when tribal elections result in
a change in  leadership.  The frequent departure or replacement of trained tribal environmental
staff can deprive tribes of the institutional expertise needed to sustain continuity and grow their
environmental programs. Staff turnover can also diminish other factors that build  capacity, such
as information sharing among tribes, communication with EPA, and access to non-GAP funding.
Historically, POs report that the term of employment for a  tribal environmental  director  has
varied across tribes,  ranging from a low of <1 year to a high of 20 years, with  an average of
approximately 2 to 10 years, depending on the region.

Another potential measure of a tribal council's support for the tribe's environmental program is
its commitment of tribal funding.  According to one PO, a tribe's willingness to support the
environmental program with its own money means the tribe values the program enough to help it
succeed.   For example, a tribe in the region conducted extensive community outreach among
tribal members to obtain  their ideas for  a creek  restoration project.  The  tribal council's
investment of discretionary funds, in addition to their official endorsement, was critical in
providing the environmental program with the means to sustain community participation in its
planning and restoration efforts.
                                           3-20

-------
         Exhibit 3-14: Factors that Project Officers Said Influence Tribal Environmental Capacity
         	(in descending order of importance)	
   More Influential (Almost Always or Often):
   •   Tribal council support for environmental programs
   •   Qualifications of tribal environmental director and/or staff
   •   Turnover rate of tribal environmental director and/or staff
   •   Clear tribal environmental priorities
   •   Degree of information sharing among tribes
   •   Access to funding outside of GAP
   •   Experience, knowledge, and longevity of EPA POs
   •   Rate of change in tribal government leadership (e.g., council members)
   •   Frequency with which tribes request information from EPA	
   Less Influential (Sometimes, Seldom, or Almost Never):
   •   Year-to-year shifts in funding priorities due to changing priorities at EPA
   •   Cohesiveness of a tribe's land base
   •   Planning documents prepared by tribes (e.g., five year plans)
   •   Extent to which the regions consult with tribes
   •   Changes in the GAP funding process
   •   Different perceptions about funding priorities between tribes and EPA
   •   Status of tribal office in region (e.g., its location in the organizational chart)
The factors tribal representatives consider most influential correspond with many of the factors
that POs identified  as  affecting tribes'  ability to build capacity, including the  stability  of
knowledgeable  tribal environmental  staff,  effective communication between tribes  and EPA
regions,  and the support of tribal  council  for  planning and funding environmental  programs
relative to  other tribal  priorities (e.g., healthcare  and economic  development).   Tribes  also
identified  factors related to EPA's  activities  that influence  tribes' environmental  capacity,
including the need of EPA  regional tribal office staff to clearly convey their expectations for
GAP, provide timely responses to grant applications and requests for assistance, and be proactive
in consulting with other EPA  media programs.  Additional factors mentioned by tribes that
influence  capacity   include the  existing   tribal  infrastructure  and  history  of  successful
implementation of tribal programs, jurisdictional issues, and the length of time a tribe has been
federally recognized.

Tribes stressed that consistent access to funding across a range of media  areas is critical to their
ability to build environmental capacity and expand their programming to respond to tribal  needs.
Since  GAP does not fund implementation activities  and many media-specific grants  are
competitive, tribes cannot be assured that they  will have the funds to maintain their  programs
from year to year. One tribe indicated that the ability of EPA to find ways to fit a broader array
of tribal environmental  activities within the GAP guidelines  is one way to address funding
consistency and help tribes attain environmental capacity. Another suggested that the Treatment
as a State (TAS) designation,  if tied to  sustained  funding such as  states receive, could help
ameliorate this problem.

A number of the most influential factors, such as  the level and consistency of  GAP funding
available  to sustain and  train tribal  environmental  staff,  the timely  provision  of technical
                                             3-21

-------
assistance, access to non-GAP EPA funding, and effective communication with tribes, remain
within the purview of EPA.  Other factors related to a tribe's internal structure and leadership,
including the clarity of tribal environmental priorities, the rate of change of tribal leaders, and the
support  provided by  tribal council for planning  and funding environmental programs,  are,
however, generally beyond EPA's  direct control.  Yet, given the significant role of GAP and
other EPA support in  building, promoting, and sustaining tribal environmental programs, EPA
may still have the ability to indirectly influence these internal tribal factors.

D.  Relative Contribution of GAP Toward Achieving Environmental Capacity

As  discussed above, there are many factors  that can potentially  affect  tribes' achievement of
environmental capacity  such as GAP funding and technical assistance, the stability  of tribal
leadership and staffing, and the degree to which Tribal council members focus on environmental
concerns.  Another potential factor  is tribal access to other sources of funding (e.g., EPA media
programs, other federal and state agencies, and tribes themselves).

Interviews with tribal  representatives and POs make clear that they perceive GAP funding as
essential to achieving environmental capacity. Many tribes say that without GAP funding, they
would be able to do very little environmental work.  They stress that GAP is the foundation for
their environmental programs, and GAP resources enable them to establish  a basic  program
infrastructure,  through which they  can  apply for other types of environmental funding.  This
view supports a basic premise of the GAP program, namely, that as GAP helps tribes build their
environmental capacity, tribes will  be able to access other sources of funding to  support their
environmental programs.

If this view is correct,  it  should be evident from the data on the types of grants tribes are
accessing to support their environmental programs.  We would expect tribes that had accessed
GAP for a longer period of time would have built environmental capacity to  a greater degree,
would have  more established environmental  programs, and as a result would be more able to
obtain greater amounts of non-GAP funding than tribes that had accessed GAP over a short time
period.  To  test this hypothesis, we conducted an analysis of the amount of non-GAP grants
receive by tribes in our sample. Note that because we only had access to  data for EPA grants,
this analysis does not  include any  non-EPA  grants (such as water resources funding available
through other federal agencies like  the Bureau of Indian Affairs).  Out of the  111 tribes in our
sample,  69 (62 percent) received at least one non-GAP EPA grant.  Exhibit 3-15 lists the number
of non-GAP EPA grants in each category awarded to tribes  in our sample (in descending order),
along with the total amount of funding awarded in each  category, the number of tribes in our
sample  that received  each type of  grant, the  average  grant  amount  awarded, the  average
cumulative amount awarded to each tribe that received a grant, and the average number of grants
awarded to tribes in each category.  Tribes in  our sample received a total of $89,251,881 in non-
GAP EPA grant funding from 1994-2004. They most often received  grants in the general water,
multimedia,  and air categories.  On average, individual tribes received the most total funding
from general water, water point-source, and multimedia grants.  Appendix J lists examples of the
non-GAP EPA programs and other federal agencies identified by tribes that provided funding
and/or technical support.
                                          3-22

-------
Exhibit 3-15: Number and Dollar Amount of Non-GAP EPA Grants Awarded to Tribes,
by Grant Category, 1994-2004 (n=lll)*
Category
Water
Multimedia
Air
Land
Land
Water
Land
Water
Land
Air
Water
Other
Special
Emphasis
Land
Other
Land
Water
Land
Special
Emphasis
Land
Land
Subcategory



Pesticides
Superfund
Wetlands
Lead
NFS
Solid Waste
IAQ
Source water
protection
Education
Environmental
Justice
UST

Brownfields
Point Source
Emergency
Response
Pollution
Prevention
Hazardous Waste
Persistent
Organic
Pollutants
Number of
Grants
386
198
93
93
59
57
52
52
43
41
36
28
26
24
23
7
7
6
6
4
1
Total Funding
Awarded
$28,349,377
$15,856,844
$8,559,419
$3,217,600
$4,824,484
$3,773,206
$2,887,537
$3,648,428
$1,237,792
$1,968,785
$5,998,683
$1,496,762
$737,370
$1,132,701
$1,283,880
$1,360,939
$2,271,025
$145,975
$207,889
$243,185
$50,000
Number of
Tribes with
Grant
56
41
24
16
19
29
19
13
20
13
16
16
19
8
10
6
5
6
4
2
1
Average
Grant
Amount
$73,444
$80,085
$92,037
$34,598
$81,771
$66,197
$55,530
$70,162
$28,786
$48,019
$166,630
$53,456
$28,360
$47,196
$55,821
$194,420
$324,432
$24,329
$34,648
$60,796
$50,000
Average
Total
Amount
Awarded per
Tribe
$506,239
$386,753
$356,642
$201,100
$253,920
$130,110
$151,976
$280,648
$61,890
$151,445
$374,918
$93,548
$38,809
$141,588
$128,388
$226,823
$454,205
$24,329
$51,972
$121,592
$50,000
Average
Number of
Grants per
Tribe
6.9
4.8
3.9
5.8
3.1
2.0
2.7
4.0
2.2
3.2
2.3
1.8
1.4
3.0
2.3
1.2
1.4
1.0
1.5
2.0
1.0
*Funding amounts are rounded to the nearest whole dollar.
We then analyzed the sequence of GAP and non-GAP EPA funding between 1994 and 2004 for
the 111 tribes in the sample.  Based on our hypothesis, we anticipated that tribes would access
GAP funds first, followed by non-GAP funds.  We found, however, that GAP funds tend to be
accessed year after year, and therefore, a majority of tribes in our sample (90 percent), received
non-GAP EPA  funds concurrently with GAP funding.  For the remaining tribes that  did not
receive non-GAP EPA  funds concurrently with GAP funds, about four percent them received
GAP funding first, while about seven percent received non-GAP EPA funding first.13 With
        These figures do not add to 100 percent because of rounding error.
                                         3-23

-------
regard to the sequencing of GAP and non-GAP EPA funding, one PO noted that non-GAP EPA
funding was the first type of funding available to tribes. Specifically, Clean Water Act 106 funds
became available in 1989, and this funding was the first grant program accessed by many tribes.
Once GAP became available in 1994, GAP began  to offer more money to tribes than other
program funds.  This may  also explain, in part, why tribes have accessed significantly more
water-related grants than other types of non-GAP EPA grant. Another explanation for the large
number of water-related grants awarded to tribes may be the greater amount and variety of grants
offered by EPA's water programs relative to other media programs.

Since our analysis of the sequence of grants was not informative for the majority of tribes that
received GAP  and non-GAP EPA funding concurrently, we conducted an additional analysis to
compare funding  patterns for tribes that received GAP early in the program's existence (from
1994 to 1999), to  tribes that received GAP later (from 2000 to 2004).14 Specifically, we would
expect tribes  that  began receiving GAP  during  the earlier period ("early adopters") to have
accessed a  greater percentage of funding  from non-GAP EPA sources compared to tribes that
began receiving GAP during the latter period  ("late adopters").  If true, this would support the
claim that as tribes receive GAP funds over time, they are able to build environmental capacity
and basic program infrastructure, and then apply for and receive other EPA funds to support their
programs.  Note that the small number of late adopters in our sample limits the degree to which
we can make inferences from this analysis, since it is possible that this sample of tribes may not
be representative  of the overall population of tribes that received GAP funding from 2000 to
2004.

Our comparison of early vs. late adopters of GAP  shows that in fact early adopters have received
a greater percentage of their funding from non-GAP EPA sources.  Specifically, as presented in
Exhibit  3-16, only  44 percent of early adopters' total EPA funding between 1994 and 2004 came
from GAP, compared to 80 percent of GAP funding for  late adopters.  Moreover, there is a
marked  increase (1,549 percent)  in the  amount of non-GAP EPA funding  awarded to late
adopters after  they received  GAP,  compared to before they received GAP.  This supports the
view that GAP is  helping tribes expand their sources of environmental funding, which suggests
that tribes have increased their environmental capacity accordingly.
       14 Based on AIEO's suggestion, we have excluded Alaskan tribes from this analysis, because the Alaska
Native Claims Settlement Act treats Alaskan tribes differently than other tribes, and they do not have access to the
same funding as other tribes.  When the Alaskan tribes are removed, the sample size is reduced to 70 tribes.
                                          3-24

-------
Exhibit 3-16: Comparison of GAP and Non-GAP EPA Grant Funding for
Early vs. Late Adopting Tribes, 1994-2004 (n=70)*

Number of Tribes
Number of GAP Awards
Number of Non-GAP Awards
Amount of GAP Funding
Amount of Non-GAP Funding
Percent of Total Funding from GAP Grants
Percent of Total Funding from Non-GAP Grants
Average Non-GAP Funding Per Tribe
Median Non-GAP Funding Per Tribe
Total Amount of Non-GAP Funding Received Before
GAP
Amount of Non-GAP Funding received After GAP
Percent Change in Non-GAP Funding
Early Adopters (1994-1999)
61
563
1218
$69,161,568
$87,670,474
44%
56%
$1,437,221
$707,896
N/A
N/A
N/A
Late Adopters (2000-2004)
9
31
12
$3,426,273
$874,395
80%
20%
$97,155
$40,000
$50,000
$824,395
1,549%
*The percent change in non-GAP funding is calculated only for late adopters since data were not available on early adopter receipt of
non-GAP grants for the years preceding 1994.
We also compared achievement of specified indicators of environmental capacity between early
vs. late adopters (Exhibit 3-17).  This analysis shows that for five indicators of environmental
capacity  (those shown in bold in Exhibit 3-17), a greater percentage of early adopters had
achieved the indicators of capacity compared to late adopters. For example, more early adopters
are participating in waste and water activities, which are both indicators of technical capacity,
compared to late adopters.  However, for the  remaining seven indicators of capacity, a greater
percentage of late adopters had achieved the indicators. In some cases, this is consistent with our
understanding of how GAP is intended to work.  For example, more  late adopters than early
adopters  have hired  a professional  employee with  GAP funds.   This may be because early
adopters  are now using other grants to fund  salaries for professional  employees. In general,
however, the results of this analysis run counter to our hypothesis that tribes that have had GAP
funding for a longer period  of time would  be further along  in the process  of developing
environmental capacity, compared to tribes that had  received GAP funding for a shorter period
of time.
                                          3-25

-------
Exhibit 3-17: Comparison of Achievement of Environmental Capacity
Early vs. Late Adopting Tribes, 2000-2004 (n=70)*
Type of Tribal
Capacity
Legal
Enforcement
Technical
Administrative
Communications
Indicator of Environmental
Capacity
Developed a Code, Ordinance, or
Standard
Participated in an Activity to
Increase Legal Capacity
Implemented a Code, Ordinance,
or Standard
Participated in an Activity to
Increase Enforcement Capacity
Hired a Professional Employee
Participated in Water Activities
Participated in Waste Activities
Participated in Air Activities
Participated in an Activity to
Increase Fiscal Administration
Capacity
Participated in Internal
Communication Activities
Participated in External
Communication Activities
Participated in General
Communication Activities
Early Adopters (1994-1999)
Number of
Tribes
12
12
1
21
47
43
41
29
7
38
31
24
Percent of
Tribes
(out of 57)
21%
21%
2%
37%
82%
75%
72%
51%
12%
67%
54%
42%
Late Adopters (2000-2004)
Number of
Tribes
2
0
1
2
9
6
o
J
5
4
7
8
3
Percent of
Tribes
(out of 9)
22%
0%
11%
22%
100%
67%
33%
56%
44%
78%
89%
33%
IV. GAP PROCESS OUTPUTS TO ACHIEVE TRIBAL GOALS AND PRIORITIES

Many of the tribal representatives  interviewed stated that a key  goal for tribes is having an
environmental presence on tribal land, i.e., a qualified staff person who can coordinate the tribe's
environmental programs, maintain a cohesive program, and be a point of contact for members of
the tribal  community  and neighboring  communities.   GAP enables tribes to establish this
environmental presence by providing the funds to hire, train,  and  retain professional and
technical environmental staff.  As discussed in the previous section, GAP funds the salary of the
environmental director or manager, and thus enables the  coordination  of all  environmental
program work.  One tribe noted that GAP funds the salaries of environmental specialists who
conduct virtually all of the tribe's permitting work, as well as training tribal staff in specialized
topics such as GIS and Autocad. In some cases, GAP is the primary source of funding for tribal
programs. In other cases, GAP funds only a portion of environmental programs (e.g., 25 percent
of environmental programs for two tribes in EPA Region 5); however, even in these cases, tribes
consider GAP funding essential.
                                          3-26

-------
Tribes emphasize how GAP provides a foundation for tribal  environmental programs.   One
important way that GAP does this is by providing the framework to leverage  and coordinate
other resources,  such  as  other  EPA  grants.   GAP also  provides  continuity for tribal
environmental programs over time, while at the same time allowing tribes to address new and
emerging environmental issues.  Tribes stress the importance of GAP's flexibility: several tribes
perceive  that GAP funding has fewer  "strings attached" than other sources of funding,  and
therefore  can be used  to  address tribal  priorities. One  tribal  representative compared
environmental capacity to a bush, growing in many different  directions.  The branches are the
various components of a tribes environmental programs and GAP is the root.

While tribal representatives state that GAP funding is vital for establishing and  maintaining an
environmental presence, many perceive current levels of GAP funding  as insufficient.  Several
tribes say that they have received  static levels of GAP funding since the inception  of the GAP
program, even as they develop new environmental programs and carry out ever more diverse
environmental program activities.  For example, one tribe notes that it  wants to take on issues
like recycling, green buildings, and pollution prevention, but existing funding is not  sufficient to
address these emerging areas of environmental concern.   One  Eastern tribal representative
expressed the view that GAP funding is primarily going to tribes in the West.  This  leads to
insufficient funding for the tribe, which results in one staff person having to assume  multiple
responsibilities, and ultimately, limits the tribe's ability to maintain environmental capacity.

When asked  about additional resources  tribes  need to develop their environmental programs,
several tribes indicate that above all else, they need sustained, consistent funding over time to
enable them to  hire and retain  sufficient  qualified  staff,  and  thereby  retain  institutional
knowledge.  GAP provides an important source of sustained funding, although some tribes note
that the requirement to re-apply for GAP funding every year takes away from the stability of the
GAP program and the staff that GAP supports.  In addition to funding, tribes say they  need
infrastructure (especially for water and  wastewater), equipment (e.g., computers, vehicles, and
sampling equipment), and more staff to  develop their environmental  programs.  Several tribes
also  mention the need for more support from  EPA on enforcement  programs,  including both
funding and technical support.  One tribe mentioned that it would be helpful if EPA  regions had
more staff available to assist tribes with  enforcement and developing codes and  ordinances. A
few tribes noted  the difficulties of preparing TAS applications, and one tribe asked for EPA to
speed its review  of these applications.  Another tribe explained that it needed help in preparing
environmental assessments or environmental impact statements under NEPA.

Tribes sometimes have goals and priorities that GAP does not address, either because GAP
funding is insufficient to meet  these goals, or because these goals involve implementation of
environmental programs.  For example,  one tribe mentioned that it has a goal  of  establishing
regulatory programs, and tribal staff are  looking for additional funding  beyond GAP to support
this goal.  Another tribe said that it has goals and ordinances in place, and has trained staff, but
now needs to start implementing  its environmental  programs and taking enforcement actions
where needed. Overall, most tribes interviewed say that in order to meet tribal goals, they need
to be able to use GAP funding for  program implementation and maintenance,  and they  need
additional funding to support this additional effort.
                                          3-27

-------
While most tribes perceive overall consistency between GAP goals and tribal priorities, they also
pointed out that it is difficult to mesh the cultural and traditional values of the tribes with the
bureaucratic and regulatory guidelines and definitions established by GAP.  For example, some
of the terms GAP uses,  such as a "Tribal Environmental  Agreement," carry  historical and
cultural connotations that raise  concerns for tribes  (e.g., concerns  about signing a  formal
agreement with the U.S. Government).   Some tribal representatives are concerned that EPA's
interest in measuring tribal capacity will create a "gold standard" that EPA is expecting tribes to
meet, and this may interfere with tribal sovereignty.  One tribe recommended that EPA hire more
American Indian POs, and suggested that POs visit the tribes and tribal  councils more often to
learn about their culture and the  differences among the different tribes.   Tribes also point out
that EPA sometimes prioritizes specific programs (e.g., brownfields) that are not in response to a
tribal priority or that tribes are not yet ready to address.15 In other cases, tribal councils want
their  environmental programs to do more than  what is allowed under GAP funding.   For
example,  one tribe noted that GAP does not support a tribe's priority to protect  sacred and
historical  sites from development using NEPA.  Tribes  emphasize that tribal  sovereignty is a
critical issue. While many tribal representatives say that EPA tries hard to address tribes' needs
and priorities, others feel that EPA is seeking "manifest destiny," and is trying to change Indian
ideals and erode tribal sovereignty.  Finally, many tribal representatives emphasized that EPA
has a trust responsibility with regard to tribes, and the federal government has the responsibility
to help Tribal programs grow and fulfill their potential.
V.  GAP  SUPPORT FOR EPA'S  STRATEGIC GOAL  OF INCREASING TRIBES'
    ABILITY TO BUILD ENVIRONMENTAL PROGRAM CAPACITY

EPA's 2003  - 2008  Strategic  Plan includes Objective 5.3, which is to build tribal capacity.
Specifically,  EPA seeks to "Assist all federally recognized tribes in assessing the condition of
their environment, help in building their capacity to implement environmental programs where
needed to improve tribal  health and environments, and implement programs in Indian  country
where needed to  address environmental issues."16   The  strategic plan discusses means and
strategies for achieving Objective 5.3, and mentions the GAP program as part of this discussion.
It specifically states that  "EPA will continue to distribute Indian General Assistance Program
capacity building  grants with the goal of establishing an  environmental presence  in all  57217
federally recognized tribes in the United States." AIEO has tracked progress towards this goal in
its Goal 5,  Objective 5.3 Reporting System,18 and reports that the percentage of tribes that "had
access to an  environmental presence," increased from 36 percent in 1996 to an estimated 90.4
percent in FY 2006, with a peak of 97 percent in FY 2004.  Note that access to an environmental
       15 It is not clear whether this comment was made specifically regarding GAP, or EPA funding sources in
general.
       16 The 2003 - 2008 Strategic Plan is available online at http://www.epa.gov/ocfo/plan/2003sp.pdf. Last
accessed April 2007.
       17 The number of federally recognized tribes has changed since the publication of the Strategic Plan: as of
this writing there are 561 such tribes.
       18 The Goal 5, Objective 5.3 Reporting System is available online at https://oasint.rtpnc.epa.gov/TATS/
tats_prv/entry_page. Last accessed April 2007.
                                           3-28

-------
presence is defined as an annual dollar value of GAP funding that AIEO determines is needed to
establish an environmental presence.  That dollar value equaled $75,000 in FY 1996 - 1997, and
thereafter increased to $110,000.  EPA calculates the percentage of tribes with access to  an
environmental presence as the total amount of GAP funding awarded to all tribes divided by the
product of the number of federally recognized tribes and the allocation needed to establish  an
environmental presence (i.e., $110,000 or $75,000, depending on the year).

EPA's updated 2006 - 2011 Strategic Plan includes an updated Objective 5.3 to "improve human
health and the environment in Indian country."  Specifically, EPA is working to, "protect human
health  and the environment on  tribal lands by  assisting  federally-recognized tribes to build
environmental management capacity, assess environmental conditions and measure results, and
implement environmental programs in Indian country."19 EPA has modified its strategic targets
in light of this new goal. Of the three strategic targets in the updated plan, one relates directly to
implementation of environmental  programs: increasing  the  percent  of tribes implementing
federal  environmental programs in  Indian  country.   The other  two strategic targets  could
potentially be addressed through GAP:

   •   By  2011,  increase the  percent  of  tribes  conducting  EPA-approved  environmental
       monitoring and assessment activities in Indian country to  26 percent. (FY 2005 baseline:
       20 percent of 572 tribes.)

   •   By 2011, increase the percent of tribes with an environmental program to 67 percent. (FY
       2005 baseline: 54 percent of 572 tribes.)

The strategic plan notes that, "A tribe is  counted as having an environmental program  for the
purposes of this measure if the tribal government  has taken at least one  of the following actions,
in  combination  with  having  an  organizational  structure   which  includes EPA-funded
environmental office  or coordinator that has been staffed in the most recent year:

       (a)  Complete  a Tier III TEA,  as  evidenced by  a  document signed by  the tribal
          government and EPA.

       (b) Establish environmental laws, codes, regulations, ordinances, resolutions, policies, or
          environmental compliance programs, as evidenced by a document signed by the tribal
          government.

       (c) Complete solid and/or hazardous waste implementation activities.

       (d)  Complete  an inter-governmental  environmental agreement (e.g., state-tribe MOA,
          federal-tribe MO A, etc)."
       19 The 2006 - 2011 Strategic Plan is available online at http://www.epa.gov/ocfo/plan/plan.htm. Last
accessed April 2007.
                                          3-29

-------
While we do not have data to assess progress toward all aspects of these strategic targets,20 we
can say that during the period 2000-2003, 26 percent of the 96 tribes in our sample for which
have GAP activity data developed a code, ordinance, or standard, although these activities may
not be evidenced by a document signed by the tribal government.  We also know that during the
same time period, 73 percent of these tribes participated in waste  activities, although these may
or may not have included implementation activities.

As part of this evaluation, we asked regional POs about the extent to which they believe that
GAP's goals, objectives, and activities align with or diverge from  EPA's strategic goal for tribal
environmental programs.21 Most POs responded that the  GAP program is aligned with EPA's
strategic goals for  tribes,  in that both GAP  and EPA's strategic goals seek to build tribal
environmental capacity, and GAP funds activities that are  the building blocks of environmental
capacity.  GAP supports four common activities that support environmental programs: outreach,
education, enforcement, and  training, although the emphasis placed on any of these types of
activities will vary  depending on the specific region and  the tribe.  GAP also enables greater
tribal participation in state and EPA environmental processes that lead to sounder environmental
decisions. For example,  in the state of Washington, tribal input and data led EPA to conclude
that state-proposed water quality standards were not sufficiently protective of salmon, and EPA
ultimately required the state to revise its standard.

POs  expressed concern about the limitations  of GAP  that restrict  funding to only capacity-
building activities.  In addition to building tribal program capacity, many POs believe it is
necessary for EPA to support maintenance of mature tribal environmental programs, otherwise
such programs will cease to operate, and it will be necessary to re-build program capacity.  Some
POs pointed  out that EPA's strategic goals for Indian country include program implementation,
but such implementation is not supported by GAP, an observation echoed by tribes.  One tribe
noted that it can acquire non-GAP grants to develop multi-media  programs,  but the  question of
whether the tribe can sustain these programs with  non-GAP funds  remains unanswered. Another
tribe identified the apparent contradiction between EPA's strategic performance measures, which
are tied to implementation outputs, and the focus  of  the  GAP guidelines on core program
development and capacity building.

In addition to raising concerns about the issue of program implementation, POs identified other
challenges resulting from using GAP to meet EPA's strategic goals.  One PO raised the concern
that it  will  be difficult for some tribes to ever have  delegated authority for  environmental
programs, given the  frequent turnover of tribal  staff.   In  addition, for tribes  that have
"checkerboard"  land   holdings  and  non-tribal   residents living within  the boundaries  of
reservations,  tribes  that seek to develop and enforce codes and ordinances may  face  fierce
resistance from non-tribal residents that do not accept the tribe's jurisdiction.  Many POs report
that tribal needs and concerns vary widely.  One PO expressed concern that regional allocations
of GAP funds do not sufficiently align with tribal needs.  Since all regions get the same amount
of funding per tribe, a greater share of funding is not being distributed to those tribes with more
       20 This evaluation was designed prior to the release of the 2006 - 2011 Strategic Plan, therefore it was not
tailored to measure progress toward strategic targets under this updated plan.
       21 During the interview, the interviewer referred to the 2003 - 2008 Strategic Plan Objective 5.3.
                                           3-30

-------
pressing environmental concerns. Another PO pointed to the difficulty of quantifying the impact
of the GAP program for a tribe, particularly the value of having a qualified staff person and
environmental presence, and demonstrating the program's contributions to EPA's strategic goals.
                                          3-31

-------
CHAPTER 4:  CONCLUSIONS AND RECOMMENDATIONS

The  results  of  this evaluation clearly establish that GAP has been effective  in building the
foundation of environmental capacity among tribes, defined as capability in one or more of the
five  indicator areas - technical, legal, enforcement, administrative, and communications.  This
capability, in turn, has allowed tribes to achieve an environmental presence in Indian country.
Many tribes  consider having an environmental presence, i.e., the ability to respond promptly and
effectively to tribal environmental concerns as they  arise, as  the  overarching indicator  of
environmental capacity.

To conclude, our review of data from fiscal years 2000 - 2003, along with recent interviews and
discussions with EPA regional project officers (POs) and tribal representatives, suggests that:

   •  The  extent of  capacity-building varies across indicator areas for tribes receiving
       GAP grants.   All but one tribe  studied has demonstrated capability for at least one
       indicator; the largest proportion of tribes (44 percent) has demonstrated capability for two
       indictors. Only one tribe (1 percent) demonstrated capability for all five indicators.
   •  Tribes receiving GAP have relatively well-developed technical and communications
       capabilities.  A majority  (90 percent)  of tribes  studied  have hired a professional
       employee, and most have participated in water and waste activities (73 percent), as well
       as internal  and external communications activities (71 and 69 percent, respectively).
       Tribes use GAP funding to access the training and technical assistance needed to develop
       staff  expertise and establish an environmental presence.  GAP facilitates tribes' ability to
       network with  other tribes,  participate  in  regional partnerships,  and communicate
       effectively with tribal Council and the reservation community about tribal environmental
       priorities and initiatives.
   •  Tribes receiving GAP have less developed legal, enforcement, and  administrative
       capacity compared to  the other two indicators.  A quarter (26 percent) of tribes have
       developed codes,  ordinances, or standards with assistance from  GAP,  but  only a few
       (seven percent)  have gone on to adopt them.  An equal number of tribes (26 percent) have
       the ability  to conduct  inspections or other enforcement  activities.   Among the tribes
       studied,  only 16 percent demonstrated fiscal administrative capability specific to the
       development of property  and procurement standards.  The results  of A133 audits and
       regional GAP grant reviews suggest that a few tribes have experienced difficulty  with
       executing fiscal administrative functions, such as tracking and documenting expenditures
       paid  for by  GAP, and tracking how GAP funds are spent when they are pooled with other
       grants. Note that the data that leads to this conclusion is now several years out of date, as
       with  all of the other indicator data.   Interviews with EPA POs  suggests that tribal
       administrative capacity  may have increased in recent years, as described below.
   •  GAP has done  much in recent years to clarify grant expectations  and administrative
       requirements  for  tribes,  and  tribes  in  turn  are   increasingly  meeting these
       expectations and requirements.   For example, EPA  has developed a standard GAP
       work plan format for tribes to use, and has offered GAP management workshops.  With
       continued  outreach and assistance from GAP POs, tribes have significantly improved
       their  ability to  fulfill regional expectations for grants management, such as the timely
                                           4-1

-------
       submittal of quality work plans and progress reports.  Currently, a majority of tribes in
       most regions are submitting their work plans and progress reports on time. Apart from
       isolated instances where fiscal problems have been documented, tribes overall  almost
       always or often meet their region's  expectations  for the execution  of administrative
       functions under GAP.
    •   Tribes report that restrictions on GAP funding that preclude using GAP grants for
       program  implementation  are now  hindering tribal  environmental  program
       development.  Tribes say that while such restrictions made sense at the outset of GAP,
       the situation of tribal programs has evolved. Even though most tribes have  more to do to
       develop the full complement of capabilities envisioned under GAP (as evidenced by the
       fact that only one percent of tribes studied have  demonstrated capability for all five
       indicators of environmental  capacity),  some tribes feel  that they have  already built
       sufficient program capacity.  These  tribes now want to use their GAP funds to address
       other pressing needs for program maintenance and implementation.  POs agree that tribes
       with mature programs have met the limits of capacity building  and need funds to  sustain
       what has been built with GAP.
    •   Tribes perceive that GAP funding is essential  to  achieving their environmental
       goals, but  current  levels of funding  are insufficient  to address  tribal priorities.
       Consistent,  stable, and sufficient funding is a key concern of tribes,  since such funding is
       viewed as essential for maintaining  a trained  environmental  staff that can  respond to
       tribal  environmental concerns.   Tribes have successfully leveraged GAP resources to
       acquire  additional grant  funding from EPA  media  programs and  non-EPA  federal
       agencies; however, tribes caution that non-GAP grants are extremely competitive and
       often have  more administrative  requirements than  GAP.  These grants may  come with
       limits on the type of programs and activities they support and  may require substantially
       more matching funds than GAP.  Tribes expressed concerns about relying on these grants
       to implement their environmental programs when their availability and level of funding
       can fluctuate from year to year.

Based on the results of our analysis and conversations with tribes, regional POs, and staff at EPA
headquarters,  we  offer  the   following   recommendations  to  EPA's  American  Indian
Environmental Office for ways that EPA can enhance GAP to further support tribes' ability to
establish and sustain their environmental programs.

RECOMMENDATIONS TO EPA

Recommendation  1:   Consider  developing  a  mechanism to support  Tribal  program
implementation. EPA could pursue this in a number of different ways.  As an initial step, EPA
headquarters  and regions could continue  to promote and expand  the  use of Performance
Partnership Grants (PPGs) by tribes.  By combining GAP and other  media-specific funds into
one grant, PPGs  afford tribes  with  the  flexibility  to  allocate funding for programs and
implementation  activities that  most  effectively address  tribal needs.     They  also improve
efficiency by streamlining administrative requirements for tribes, such as the preparation of work
plans and progress reports. AIEO has been encouraging use of PPGs in recent years, although
relatively few tribes are currently using them. The relatively infrequent use of PPGs is partially
due to regional preferences, but mainly a result of tribes' perception that PPGs are difficult to put
                                          4-2

-------
in place and administer. Further work with tribes to communicate and possibly simplify the PPG
process could potentially increase the use of PPGs.

Note  that  since PPGs  comprise grants that are  subject to variations  in  funding levels  and
availability, this may not be a sustainable option for tribes over the long-term. For this reason,
AIEO could eventually  establish a second tier of GAP funding - "GAP plus" - to fund program
implementation for those Tribes that show they have met key indicators of capacity under GAP.
Another  approach could be to establish a block grant for tribes similar to those established for
U.S. territories. In the case of a second-tier GAP grant or block grant, the guaranteed availability
of funds  for implementation would alleviate the uncertainty currently associated with individual
media grants. This approach would also provide tribes with funding for program activities not
covered under the 20 grants now eligible  for incorporation into a PPG. Finally,  a second tier
GAP  or inclusive block grant would reduce the administrative burden on tribes associated with
applying for and managing numerous grants.

Recommendation 2: Consider working with tribes and regions to enhance administrative,
legal, and enforcement capacity.  To help tribes build administrative capacity, EPA could
coordinate more directly with regions to ensure that the  frequency and level of programmatic
resources provided, such as GAP training and support for tribal grants management, keep pace
with tribal needs, particularly as new tribal administrative personnel are  hired.   For tribes that
do not have  sufficient  funding through GAP or other sources to develop legal services, EPA
regions could offer legal support to help tribes enact their own codes, ordinances, and standards.
In cases where tribes feel that they cannot or do not wish to implement their own environmental
laws and regulations, AIEO should coordinate directly with tribes and clarify roles between tribal
environmental programs and EPA.  Although EPA considers tribal delegation of environmental
programs as  an important  long-term objective, TAS may not be desired or feasible  for many
tribes. In these instances, AIEO should consider developing a coherent plan for working with
tribes to protect the environment, while respecting tribal sovereignty.

Recommendation  3: Raise awareness of innovative environmental  policy approaches to
complement traditional codes and standards.1  For example, some tribes may benefit from a
greater emphasis on pollution prevention education, self-certification, and compliance assistance
inspections, rather  than solely focusing on writing codes, conducting inspections, and taking
enforcement  actions.   For  example, tribes may wish to  consider using pollution prevention
materials that have been developed for particular sectors of concern (see, for example, materials
on sectors such as medical facilities, schools, and auto repair shops available from the Pollution
Prevention Resource Exchange2). Tribes may also wish to consider using Environmental Results
Programs (ERP), such  as those that have  been funded under  the EPA  State Innovation Grant
program,  or adopting  elements of ERP.3  To the extent that  tribes undertake innovative
       1 Innovative policy approaches could potentially used in place of traditional regulatory and enforcement
programs, if such approaches could ensure environmental protection.  Innovative policy approaches could also be
used as an interim step, to be used until such time as tribes have traditional regulatory programs in place.
       2 Available online at http://www.p2rx.org/P2InfoNexpert/TopicHubs 2.cfm. Last accessed April 2007.
       3 Information about State Innovation Grants, including ERPs, is available online at http://www.epa.gov/
innovation/stategrants/. Note that State Innovation Grant funding is not currently available directly for tribes (but
could be available to a tribe in partnership with a state).
                                            4-3

-------
approaches to preventing pollution and encouraging environmental stewardship, it would make
sense for tribes to leverage the considerable experience of EPA  and states in developing
innovative policy tools and approaches, as well as specific outreach materials such as pollution
prevention and compliance assistance fact sheets. Tribes may also benefit from working together
to implement innovative approaches.  For example, if tribes were interested in exploring ERP as
a potential policy approach, it may make sense for multiple tribes in a region or in neighboring
regions to work together to develop an ERP for a sector of concern.   AIEO and regions could
help tribes by raising awareness of innovative policy approaches, readily available materials, and
potential funding sources.

Recommendation 4: Acknowledge  cross-cultural differences, and continue working with
tribes to maintain a respectful dialog. A key difference in perspective is that tribes see GAP
funding as a right - an extension of the trust responsibility that EPA has to tribal  nations. EPA
views tribes   as  grantees that  must meet certain  requirements, particularly  administrative
requirements, to show that tribes are accountable for funds that have been spent. Because tribes
vary considerably in their population, land base, wealth, needs,  and priorities,  they feel that
environmental performance measures should be couched in terms of measuring tribes' progress
toward their goals and not achieving a single "gold  standard." There is inherent tension in the
differences between tribal and EPA perspectives, but opportunities to bridge these differences
exist. From the tribes' perspective, one way to strengthen understanding is to hire more Native
Americans to serve as regional POs and tribal coordinators. More frequent site visits to tribes by
AIEO and EPA regional program staff would also help to underscore the diversity of tribal
perspective, priorities, and approaches to environmental protection.

Recommendation 5: Track progress toward achievement of the  new 2006-2011 strategic
goals and  targets.   With the  addition of new strategic targets aimed  at measuring tribes
implementation of monitoring and assessment activities, EPA needs  to develop indicators that
support  these  goals and  targets (e.g., number of tribes that have completed  a Tier III TEA,
number of tribes that have completed an inter-governmental environmental agreement (MOAs),
etc.), and regularly track how many tribes are meeting these indicators.  At a fundamental level,
AIEO will need to ensure that its data collection systems allow for the effective capture and
tracking of indicators related to these targets.  In establishing future strategic goals and targets,
AIEO will also need to consider the  degree to which  its proposed  performance measures align
with tribal  priorities and perspectives and the feasibility of tribes'  of achieving them.  For
example, given the reluctance of many tribes to enter into TEAs and MO As because of concerns
over tribal sovereignty, the selection of these agreements as indicators  of tribal performance may
not provide the  most accurate estimation of a tribe's environmental  capacity  or the degree to
which it has  successfully implemented specific program components. Further, AIEO needs to
consider that  its five-year cycle for  setting  strategic goals and  targets may  be too  short to
effectively track and measure  tribal  progress, given that tribal  environmental  priorities and
abilities may necessitate implementation of programs over a longer time period.
                                           4-4

-------
         Appendix A




GAP EVALUATION QUESTIONS

-------
APPENDIX A
EVALUATION QUESTIONS
la. Is GAP being accessed by all federally recognized tribes?
Ib. Why are some tribes not involved in GAP?
Ic. Are there tribes that received GAP grants at one time but which
no longer receive GAP grants? If so, why?
2a. Are tribal governments using the resources (technical, fiscal, and
programmatic) provided as a component of GAP? How often are
GAP resources accessed?
2b. How are tribes using GAP resources?
2c. To what extent have tribes met program expectations for grants
management, execution of administrative functions, and carrying out
proposed activities?
INFORMATION THAT CAN HELP ANSWER QUESTION
• Number and percentage of federally recognized tribes that have ever received GAP funds
• Regional coordinators' perceptions as to why tribes may not seek GAP funding (these
may include having access to other sources of funds, the perception that participation in
GAP is too onerous, etc.)
• Regional coordinators' perceptions as to why tribes may have dropped of GAP grant
rolls.
• Number and type of GAP resources that have been delivered to tribes:
A GAP funding provided to tribes (i.e., fiscal resources)
A Technical assistance and media specific trainings (i.e., technical resources)
A Grants management training (i.e., programmatic resources)
• Tribal access of GAP resources:
A Tribal participation in technical and media-specific trainings
A Tribal participation in grants management training
• Tribal staff and activities funded through GAP (include solid waste implementation)
• Regional coordinators' perceptions about the quality, timeliness, and completeness of
work plans and progress reports received
• Timing of grant end date vs. final close out of the grant - this is an indicator of the degree
to which the grantee met program expectations - the shorter the period of time between
grant end date and final closeout, the more likely that tribes met expectations.
• Results of administrative post award monitoring audits
    A-l

-------
                  EVALUATION QUESTIONS
            INFORMATION THAT CAN HELP ANSWER QUESTION
2d. How does participation in GAP increase understanding of how to
develop a tribal environmental program?
•   Self reported increase in knowledge and understanding about the necessary steps in
    developing a tribal environmental program
•   Self-reported increase in skills needed to develop tribal environmental programs
•   Self-reported change in awareness and commitment to environmental programs in tribes
3a. What indicators of tribal environmental capacity exist?
3b. To what extent have tribes achieved environmental capacity as
suggested by the presence of these indicators?
Overarching Indicator of Tribal Environmental Capacity:
        • Number of GAP recipients that secured ongoing funding from other EPA
            sources.  [Note that availability of other sources of funding may be a limiting
           factor unrelated tribes' environmental capacity.}
•   Legal Capability
       •   Number of GAP recipients that have developed tribal codes, standards, and/or
           enforcement programs to control pollution
•   Enforcement Capability
        • Presence of tribal environmental staff person(s) charged with enforcement duties
•   Technical Capability
       •   Number of GAP recipients with one or more staff specifically tasked with
           managing environmental programs (e.g., Environmental Director)
       •   Size and composition of tribal environmental staff
       •   Number of environmental programs being carried out in different media annually
           by tribes.
       •   Number of GAP recipients that have taken environmental training
 •   Communications Capability
       •   Percent of GAP recipients that have conducted community education and
           outreach, based on the grant work plan
       •   Number of GAP recipients that have executed agreements with other
          jurisdictions for management of on- or off-reservation resources
       •   Tribal participation in EPA or tribal workgroups and/or Task Forces
       •   Extent of tribal environmental staffs communication of with tribal Council
                                                                        A-2

-------
                   EVALUATION QUESTIONS
         INFORMATION THAT CAN HELP ANSWER QUESTION
                                                                        Administrative Capability
                                                                          •   How long has person tasked with managing environmental programs (e.g.
                                                                              Environmental Director) been in that position
                                                                          •   Number of major findings on A133 audits (which are conducted for any tribe
                                                                              that spends $500K or more)
                                                                          •   On-site grants management review
3c. What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
Tribal Priorities
Tribal Staffing
Tribal Funding
Commun ication
Degree of tribal Council support for environmental programs
Does tribe have clear environmental priorities?
Rate of change in tribal governments - how often do Council members change?
Turnover rate of tribal Environmental Director and/or staff
Qualifications of tribal Environmental Director and/or staff
Tribal Environmental Staff and/or Director salary levels
GAP funding amounts
GAP funding consistency over time (e.g., number of consecutive GAP grants awarded
over time, range and average number of consecutive GAP grants received by tribes over
time)
Tribe's ability to secure funding beyond GAP (Note, securing funding is both an
indicator of tribal environmental capacity and a factor influencing achievement of
environmental capacity - through interview discussions, try to understand whether
securing other sources of funding is more a cause or effect of success in the GAP
program.)
Degree of information sharing among tribes
Degree to which tribes request information from EPA
Degree to which EPA provides information to tribes ~ specifically extent to which tribes
receiving hands-on technical assistance, oversight, or on-site visits.
                                                                        A-3

-------
                   EVALUATION QUESTIONS
             INFORMATION THAT CAN HELP ANSWER QUESTION
Regional Activities
•   Structure of EPA regional office - specifically, at what level the tribal office located (in
    the administrators office or elsewhere - the hypothesis is that the higher the level, the
    more likely it is that tribal issues will receive attention and resources).

•   Experience and longevity of EPA project officers and technical contacts

•   Grantee caseload for EPA project officers
3d. What is the relative contribution of GAP toward achieving
capacity?
•   Sequence of grants received by tribes (i.e., do tribes receive GAP grants before other
    grants? If so, this would suggest that GAP may contribute to ability to receive other
    grants)

•   Use of non-GAP funding
(Note - both of these indicators are limited by data on other sources of funds outside of EPA,
e.g. grants from BIA or tribes' internal funds.)
4. Is the GAP providing adequate outputs to achieve tribal goals and
priorities?
•   Tribes' perceptions about whether GAP is providing the type and amount of resources
    that they need to meet their environmental goals and priorities

•   Are there additional resources that tribes feel they would need to have in order to address
    their goals and priorities?

•   Is environmental capacity building a priority for tribes receiving GAP grants?
5. To what degree does GAP support EPA's strategic goal of
increasing tribes' ability to build environmental program capacity?

•   How do GAP's goals currently align with or diverge from EPA's
    strategic goal for GAP and other multi-media programs?

•   How do GAP's objectives align with or diverge from EPA's strategic
    goal for GAP and other multi-media programs?

•   How do tribal GAP activities align with or diverge from EPA's
    strategic goal for GAP and other multi-media programs?
•   Summary of GAP goals and objectives

•   Summary of EPA's strategic goals and objectives

•   Summary of tribal activities funded by GAP, and other outcomes assessed through the
    evaluation
                                                                         A-4

-------
                      Appendix B




INTERVIEW GUIDE FOR REGIONAL GAP PROJECT OFFICERS

-------
                                 APPENDIX B

                Interview Guide for GAP Regional Project Officers

       [Introductions] Thank you for agreeing to talk with us today. As you may know,
this interview is part of a broader evaluation of the GAP program that we are conducting
at the  request of EPA's American Indian Environmental Office (AIEO).   The EPA
General  Assistance Program  (GAP) as  envisioned by  Congress  includes  two key
elements:
1) To provide general assistance grants to build  capacity to administer  environmental
   regulatory programs.

2) To provide technical  assistance  from EPA to  tribal  governments and intertribal
   consortia in the development of multimedia programs to address environmental issues
   on tribal lands.

       While tribes and EPA focus on the funding aspects of GAP, technical assistance is
a substantial and significant component of the Program. Examples of assistance include:

•  EPA linking tribal staff with the appropriate EPA contacts.
•  EPA-sponsored training on administrative or technical skills needed for establishing
   tribal multimedia programs.
•  EPA  review of tribal  proposals for establishing programmatic capability, such  as
   codes, ordinances, and management plans.
•  EPA  site visits  to review  and assist tribes  with programmatic and  administrative
   decision making.

       The purpose of this interview is to  gather your  perspective  regarding  the
effectiveness of the GAP program and how GAP supports EPA's strategic goals  of
increasing  Tribes'  ability  to build environmental capacity.  Your participation in this
interview, along with input from Tribes and our initial  database research, will enhance
our understanding  of GAP and will  form an important source of information for this
evaluation.  Because "environmental capacity" is such a key concept for this evaluation,
we are using a definition  based  on EPA's 2000 GAP guidelines. For this evaluation,
"environmental capacity" means that a Tribe has established the administrative,  legal,
technical  and  enforcement capability necessary  to develop and implement a Tribal
environmental program,  as well as the communications capability to work with Federal,
State, Local, Tribal, and  other environmental officials.  We  ask that you  base your
answers to questions about environmental capacity  on this statutory definition.

       We have completed our analysis of data from several EPA databases that contain
information about Tribal grants and will ask you for your insights on these preliminary
findings during this interview. Upon completion of the data collection and analysis phase
of the evaluation, we will  compile the results and our conclusions in a report to AIEO,
which  will  be  available for you  to review.   We anticipate  that the results of this
evaluation will help AIEO demonstrate the successes of the Tribal GAP to stakeholders
and identify opportunities for improvement. Finally, we will maintain the confidentiality
                                       B-l

-------
of your responses to the interview questions; any data obtained through this interview
will be analyzed in aggregate with other interview data.

       Do you have any questions to ask before I start the interview?

Interview questions

    1.  To begin,  please describe how you are involved with GAP grants in your current
       position with EPA.
Evaluation Questions:  (la) Is GAP being accessed by all federally recognized
Tribes? (Ib) Why are some Tribes not involved in GAP? (Ic) Are there Tribes that
received GAP grants at one time but which no longer receive GAP grants?  If so,
why?
 The next two questions in this interview pertain to Tribes' participation in GAP in your
 Region:

    2.  To your knowledge, are there any federally recognized Tribes in your Region that
       have not received GAP grants since 1994? If yes, what do you think is the reason
       that these Tribes have not received GAP grants (choose all that apply):

       a  They have not applied for GAP grants. If so, why do you think they have not
          applied?
       a  Their applications have not been accepted. If so, what led them  to be not
          accepted?
       a  They have adopted a policy  of not accepting Federal grant money.
       a  Other reasons. (Please explain)

    3.  To your knowledge, are there any federally recognized Tribes in your Region that
       have received GAP grants since 1994, but no longer have a GAP grant?  If so,
       what do you believe caused the  Tribe to drop off of the GAP grant rolls?

    4.  For Tribes that did not receive  GAP grants, do you think the lack of GAP grants
       has hindered these Tribes' development of environmental programs?  How?
Evaluation Question:  (2a) Are Tribal governments using the resources (technical,
fiscal, and programmatic)  provided through the  GAP?   How  often  are GAP
resources accessed?
According to statute, GAP was established to provide general assistance to Tribes in the
form of monetary support to build environmental capacity as well as technical assistance
for developing multimedia environmental programs on Indian lands.  This next question
pertains to Tribes' utilization of all the resources -fiscal,  technical, and programmatic -
provided directly and indirectly through the GAP.
                                       B-2

-------
   5.  Overall, would you say Tribes in your Region utilize the resources (e.g., technical
       assistance and training, grants management training) provided as a component of
       GAP?  If yes,  which resources  do Tribes most frequently  access?   Which
       resources do Tribes least frequently access?
Evaluation Question:  (2c) To what extent have Tribes met program expectations
for grants management, execution of administrative functions, and carrying out
proposed activities?
The next set of questions pertains to the administration of GAP grants by Tribes in your
Region for which you have oversight:

   6.  To what extent have Tribes in your Region met expectations established by your
       region for GAP for each of the following areas?

Grants management
Execution of
administrative
functions
Carrying out
proposed activities
Almost
always



Often



Sometimes



Seldom



Almost
never



Don't
Know



   7.  To your knowledge, what proportion of Tribes in your Region submits GAP work
       plans and progress reports on time?

   8.  What trends have you seen in the  timeliness of GAP work plans and progress
       reports  submitted by Tribes in your Region over time?

   9.  To your knowledge, what proportion of Tribes in your Region submits GAP work
       plans and progress reports that are complete?

   10. What trends have you seen in the completeness of GAP work plans and progress
       reports  submitted by Tribes in your Region over time?

   11. To your knowledge, what proportion of Tribes in your Region submits quality
       GAP work plans and progress reports?

   12. What trends have you seen in the quality of Tribes' GAP work plans and progress
       reports  over time?
Evaluation Questions: (3a) What indicators of Tribal environmental capacity exist?
(3b) To what extent have Tribes achieved environmental capacity as suggested by
these indicators? (3c) What factors contribute to the achievement of environmental
capacity, and what is the impact of each factor?
                                      B-3

-------
    13. In your  experience, what is the best  indicator  that  a Tribe  is  achieving
       environmental capacity?

    14. To  your knowledge,  to what  extent  are  Tribes  in your  Region  achieving
       environmental capacity as defined by this indicator?

       a  Almost never
       a  Seldom
       a  Sometimes
       a  Often
       a  Almost always

    15. What proportion of Tribes in your Region would you say have received funding
       for environmental programs from sources other than GAP?

    16. In your experience, when a Tribe secures funding from sources outside of GAP,
       does that indicate that the Tribe has been successful in using its GAP grant to
       develop environmental capacity?

[Provide summary of findings from database analysis of legal, enforcement, technical
and communications capabilities, and ask for comments.]

The next set of questions pertains to administrative capability achieved by Tribes in your
Region that have received GAP funding. We are considering administrative capability as
one element of environmental capacity.

    17. What would you  say, historically, is the average  length of time for a Tribal
       environmental manager or director to remain in his or her position?

    18. Do you know if there have been any major findings on audits (e.g., A133,  IG,
       etc.) conducted for Tribes in your Region?  If yes, please describe.

    19. What information  do  you  have  about  the  results  of on-site  GAP  grants
       management reviews for Tribes in your Region?

The next set of questions pertains  to factors  that may influence the achievement of
environmental capacity:

   20. What factors would you  say are the most important contributors to Tribes' ability
       to achieve environmental capacity? Please describe how these factors influence
       Tribe's achievement of environmental capacity.

   21. Next, I am going to read you a list of factors that may influence Tribes ability to
       achieve environmental capacity.  Based on your knowledge and  experience, to
                                       B-4

-------
   what degree would you  say  each factor influences Tribes' ability to achieve
   environmental capacity?
For each factor where the interviewee indicates it almost always has an influence on
environmental  capacity,  follow  up  by  asking  how  the  indicator  influences
environmental capacity.

Access to funding outside
of GAP
Tribal Council support
for environmental
programs
Clear Tribal
environmental priorities
Planning documents
prepared by Tribes (e.g.,
five year plans)
Rate of change in Tribal
government leadership
(e.g., Council members)
Turnover rate of Tribal
Environmental Director
and/or staff.
Qualifications of Tribal
Environmental Director
and/or staff
Degree of information
sharing among Tribes
Frequency with which
Tribes request
information from EPA
Cohesiveness of a Tribe's
land base
The status of the Tribal
Office in your Region
(e.g., its location in the
organizational chart)
Experience, knowledge,
and longevity of EPA
project officers
Year-to-year shifts in
funding priorities due to
Almost
always













Often













Sometimes













Seldom













Almost
never













Don't
Know













                                    B-5

-------

changing priorities at
EPA
Different perceptions
about funding priorities
between Tribes and EPA
Extent to which the
Regions consult with the
Tribes
Changes in the GAP
funding process
Almost
always




Often




Sometimes




Seldom




Almost
never




Don't
Know




Evaluation Question:  (5) To what degree does GAP support EPA's strategic goal of
increasing Tribes' ability to build environmental program capacity?	
The final three questions pertain EPA's strategic goal of building Tribal environmental
capacity under Goal 5.3 of its 2003-2008 Strategic Plan:

   22. Based on your  understanding of GAP,  how  do you think GAP's  goals  and
      objectives  align  with  or  diverge from  EPA's  strategic  goal  for  Tribal
      environmental programs?

   23. Based on your understanding  of GAP,  how do you think Tribal GAP activities
      align with  or  diverge  from  EPA's  strategic goal  for  Tribal environmental
      programs?

   24. Based on your experience, how would  you change GAP to improve support for
      EPA's  strategic  goals for Tribal  environmental programs?   How would  you
      change GAP to improve support for Tribal goals and priorities?
                                      B-6

-------
          Appendix C




DISCUSSION GUIDE FOR TRIBES

-------
                                      Appendix C

          Discussion Guide and Proposed Questions for Tribal Panel Discussion

       Thank you for joining us today.  This panel discussion is part of a broader evaluation of
the General Assistance Program (GAP) that we are conducting at the request of EPA's American
Indian Environmental Office (AIEO).  We are conducting this discussion to solicit information
about Tribes' environmental  goals and your opinions on how GAP has supported your Tribe's
environmental programming efforts.   Your participation  on  the  panel will  enhance  our
understanding  of GAP and will  form an important source of information for this evaluation.
Upon completion of the data  collection and analysis phase of the evaluation, we will compile the
results and our conclusions in a report to AIEO, which will be available for you to review.

       As you may know, from EPA's perspective the primary purpose of the GAP is to help
federally recognized Tribes  and  intertribal  consortia  build the basic  components of  a Tribal
environmental program,  which  may  include  planning,  developing,  and  establishing  the
administrative, technical,  legal, enforcement, communication, and outreach infrastructure. The
primary purpose  of this evaluation is to determine how effective GAP has been in building Tribal
environmental capacity among those Tribes receiving funds.  Since "environmental capacity" is
such a key concept for this evaluation and we want to very clear about its meaning when we ask
about it in our questions, we are providing a definition based on EPA's 2000  GAP guidelines.

       For the purpose of this evaluation,  "environmental capacity" means that a Tribe has
established the administrative,  legal, technical and enforcement capability necessary to develop
and implement a Tribal environmental program,  as well as the  communications capability to
work with Federal,  State, Local, Tribal, and other environmental officials.

       You may  also be aware that the EPA General Assistance Program (GAP) as envisioned
by Congress includes two key elements:

   1)  To provide general assistance grants to build capacity to administer environmental
       regulatory programs.

   2)  To provide technical assistance from EPA to Tribal governments and intertribal consortia
       in the development of multimedia programs to address environmental issues on Tribal
       lands.

       While Tribes and EPA focus on the funding aspects of GAP, technical assistance is a
substantial and significant component of the Program. Examples of assistance include:

    •   EPA linking Tribal staff with the appropriate EPA contacts.
    •   EPA-sponsored training on administrative or technical skills needed  for establishing
        Tribal multimedia programs.
    •   EPA review of Tribal proposals for establishing programmatic capability, such as codes,
        ordinances, and management plans.
                                          C-l

-------
    •  EPA site visits to review and assist Tribes with programmatic and administrative
       decision-making.

       Note that this is an evaluation of the GAP program, not the Tribes.  Where we ask
questions about  Tribes'  achievements in building environmental capacity, or barriers to those
achievements,  the purpose of our question is to understand whether EPA's  GAP program is
working for its intended purpose for the wide array of Tribes that receive GAP grants, and how it
could be improved.   This evaluation is not intended as a critique of Tribes' environmental
achievements or a comparison of achievements across Tribes.

       We will begin our conversation with introductions and then proceed with a discussion of
Tribal environmental programs and your Tribes' participation in and experiences with GAP, per
the questions below.
Introductory Questions:

Let's begin with  questions  about  Tribes' environmental  goals  and  priorities  and  their
relationship to GAP and environmental capacity.  While Question 2,  below, asks for Tribal
definitions  of environmental capacity, we  ask  that you  base your answers to all subsequent
questions about environmental capacity on the statutory definition provided.

A. Tribal Environmental Capacity

    1.  Based on your understanding of GAP,  how do the program's goals compare with the
       environmental goals and priorities of your Tribe?

    2.  How does your Tribe define environmental capacity?

    3.  What is the most important indicator of a Tribe's  environmental capacity,  as defined by
       GAP?

B. Contributors to Environmental Capacity Development

    4.  Which factors influence environmental capacity attained by Tribes (for example, specific
       characteristics of Regional EPA offices or of Tribes)?

    5.  How have other program areas within EPA, e.g., media programs, helped your Tribe
       build environmental capacity?

    6.  How have non-EPA  entities, e.g.,  other federal  agencies,  state agencies, and NGOs,
       helped your Tribe build environmental capacity?
                                          C-2

-------
Transitional Questions:

Next let's discuss GAP grants  and their overall  impact on Tribes' level  of resources and
environmental programming efforts.

C. GAP Funding Support

   7.  In thinking about your Tribe's funding over  time, how has GAP supported your Tribe's
       environmental goals and priorities compared  to other funding sources?

   8.  Which resources provided by GAP (e.g., funds, technical assistance, training) have been
       most helpful to your Tribe's environmental programming efforts?  Which GAP resources
       have not been particularly helpful?

D. Other Sources of Support

   9.  What other  kind  of funding has your Tribe received  in  support of your Tribe's
       environmental goals and priorities?

   10. What additional resources  does your  Tribe need  to  develop your  environmental
       programs?

Key  Questions:

Now let's talk about specific ways in which GAP  has influenced Tribes' ability to carry  out
environmental programs.

E. GAP Influence on Tribal Approach to Environmental Programming

   11. How has GAP influenced your  Tribes  approach  to developing and sustaining Tribal
       environmental programs?

   12. How has GAP influenced the way you, Tribal members,  and particularly members of
       your Tribal Council:

       a.  Establish priorities and plan your environmental programs?

       b.  Administer your Tribe's environmental programs, (e.g., hiring, training, funding)?

       c.  Communicate to others within and outside your Tribe about environmental issues of
          importance to your Tribe?
                                         C-3

-------
F. Environmental Programs Funded by GAP

    13. What kinds of activities or program elements have been funded through your Tribe's
      GAP grants?  How  does  this compare to the activities  or program elements  funded
      through other funding sources your Tribe has received?

    14. Which of your environmental programs have benefited most from GAP? Which of these
      programs have benefited least from GAP?

Final Questions:

    15. Considering everything we have discussed so far, how would you change GAP to support
      your Tribe's goals and priorities?

    16. Have we missed anything?
                                         C-4

-------
              Appendix D




LIST OF TRIBES IN EVALUATION SAMPLE

-------
                                             Appendix D
Tribes Selected for GAP Sample
EPA
Region
1
1
2
4
5
5
5
5
5
5
5
6
6
6
6
6
6
6
6
6
6
6
6
6
6
7
7
8
8
8
8
8
8
Tribe Name1
Aroostook Band of Micmac Indians of Maine
Penobscot Tribe of Maine
Seneca Nation of New York2
Miccosukee Tribe of Indians of Florida
Bad River Band of the Lake Superior Tribe of Chippewa Indians of the Bad River
Reservation, Wisconsin
Fond du Lac Band
Grand Portage Band
Grand Traverse Band of Ottawa and Chippewa Indians, Michigan (previously listed as
the Grand Traverse Band of Ottawa & Chippewa Indians of Michigan)
Keweenaw Bay Indian Community, Michigan
Lower Sioux Indian Community in the State of Minnesota
Oneida Tribe of Indians of Wisconsin
Absentee-Shawnee Tribe of Indians of Oklahoma
Cherokee Nation, Oklahoma
Kaw Nation, Oklahoma
Muscogee (Creek) Nation, Oklahoma
Otoe-Missouria Tribe of Indians, Oklahoma
Pueblo of Laguna, New Mexico
Pueblo of Picuris, New Mexico
Pueblo of San Felipe, New Mexico
Pueblo of Taos, New Mexico
Seminole Nation of Oklahoma
Shawnee Tribe, Oklahoma
Tonkawa Tribe of Indians of Oklahoma
Wichita and Affiliated Tribes (Wichita, Keechi, Waco & Tawakonie), Oklahoma
Ysleta Del Sur Pueblo of Texas
Prairie Band of Potawatomi Nation, Kansas
Sac & Fox Tribe of the Mississippi in Iowa
Sisseton-Wahpeton Oyate of the Lake Traverse Reservation, South Dakota (formerly
the Sisseton-Wahpeton Sioux Tribe of the Lake Traverse Reservation)
Skull Valley Band of Goshute Indians of Utah
Southern Ute Indian Tribe of the Southern Ute Reservation, Colorado
Standing Rock Sioux Tribe of North & South Dakota
Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota
Ute Mountain Tribe of the Ute Mountain Reservation, Colorado New Mexico & Utah
In GAP
Database
Y
Y

Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Tribes
per
Region
2
1
1
7
14
2
6
        1 Per the BIA List published in 70 FR 71194 (11/25/05).
        2Although limited data for the Seneca Nation of New York are included in the GAP Activity table for the
years 2000-2003, there are no data for the tribe included in the GAP Position table. For this reason, we did not list
this tribe with the other for which data are available in both the Activity and Position tables. We used data for this
tribe obtained from file reviews to complement the limited data included in the GAP database.
                                                 D-l

-------
Tribes Selected for GAP Sample
EPA
Region
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
Tribe Name
Agua Caliente Band of Cahuilla Indians of the Agua Caliente Indian Reservation,
California
Buena Vista Rancheria of Me-Wuk Indians of California
Cabazon Band of Mission Indians, California (previously listed as the Cabazon Band
of Cahuilla Mission Indians of the Cabazon Reservation)
Campo Band of Diegueno Mission Indians of the Campo Indian Reservation,
California
Elem Indian Colony of Porno Indians of the Sulphur Bank Rancheria, California
Enterprise Rancheria of Maidu Indians of California
Ewiiaapaayp Band of Kumeyaay Indians, California (formerly the Cuyapaipe
Community of Diegueno Mission Indians of the Cuyapaipe Reservation)
Hopi Tribe of Arizona
Jamul Indian Village of California
Kashia Band of Porno Indians of the Stewarts Point Rancheria, California
La Jolla Band of Luiseno Mission Indians of the La Jolla Reservation, California
La Posta Band of Diegueno Mission Indians of the La Posta Indian Reservation,
California
Manchester Band of Porno Indians of the Manchester-Point Arena Rancheria,
California
Middletown Rancheria of Porno Indians of California
Paiute-Shoshone Indians of the Lone Pine Community of the Lone Pine Reservation,
California
Paiute-Shoshone Tribe of the Fallen Reservation and Colony, Nevada
Pala Band of Luiseno Mission Indians of the Pala Reservation, California
Paskenta Band of Nomlaki Indians of California
Quartz Valley Indian Community of the Quartz Valley Reservation of California
Quechan Tribe of the Fort Yuma Indian Reservation, California & Arizona
Redding Rancheria, California
Santa Rosa Indian Community of the Santa Rosa Rancheria, California
Santa Ynez Band of Chumash Mission Indians of the Santa Ynez Reservation,
California
Scotts Valley Band of Porno Indians of California
Sherwood Valley Rancheria of Porno Indians of California
Tuolumne Band of Me-Wuk Indians of the Tuolumne Rancheria of California
Habematolel Porno of Upper Lake, California (formerly the Upper Lake Band of Porno
Indians of Upper Lake Rancheria of California)
Walker River Paiute Tribe of the Walker River Reservation, Nevada
White Mountain Apache Tribe of the Fort Apache Reservation, Arizona
Yavapai- Apache Nation of the Camp Verde Indian Reservation, Arizona
In GAP
Database
Y

Y
Y
Y
Y

Y
Y
Y
Y
Y

Y
Y

Y
Y

Y

Y

Y
Y
Y
Y
Y
Y

Tribes
per
Region
30
D-2

-------
Tribes Selected for GAP Sample
EPA
Region
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
Tribe Name
Agdaagux Tribe of King Cove
Arctic Village
Chilkat Indian Village (Klukwan)
Chilkoot Indian Association (Haines)
Chinik Eskimo Community (Golovin)
Circle Native Community
CoeurD'Alene Tribe of the CoeurD'Alene Reservation, Idaho
Confederated Tribes of the Grand Ronde Community of Oregon
Confederated Tribes of the Siletz Reservation, Oregon
Egegik Village
Eklutna Native Village
Emmonak Village
Gulkana Village
Hoonah Indian Association
Inupiat Community of the Arctic Slope
Kenaitze Indian Tribe
Kootenai Tribe of Idaho
McGrath Native Village
Naknek Native Village
Native Village of Ambler
Native Village of Eagle
Native Village of Eyak (Cordova)
Native Village of Kivalina
Native Village of Kongiganak
Native Village of Kotzebue
Native Village of Kwigillingok
Native Village of Napaimute
Native Village of Nightmute
Native Village of Nunapitchuk
Native Village of Point Lay
Native Village of Ruby
Native Village of Selawik
Native Village of Tanacross
Native Village of Tatitlek
Native Village of Tazlina
Newtek Village
Nondalton Village
Northway Village
Organized Village of Kwethluk
Pedro Bay Village
Petersburg Indian Association
Sauk-Suiattle Indian Tribe of Washington
In GAP
Database


Y









Y











Y

















No. Tribes
per Region
48
D-3

-------
Tribes Selected for GAP Sample
EPA
Region
10
10
10
10
10
10
Sample
Size
Tribe Name
Shoshone-Bannock Tribes of the Fort Hall Reservation of Idaho
Skokomish Indian Tribe of the Skokomish Reservation, Washington
Twin Hills Village
Village of Aniak
Village of Salamatoff
Yakutat Tlingit Tribe

In GAP
Database







No. Tribes
per Region

111
D-4

-------
              Appendix E




GPRA CODES AND ACTIVITY CATEGORIES

-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 383 AIR ACTIVITIES
n 109 AIR ACTIVITIES
O 100 AIR ACTIVITIES
n 101 AIR ACTIVITIES
EH
102 AIR ACTIVITIES
EH
103 AIR ACTIVITIES
EH
104 AIR ACTIVITIES
EH 105 AIR ACTIVITIES
EH 384 AIR ACTIVITIES
EH 106 AIR ACTIVITIES
EH 107 AIR ACTIVITIES
EH
108 AIR ACTIVITIES
EH 119 AIR ACTIVITIES
EH 110 AIR ACTIVITIES
EH 111 AIR ACTIVITIES
EH
112 AIR ACTIVITIES
EH
113 AIR ACTIVITIES
EH
114 AIR ACTIVITIES
EH 115 AIR ACTIVITIES
EH 385 AIR ACTIVITIES
EH 116 AIR ACTIVITIES
EH 117 AIR ACTIVITIES
EH
118 AIR ACTIVITIES
EH GENERAL MANAGEMENT
236 AND ADMINISTRATION
EH GENERAL MANAGEMENT
428 AND ADMINISTRATION
EH
GENERAL MANAGEMENT
421 AND ADMINISTRATION

GENERAL MANAGEMENT
420 AND ADMINISTRATION
SUB1

External Air Quality
External Air Quality
External Air Quality

External Air Quality

External Air Quality

External Air Quality
External Air Quality
External Air Quality
External Air Quality
External Air Quality

External Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality

Indoor Air Quality

Indoor Air Quality

Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality
Indoor Air Quality

Indoor Air Quality



Communication


Communication


Communication
Grant Number(s):
SUB2 SUB3 Note


Air Grants Administration
Building Survey
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Source Inventory
Source Inventory-T1 4(9005)
Staff Program Capacity
Development

Air Grants Administration
Building Survey
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Source Inventory
Source Inventory-T1 4(9005)
Staff Program Capacity
Development




External Communication (e.g.,
with regulated community or other
governments)
Internal Communication (e.g., with
Tribal Executive, community, K-
12, or adult education)
   GENERAL MANAGEMENT
382 AND ADMINISTRATION
   GENERAL MANAGEMENT
225 AND ADMINISTRATION
Program Establishment
Activities
Program Establishment
Activities
Baseline Environmental
Assessment
                                                        Page 1 of 10

-------
Tribe Name:
   ACT GPRA I
                       CATEGORY
                                                  SUB1
                                Grant Number(s):
                                     SUB2
                                                                                                 SUB3
                                                                                                                               Note
                GENERAL MANAGEMENT
             228 AND ADMINISTRATION
                GENERAL MANAGEMENT
             229 AND ADMINISTRATION
                GENERAL MANAGEMENT
             230 AND ADMINISTRATION
Program Establishment
Activities
Program Establishment
Activities
Program Establishment
Activities
Enforcement Capability

Fiscal Administration Capacity



Fiscal Administration Capacity
Standards in
Place

Standards for
Property
Management
                GENERAL MANAGEMENT
             231 AND ADMINISTRATION
                GENERAL MANAGEMENT
             232 AND ADMINISTRATION
                GENERAL MANAGEMENT
             235 AND ADMINISTRATION
                GENERAL MANAGEMENT
             233 AND ADMINISTRATION
                GENERAL MANAGEMENT
             234 AND ADMINISTRATION
             436 GRANT WRITING
             429 GRANT WRITING
             430 GRANT WRITING
             431 GRANT WRITING
             432 GRANT WRITING

             433 GRANT WRITING

             434 GRANT WRITING
             435 GRANT WRITING
             381 LAND ACTIVITIES
             344 LAND ACTIVITIES
             333 LAND ACTIVITIES

             339 LAND ACTIVITIES
             338 LAND ACTIVITIES

             340 LAND ACTIVITIES
             341 LAND ACTIVITIES
             391 LAND ACTIVITIES
             342 LAND ACTIVITIES

             343 LAND ACTIVITIES
Program Establishment
Activities
Program Establishment
Activities

Staff

Staff

Staff

Clean Air Quality
Clean Water Activities
Cross Media Activities
Drinking Water Quality
General Management and
Administration
Solid and Hazardous Waste
Activities
Toxic Substances Activities

Asbestos
Asbestos

Asbestos
Asbestos

Asbestos
Asbestos
Asbestos
Asbestos

Asbestos
                            Standards for
Fiscal Administration Capacity    Procurement

Legal Capability
Develop Position Descriptions

Training Program
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
                                                                       Page 2 of 10

-------
Tribe Name:
   ACT GPRA I
                        CATEGORY
                                                    SUB1
                                 Grant Number(s):
                                      SUB2
                                                                                                    SUB3
                                                                                                                                   Note
             252 LAND ACTIVITIES
             237 LAND ACTIVITIES

             247 LAND ACTIVITIES
             246 LAND ACTIVITIES

             248 LAND ACTIVITIES
             249 LAND ACTIVITIES
             395 LAND ACTIVITIES
             250 LAND ACTIVITIES

             251 LAND ACTIVITIES
             268 LAND ACTIVITIES
             253 LAND ACTIVITIES

             263 LAND ACTIVITIES
             262 LAND ACTIVITIES

             264 LAND ACTIVITIES
             265 LAND ACTIVITIES
             396 LAND ACTIVITIES
             266 LAND ACTIVITIES

             267 LAND ACTIVITIES
             356 LAND ACTIVITIES
                                          Emergency Response
Emergency Response

Emergency Response
Emergency Response

Emergency Response
Emergency Response
Emergency Response
Emergency Response

Emergency Response
Hazardous Waste
Hazardous Waste

Hazardous Waste
Hazardous Waste

Hazardous Waste
Hazardous Waste
Hazardous Waste
Hazardous Waste

Hazardous Waste
Lead
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
             345 LAND ACTIVITIES

             351 LAND ACTIVITIES
             350 LAND ACTIVITIES

             352 LAND ACTIVITIES
             353 LAND ACTIVITIES
             392 LAND ACTIVITIES
Lead

Lead
Lead

Lead
Lead
Lead
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
                                                                          Page 3 of 10

-------
Tribe Name:
   ACT  GPRA I
                        CATEGORY
                                                     SUB1
                                 Grant Number(s):
                                      SUB2
                                                                                                     SUB3
                                                                                                                                    Note
             354 LAND ACTIVITIES

             355 LAND ACTIVITIES
             368 LAND ACTIVITIES
             357 LAND ACTIVITIES

             363 LAND ACTIVITIES
             362 LAND ACTIVITIES

             364 LAND ACTIVITIES
             365 LAND ACTIVITIES
             393 LAND ACTIVITIES
             366 LAND ACTIVITIES

             367 LAND ACTIVITIES
             380 LAND ACTIVITIES
             369 LAND ACTIVITIES

             375 LAND ACTIVITIES
             374 LAND ACTIVITIES

             376 LAND ACTIVITIES
             377 LAND ACTIVITIES
             394 LAND ACTIVITIES
             378 LAND ACTIVITIES

             379 LAND ACTIVITIES
             284 LAND ACTIVITIES
             269 LAND ACTIVITIES

             279 LAND ACTIVITIES
             278 LAND ACTIVITIES

             280 LAND ACTIVITIES
Lead

Lead
Pesticides
Pesticides

Pesticides
Pesticides

Pesticides
Pesticides
Pesticides
Pesticides

Pesticides
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)

PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)

PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)
PoPs (PCBs, Mercury)

PoPs (PCBs, Mercury)
Recycling
Recycling

Recycling
Recycling

Recycling
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g., Lead
Sources Inventory, Blood Lead
Survey, Field PoPs Survey, or
Building Condition Survey)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
                                                                           Page 4 of 10

-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 281 LAND ACTIVITIES
n 397 LAND ACTIVITIES
O 282 LAND ACTIVITIES
EH
283 LAND ACTIVITIES
n 300 LAND ACTIVITIES




285 LAND ACTIVITIES
CH
295 LAND ACTIVITIES
O 294 LAND ACTIVITIES
CH
296 LAND ACTIVITIES
n 297 LAND ACTIVITIES
O 398 LAND ACTIVITIES
n 298 LAND ACTIVITIES
n
299 LAND ACTIVITIES
O 316 LAND ACTIVITIES




301 LAND ACTIVITIES
EH
311 LAND ACTIVITIES
n 310 LAND ACTIVITIES
CH
31 2 LAND ACTIVITIES
O 313 LAND ACTIVITIES
n 399 LAND ACTIVITIES
O 314 LAND ACTIVITIES
EH
315 LAND ACTIVITIES
n 332 LAND ACTIVITIES
SUB1
Recycling
Recycling
Recycling

Recycling
Solid Waste




Solid Waste

Solid Waste
Solid Waste

Solid Waste
Solid Waste
Solid Waste
Solid Waste

Solid Waste
Superfund




Superfund

Superfund
Superfund

Superfund
Superfund
Superfund
Superfund

Superfund
UST
Grant Number(s):
SUB2 SUB3 Note
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development

Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development

Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development

317 LAND ACTIVITIES
                               UST
Baseline Assessment (e.g.,
Sources Inventory, Superfund Site
Identification, Site Evaluation, Site
Development, Remediation, and
Waste Stream Characterization)
                                                                 Page 5 of 10

-------
Tribe Name:
ACT GPRA 1 CATEGORY
n
327 LAND ACTIVITIES
O 326 LAND ACTIVITIES
EH
328 LAND ACTIVITIES
n 329 LAND ACTIVITIES
O 400 LAND ACTIVITIES
n 330 LAND ACTIVITIES
n
331 LAND ACTIVITIES
O SPECIAL EMPHASIS
205 ACTIVITIES
O SPECIAL EMPHASIS
164 ACTIVITIES
O SPECIAL EMPHASIS
155 ACTIVITIES
O SPECIAL EMPHASIS
158 ACTIVITIES
O SPECIAL EMPHASIS
159 ACTIVITIES
O SPECIAL EMPHASIS
160 ACTIVITIES
O SPECIAL EMPHASIS
161 ACTIVITIES
O SPECIAL EMPHASIS
162 ACTIVITIES
O SPECIAL EMPHASIS
163 ACTIVITIES
O SPECIAL EMPHASIS
174 ACTIVITIES
O SPECIAL EMPHASIS
165 ACTIVITIES
O SPECIAL EMPHASIS
168 ACTIVITIES
O SPECIAL EMPHASIS
169 ACTIVITIES
O SPECIAL EMPHASIS
170 ACTIVITIES
O SPECIAL EMPHASIS
171 ACTIVITIES
O SPECIAL EMPHASIS
172 ACTIVITIES
O SPECIAL EMPHASIS
173 ACTIVITIES
O SPECIAL EMPHASIS
184 ACTIVITIES
SUB^

UST
UST

UST
UST
UST
UST

UST



Endangered Species

Endangered Species

Endangered Species

Endangered Species

Endangered Species

Endangered Species

Endangered Species

Endangered Species

Environmental Justice

Environmental Justice

Environmental Justice

Environmental Justice

Environmental Justice

Environmental Justice

Environmental Justice

Environmental Justice
NEPA/TEPA/Cultural
Resources
Grant Number(s):
SUB2 SUB3 Note
Developing Permitting/Licensing
Authority
Development of a Database
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grant Administration
Staff Program Capacity
Development




Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity

Development of QAPPs

Grants Administration
Wetland Identification and
Delineation


Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity

Development of QAPPs

Grants Administration
Wetland Identification and
Delineation


Page 6 of 10

-------
Tribe Name:
   ACT GPRA  1
                       CATEGORY
                                                    SUB1
                                 Grant Number(s):
                                      SUB2
                                                                                                    SUB3
                                                                                                                                  Note
                 SPECIAL EMPHASIS
             175 ACTIVITIES
                 SPECIAL EMPHASIS
             178 ACTIVITIES
                 SPECIAL EMPHASIS
             179 ACTIVITIES
                 SPECIAL EMPHASIS
             180 ACTIVITIES
                 SPECIAL EMPHASIS
             181 ACTIVITIES
                 SPECIAL EMPHASIS
             182 ACTIVITIES
                 SPECIAL EMPHASIS
             183 ACTIVITIES
                 SPECIAL EMPHASIS
             194 ACTIVITIES
                 SPECIAL EMPHASIS
             185 ACTIVITIES
                 SPECIAL EMPHASIS
             188 ACTIVITIES
                 SPECIAL EMPHASIS
             189 ACTIVITIES
                 SPECIAL EMPHASIS
             190 ACTIVITIES
                 SPECIAL EMPHASIS
             191 ACTIVITIES
                 SPECIAL EMPHASIS
             192 ACTIVITIES
                 SPECIAL EMPHASIS
             193 ACTIVITIES
                 SPECIAL EMPHASIS
             204 ACTIVITIES
                 SPECIAL EMPHASIS
             195 ACTIVITIES
                 SPECIAL EMPHASIS
             198 ACTIVITIES
                 SPECIAL EMPHASIS
             199 ACTIVITIES
                 SPECIAL EMPHASIS
             200 ACTIVITIES
                 SPECIAL EMPHASIS
             201 ACTIVITIES
                 SPECIAL EMPHASIS
             202 ACTIVITIES
                 SPECIAL EMPHASIS
             203 ACTIVITIES
             154 WATER ACTIVITIES
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources
NEPA/TEPA/Cultural
Resources

Pollution Prevention

Pollution Prevention

Pollution Prevention

Pollution Prevention

Pollution Prevention

Pollution Prevention

Pollution Prevention

Pollution Prevention

Sustainable Development

Sustainable Development

Sustainable Development

Sustainable Development

Sustainable Development

Sustainable Development

Sustainable Development

Sustainable Development
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity

Development of QAPPs

Grants Administration
Wetland Identification and
Delineation
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity

Development of QAPPs

Grants Administration
Wetland Identification and
Delineation
Baseline Assessment (e.g.,
Sources Inventory, Survey)
Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity

Development of QAPPs

Grants Administration
Wetland Identification and
Delineation
                                                                          Page 7 of 10

-------
Tribe Name:
ACT GPRA 1 CATEGORY
O 224 WATER ACTIVITIES
n 401 WATER ACTIVITIES
O 403 WATER ACTIVITIES
n 404 WATER ACTIVITIES
406 WATER ACTIVITIES
407 WATER ACTIVITIES
408 WATER ACTIVITIES
O 409 WATER ACTIVITIES
n 405 WATER ACTIVITIES
410 WATER ACTIVITIES
O 131 WATER ACTIVITIES
n 121 WATER ACTIVITIES
SUB1

Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Ground Water
Non-Point Sources
Non-Point Sources
Grant Number(s):
SUB2 SUB3 Note


Administering EPA Grant
Baseline Assessment
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
Administering EPA Grant
122 WATER ACTIVITIES
126 WATER ACTIVITIES
127 WATER ACTIVITIES
128 WATER ACTIVITIES
129 WATER ACTIVITIES
386 WATER ACTIVITIES
130 WATER ACTIVITIES
402 WATER ACTIVITIES
412 WATER ACTIVITIES
41 3 WATER ACTIVITIES
41 5 WATER ACTIVITIES
41 6 WATER ACTIVITIES
41 7 WATER ACTIVITIES
418 WATER ACTIVITIES
41 4 WATER ACTIVITIES
41 9 WATER ACTIVITIES
21 4 WATER ACTIVITIES
206 WATER ACTIVITIES
207 WATER ACTIVITIES
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Non-Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Point Sources
Source Water Protection
Source Water Protection
Source Water Protection
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development

Administering EPA Grant
Baseline Assessment
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development

Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
      Page 8 of 10

-------
Tribe Name:
ACT GPRA 1 CATEGORY
n
208 WATER ACTIVITIES
O 209 WATER ACTIVITIES
n 389 WATER ACTIVITIES
O 210 WATER ACTIVITIES
n 211 WATER ACTIVITIES
CH
21 2 WATER ACTIVITIES
CH

213 WATER ACTIVITIES
n 223 WATER ACTIVITIES
CH
21 5 WATER ACTIVITIES
CH
21 6 WATER ACTIVITIES
CH
21 7 WATER ACTIVITIES
O 218 WATER ACTIVITIES
n 390 WATER ACTIVITIES
O 219 WATER ACTIVITIES
n 220 WATER ACTIVITIES
CH
221 WATER ACTIVITIES
CH

222 WATER ACTIVITIES
SUB1

Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection
Source Water Protection

Source Water Protection


Source Water Protection
Underground Injection Control

Underground Injection Control

Underground Injection Control

Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control
Underground Injection Control

Underground Injection Control


Underground Injection Control
Grant Number(s):
SUB2 SUB3 Note
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grants Administration
Source Water Inventory
Staff Program Capacity
Development
Wetland Identification and
Delineation. NOT IN USE.
PLEASE RECLASSIFY

Developing Permitting/Licensing
Authority
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Grants Administration
Source Water Inventory
Staff Program Capacity
Development
Wetland Identification and
Delineation. NOT IN USE.
PLEASE RECLASSIFY
142 WATER ACTIVITIES       Watershed
132 WATER ACTIVITIES       Watershed
133 WATER ACTIVITIES       Watershed

137 WATER ACTIVITIES       Watershed

138 WATER ACTIVITIES       Watershed

139 WATER ACTIVITIES       Watershed
140 WATER ACTIVITIES       Watershed
387 WATER ACTIVITIES       Watershed

141 WATER ACTIVITIES       Watershed
153 WATER ACTIVITIES       Wetlands
143 WATER ACTIVITIES       Wetlands


144 WATER ACTIVITIES       Wetlands
Administering EPA Grant
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development

Administering EPA Grant
Baseline Assessment (e.g., NPS
Source Inventory, Watershed
Inventory)
                                                            Page 9 of 10

-------
 Tribe Name:
\    ACT GPRA 1
          CATEGORY
                                      SUB1
                                                            Grant Number(s):
                                                                SUB2
                                                                                    SUB3
                                                                                                                  Wore
p
p
p
n
148 WATER ACTIVITIES        Wetlands

149 WATER ACTIVITIES        Wetlands

150 WATER ACTIVITIES        Wetlands
151 WATER ACTIVITIES        Wetlands
388 WATER ACTIVITIES        Wetlands

152 WATER ACTIVITIES        Wetlands
Developing Water Quality
Standards
Development of Codes and
Ordinances
Development of Monitoring
Capacity
Development of QAPPs
General Program Development
Staff Program Capacity
Development
                                                                        Page 10 of 10

-------
          Appendix F




TRIBAL POPULATION STATISTICS

-------
      APPENDIX F: POPULATION INFERENCES BASED ON SAMPLE DATA




Exhibit 1 below shows confidence intervals for the population proportions of various indicators.
Exhibit 1: Summary of Population Proportions

Indicator*3'
Percent of tribes that participated in activities to
increase legal capacity
Percent of tribes that developed codes,
ordinances, or standards
Percent of tribes that adopted/implemented
codes, ordinances, or standards
Percent of tribes that participated in enforcement
activities
Percent of tribes that have at least one
professional staff member
Percent of tribes participating in water activities
Percent of tribes participating in waste activities
Percent of tribes participating in air activities
Percent of tribes participating in internal
communication activities
Percent of tribes participating in external
communication activities
Percent of tribes participating in general
communication activities
Percent of tribes that took advantage of technical
resources
Percent of tribes that took advantage of
programmatic resources
Percent of non-GAP grants received
concurrently with GAP funding
Percent of tribes receiving non-GAP funding
Sample Proportion
25%
26%
7%
26%
90%
73%
73%
49%
71%
69%
29%
76%
23%
90%
62%
Sample Size(b)
96
96
96
96
96
96
96
96
96
96
96
96
96
1242(c)
111
95 Percent
Confidence Interval
Low
16%
17%
2%
17%
83%
64%
64%
39%
62%
59%
20%
68%
15%
88%
53%
High
34%
35%
12%
35%
96%
82%
82%
59%
80%
78%
38%
85%
31%
91%
71%
(a) We could not calculate population proportions for the percentage of tribes with unfavorable audit results, because
results do not meet the criteria of approximately normal distribution.
(b) Unless otherwise noted, sample size refers to the number of tribes.
(c) In this case, sample size refers to number of grants, rather than number of tribes.
                                      F-l

-------
Exhibit 2 below shows confidence intervals for the population means for various indicators.
Exhibit 2: Summary of Population Means

Indicator
Amount of GAP grant
Number of technical resources accessed
Number of programmatic resources accessed
Number of months between project end and closeout date
Number of professional and technical FTEs hired

Sample Mean
$102,472
4.4
1.2
12.7
1.2

Sample Size(a)
754
96
40
175 (b)
66
95 Percent
Confidence Interval
Low
$ 97,957
3.4
0.8
11.4
1.1
High
$106,986
5.3
1.6
14.0
1.4
(a) Unless otherwise noted, sample size refers to the number of tribes.
(b) In this case, sample size refers to number of grants, rather than number of tribes.
                                           F-2

-------
                    Appendix G




ACTIVITY COMPARISON ACROSS TRIBE SAMPLE GROUPS

-------
                                                    Appendix G
                              Percent of Tribes with Activities in each Category
         100%
           0%
                   General    Land Activities     Water     Grant Writing  Air Activities      Special
                 Management                  Activities                                  Emphasis
                    and                                                                Activities
                Administration
                                 D Tribes in Database • Tribes with File Reviews
                             Percent of Tribes Participating in each Activity Type
100%
                                       O Tribes in Database • Tribes in File Reviews
                                                     G-l

-------
                  Appendix H




REGIONAL GAP PROJECT OFFICERS INTERVIEWED

-------
Appendix H
INTERVIEWEES: REGIONAL PROJECT OFFICERS FOR GAP
EPA Region
Region 1
Region 2
Region 4
Region 5
Region 6
Region 7
Region 8
Region 9
Region 10
Cross-Region/Region 4
Primary Interviewee
Jean Crocker
Christine Yost
Cynthia Nolan
Michael Nishi
Dale Roy
Wolfgang Brandner
Judith Hervig (not completed)
Timothy Wilhite
Alan Moomaw
Dan Clone
   H-l

-------
                Appendix I




TRIBES PARTICIPATING IN PANEL DISCUSSIONS

-------
                                      Appendix I


United South and Eastern Tribes (USET) 2007 Impact Week Meeting, Arlington, VA1
    •  Miccosukee Tribe of Indians of Florida (Panelist)
    •  Poarch Band of Creek Indians of Alabama (Panelist)
    •  Narragansett Indian Tribe of Rhode Island (Panelist)

EPA Region 5 2007 Indian GAP Conference Week, Chicago, IL
    •  Forest County Potawatomi Community, Wisconsin (Panelist)
    •  Little Traverse Bay Bands of Odawa Indians, Michigan (Panelist)
    •  Grand Traverse Band of Ottawa and Chippewa Indians, Michigan
    •  Huron Potawatomi, Inc., Michigan
    •  Lac Courte Oreilles Band of Lake Superior Chippewa Indians of Wisconsin (Panelist)
    •  Leech Lake Band of Minnesota Chippewa
    •  Little River Band of Ottawa Indians, Michigan
    •  Mille Lacs Band of Minnesota Chippewa DNR/E
    •  Pokagon Band of Potawatomi Indians, Michigan and Indiana
    •  Quinault Tribe of the Quinault Reservation, Washington
    •  Saginaw Chippewa Indian Tribe of Michigan

EPA Region 8 Tribal Operations Committee Meeting, Denver, CO
    •  Ute Mountain Tribe of the Ute Mountain Reservation, Colorado, New Mexico & Utah
      (Panelist)
    •  Turtle Mountain Band of Chippewa Indians of North Dakota (Panelist)
    •  Confederated Salish & Kootenai Tribes of the Flathead Reservation, Montana (Panelist)
    •  Assiniboine and Sioux Tribes of the Fort Peck Indian Reservation, Montana
    •  Blackfeet Tribe of the Blackfeet Indian Reservation of Montana
    •  Flandreau Santee Sioux Tribe of South Dakota
    •  Northern Cheyenne Tribe of the Northern Cheyenne Indian Reservation, Montana
    •  Shoshone Tribe of the Wind River Reservation, Wyoming
    •  Sisseton-Wahpeton Oyate of the Lake Traverse Reservation, South Dakota
    •  Southern Ute Indian Tribe of the Southern Ute Reservation, Colorado
    •  Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota
    •  Ute Indian Tribe of the Uintah & Ouray Reservation, Utah
       1 Due to inclement weather, we were not able to conduct a panel discussion with tribes at this event. We
instead conducted separate interviews with each of the tribal representatives on the panel.
                                          1-1

-------
                    Appendix J




SOURCES OF NON-GAP FUNDING AND PROGRAM SUPPORT

-------
                                         Appendix J
                Sources of Non-GAP Funding and Program Support Received by Tribes

EPA Programs	
•   Clean Water Act (CWA) Section 106 - water program infrastructure, staffing, water quality standards
    development, well-protection plans, water quality lab, and training.
•   CWA Section 106b (since cut) - wetland management planning
•   CWA Section 319 - non point source pollution, water quality standards development
•   Solid waste - solid waste planning, code development, staffing, recycling
•   Clean Air Act Section 103 - air program, mercury deposition
•   Resource Conservation and Recovery Act (RCRA) Support Agency Cooperative Agreement
•   Underground Storage Tanks (UST) and Underground Injection Control (UIC) - Direct Implementation
    Tribal Cooperative Agreement (DITCA)
•   Public Water Systems Safety (since cut) - water treatment/facilities
•   Brownfields
•   Pesticides
•   Asthma Program
•   Environmental Justice	
Non-EPA Agencies and Programs	
•   U.S. Natural Resources Conservation Service (NRCS) - sedimentation issues, road issues, technical data,
    forest management, erosion control, soil protection
•   U.S. Department of Agriculture (USDA) (Rural development) - solid waste programs, equipment, buildings
•   U.S. Fish and Wildlife Service (FWS) - technical assistance, travel funding, forestry, water testing, burning,
    wildlife habitat
•   U.S. Bureau of Indian Affairs (BIA) - surface water cleaning, land resources, water resources, fish and
    wildlife compliance, land use enforcement, forestry, road maintenance, parks, wetlands permits
•   U.S. Bureau of Reclamation (BOR) - water activities; septic issues
•   U.S. Indian Health Service (HS) - solid waste  programs, equipment, buildings
•   U.S. Department of Health and Human Services (DHHS) - Administration for Native Americans (ANA)
•   U.S. Army Corps of Engineers - land use/wetlands permitting
•   State entities - WI Department of Natural Resources
•   Non-profits - National Groundwater Assn., North American Waste Management Society, Tip-of-the-Mitt
    Watershed Council, WI Assn. Of Lakes, Little Traverse Bay Organization, West Virginia University	
                                               J-l

-------