ASSESSMENT OF
INAUGURAL TWO-YEAR
NPM GUIDANCE PROCESS
OFFICE OF THE CHIEF FINANCIAL OFFICER
OFFICE OF PLANNING, ANALYSIS AND ACCOUNTABILITY
July 12, 2016
Publication # 190B16002
-------
ASSESSMENT OF INAUGURAL TWO-YEAR
NPM GUIDANCE PROCESS
CONTENTS
I. INTRODUCTION AND OVERVIEW 2
II. PARTICIPANTS 4
III. MAJOR FINDINGS 5
APPENDIX: SUMMARY OF RESPONSES 9
1. Early Engagement: Process 10
2. Early Engagement: Substance 14
3. Incorporating Flexibility 21
4. Aligning Programmatic Grant Guidances with NPM Guidance Cycle 25
5. Earlier and More Meaningful Engagement with States and Tribes on Commitment-Setting 29
6. Reviews 32
7. Collaboration/Communication Tools 36
8. Other 40
-------
ASSESSMENT OF INAUGURAL TWO-YEAR
NPM GUIDANCE PROCESS
I. INTRODUCTION AND OVERVIEW
The two-year National Program Manager (NPM) Guidance process, developed by the EPA-State
NPM Guidance/NEPPS Workgroup in 2014-2015, was first implemented in developing the 2016-
2017 NPM Guidances. Consistent with the evaluation framework established by the
Workgroup, EPA's Office of Planning, Analysis and Accountability (OPAA) in consultation with
the agency's Office of Congressional and Intergovernmental Relations (OCIR) developed an
assessment instrument to obtain feedback on the impact of key changesearly, more
meaningful engagement and flexibilitiesin the first cycle of the new two-year NPM guidances.
Importantly, the results of this assessment are intended to facilitate discussions among EPA
Headquarters, Regional and state staff and management - to identify opportunities to improve
the two-year process for the FY 2018-2019 NPM Guidance development. The assessment
instrument is not a statistical survey. Although it is likely that responses from individual
recipients convey the sentiments of members of their organizations, the results of this
assessment cannot be interpreted as representing theirs or other organizations.
The assessment instrument was developed using Google Forms and distributed to potential
participants on April 12, 2016, as an internet link in an email from Kathy O'Brien, the Director of
OPAA. Respondents were asked to complete the assessment by April 29. The assessment
included a brief overview of the following major components (key changes) of the two-year
process.
1. Early Engagement: Process
2. Early Engagement: Substance
3. Incorporating Flexibility
4. Aligning Programmatic Grant Guidances with NPM Guidance Cycle
5. Earlier and More Meaningful Engagement with States and Tribes on Commitment-Setting
6. Reviews
7. Communication/Collaboration Tools
Respondents were asked to provide their contact information and organization to enable
follow-up to responses, if necessary, and to provide a basis for analysis. For each component of
the two-year process listed above, respondents were asked to rate, using a five point scale,
how well each of the components functioned, and to provide written responses about (1) what
aspects worked well and should be continued, and (2) what did not work well and should be
improved.
-------
The remainder of this assessment is organized into three main sections.
Section II characterizes the participants (respondents) in the assessment
Section III provides major findings at an executive level
The Appendix: Summary of Responses describes the responses to the assessment and is
organized according to the seven major components (key changes) of the two-year NPM
Guidance process.
-------
II. PARTICIPANTS
The assessment was sent to approximately 130 people, including NPM Guidance/NEPPS
workgroup members, ECOS, EPA Partnership and Performance Workgroup (P&P) co-chairs, EPA
NPM and Regional Planners, Lead Region Coordinators, and Senior Budget Officers (SBO). EPA
employees were also allowed to forward the survey to other agency managers/staff who could
provide valued input. Responses were received by 29 individuals and the estimated response
rate is no greater than 22 percent. Respondents include:
8 EPA HQ offices (OAR, OCSPP, OECA, OEI, OITA, OLEM, OP, and OW)
6 EPA regions (1, 3, 4, 5, 8, and 10)
5 states (Colorado, Missouri, New Hampshire, Tennessee, and Virginia)
Respondents by Organization Type
(Number, Percent)
HQ Region State
Respondents perform a variety of roles related to NPM Guidances and NEPPS, including EPA
SBOs; HQ/NPM, EPA regional, and state planners; NEPPS coordinators and Performance
Partnership Grant (PPG) managers; P&P Workgroup members; and agency staff involved in
Annual Commitments and measures (ACS).
Respondent Role
ACS
HQ Planner
NEPPS/PPG
NPM Planner
P&P
Regional Planner
SBO
Tribal Program
Total
Number
6
3
9
2
1
5
2
1
29
-------
III. MAJOR FINDINGS
The purpose of this section is to provide a high level summary of the results from the on-line
assessment of the two-year NPM Guidance process. The on-line assessment instrument was designed
for - and successful in - eliciting candid and constructive feedback from stakeholder groups to inform
improvements to the two-year NPM Guidance process. Although the assessment was not designed to
produce statistically representative results, we received a fair number of responses from all three
stakeholder groups (states, EPA regions and EPA headquarters) and it is possible that several
respondents answered questions as representative of their organizations rather than from their
individual personal perspectives. The response rate to the on-line assessment instrument is consistent
with what generally is observed in contemporary survey research (around 20 percent).
While participants generally expressed support for each of the seven key areas of the two-year NPM
Guidance process, within each stakeholder group (States, EPA Regions and EPA Headquarters)
participants expressed a wide range of viewpoints.
In this Assessment we emphasize those responses that indicate potential opportunities for - and which
could facilitate discussion on - improving and more consistently implementing the two-year NPM
guidance process. More details on individual responses can be found in Section IV.
1. Early Engagement: Process
The majority of participants felt that the early engagement process worked well: all five of the state
participants ranked the early engagement process as effective (4 out of 5=it was the best thing since
sliced bread) and 18 of the total 26 participants (70 percent) rated the early engagement process as
effective (4).
Although participants overall felt that early engagement generally was effective, there was a wide range
of responses from EPA Region and HQ participates: 30 percent of HQ participants and almost 46 percent
of Regional participants ranked the process as a level 2 or 3, which considered together with written
comments indicate a potential to improve how early engagement is implemented.
Potential Areas for Improvement:
Communication of roles and responsibilities during early engagement (especially for EPA
Regions), and advance notice to EPA Regions of early engagement with states and tribes
Clear and concise materials to focus discussion with states and tribes
Discussion/dialogue with states and tribes, and less "report out" by EPA
Outreach to non-traditional stakeholders (such as industry, non-governmental organizations,
academia, the research community, international organizations, local agencies, agricultural and
manufacturing sectors, small businesses, communities with environmental justice concerns)
Communication/feedback to states and tribes on how the results of early engagement informed
the NPM Guidances.
2. Early Engagement: Substance
Participants were widely divided on the extent to which early engagement informed national areas of
focus, program-specific guidance, and areas for improved cross-program coordination. While 3 out of 4
state participants rated the substantive impact of early engagement as informative, ranking it a 4 out of
-------
5 (l=not at all informative; 5=extremely informative), the largest number of participants (46 percent)
rated the substantive impact of early engagement as only a 3, indicating a high degree of ambivalence.
EPA Regional and HQ responses covered a broad spectrum from 2 to 5. This variability suggests the
potential to identify possible best practices and the possibility of improving the consistency of how early
engagement is implemented.
Potential Areas for Improvement:
Education on the purpose and use of the NPM Guidances
Collaboration with states and tribes in routine business operations, "so that joint planning and
evaluation is ongoing at all levels of state/EPA partnership work"
Communication of changes that will affect state and tribal work
Opportunities for states, tribes and EPA regions to communicate their specific priorities
Identification and establishment of expectations/goals for cross-program issues.
3. Flexibility
Participants also expressed a wide range of views on the extent to which the NPM Guidance process was
successful in providing support for EPA, states, and tribes to pursue flexibilities that advance the most
important environmental and human health work. The largest number of participants (44 percent) rated
flexibility a 3 out of 5 (l=not successful; 5=extremely successful), but almost 40 percent rated it effective
(4 out of 5). States participants were more positive than that of HQ and the Regions on the extent to
which the NPM Guidances supported flexibilities: 4 out of 5 states participants rated the process
effective (4). Seventy (70) percent of HQ participants and 75 percent of Regional participants rated the
process as level 2 or 3.
Potential Areas for Improvement.
Awareness by state and Regional staff and management (beyond the "NPM Guidance/planning
universe") of the purpose of the NPM Guidances and flexibilities provided through the NPM
Guidances, and examples of past tradeoffs resulting from flexibilities identified in the NPM
Guidances
Outreach to states about flexibilities available through Performance Partnerships
Alignment of grant guidances and timing of NPM Guidances and grant work planning
Latitude in developing "common-sense work plans," balancing accountability and flexibility
Incentives for and use of flexibilities to improve systems-based approaches.
4. Aligning Program Grant Guidance and NPM Guidance Cycle
While responses varied greatly within all three stakeholder groups, less than one-third of participants
felt that workloads/burdens were reduced through aligning the grant and NPM Guidance processes, and
the mean response was less than 3 out of 5 (l=no effect on workloads/burden; 5=significant reduction
in workloads/burden associated with grant work planning).
These results should be interpreted with caution, however: as reflected in several written comments the
real impacts of aligning the grants and NPM Guidance processes may not be realized until after
completing the FY 2017 grants work planning, a process that had not yet been initiated at the time of
this assessment.
-------
Potential Areas for Improvement:
Communication of expectations, latitude, and key changes for states and tribes
Schedule for sharing grant allocations (e.g., in Winter/early Spring) to gain greater
administrative efficiencies
Two-year work planning
Collaboration with Green Grants workgroup and incorporation of Green Grants Policy.
5. Earlier and More Meaningful Engagement with States and
Tribes on Commitment-Setting
The impact of early engagement on commitment setting also received mixed reviews from participants,
with an average rating of only 3 out of 5 (l=ineffective; 5=extremely effective). Only about 40 percent of
participants felt early engagement was effective in facilitating grant work planning, with 2 HQ and 2
Regional participants rating the process as not effective (a 1 or 2 out of 5). This is yet another area
where there may be opportunities to identify best practices and improve the consistency of how we
implement the two-year NPM Guidance process.
Potential Areas for Improvement:
Communication of the purpose of NPM Guidances vis-a-vis commitment-setting
Outreach to non-traditional stakeholders (see 1, above)
Common level knowledge/understanding amongst EPA Regions, EPA HQ and states on the
relationship between commitments, grants and monitoring, to facilitate more productive
discussions with the states and tribes
Synchronization of state and NPM Guidance planning cycles
Emphasis on outcomes in commitment-setting over two-year NPM Guidance cycle.
6. Reviews
Almost half of the participants felt that internal and external reviews were effective for eliciting
meaningful input on draft NPM Guidances, rating it 4 or 5 out of 5 (l=ineffective; 5=extremely effective).
Participants again were widely divided, however, rating internal and external reviews from ineffective to
extremely effective.
Potential Areas for Improvement:
Communication/advertisement about the opportunity for external review, including the
objectives of the review process
Tools for efficiently submitting - and approach for reviewing/responding - to external
comments (e.g., make it electronic, similar to the on-line assessment)
Schedule/time for internal reviews
Input and feedback at ECOS conference calls
Educational materials and opportunities to increase understanding of NPM Guidance and cross-
agency work (e.g., sustainability)
Basic knowledge by tribes of the entire planning, budgeting, and funding system
Examples of workload reductions and increased flexibility in grant negotiations that resulted
from the NPM Guidance process.
-------
7. Collaboration/Communication Tools
Sixty (60) percent of participants felt that EPA's collaboration tools were effective in facilitating work,
communication, and information sharing, rating these tools 4 or 5 out of 5 (l=ineffective; 5=extremely
effective). Yet, a substantial number of participants (8 out of 25) were ambivalent (rating it 3 out of 5)
and two participants (a HQ and Regional participant) felt communication tools were not very effective
(rating them a 2 out of 5).
Potential Areas for improvement:
Use of multimedia and innovative approaches to promote collaboration around media/multi-
program issues
Training for new staff on NPM Guidance process and use of collaboration tools
Organization and updating of NPM Guidance internet and SharePoint sites.
8. Other Opportunities for Improvement
Annual Commitment System (ACS)
Last minute requests for inclusion of new information in NPM Guidances
Complexity of the process and NPM Guidances.
-------
APPENDIX: SUMMARY OF RESPONSES
This section provides the assessment data and analysis, organized by each major
component of the two year NPM Guidance process.
At the beginning of each component is a bar chart and two tables showing the
responses to the sealer question. Responses are analyzed according to
participants' organization type (state, EPA Region, and EPA HQ). This is followed
by participants' written responses about (1) what aspects worked well and should
be continued, and (2) what did not work well and should be changed and how to
improve it. These also are organized by participants' organization type.
-------
1. Early Engagement: Process
Overall, how well do you feel the early engagement
process worked?
It was terrible
It was the best
thing since sliced
bread!
IHQ Region State
Total
Number of Responses
It was
terrible
1
HQ
Region
State
Total
2
1
1
2
3
2
4
6
4
6
6
5
17
It was the
best thing
since
sliced
bread!
5
1
1
Total
10
11
5
26
Average
3.7
3.5
4.0
3.7
Percent of Responses within Organization Type
1234
HQ
Region
State
10.0%
9.1%
0.0%
20.0%
36.4%
0.0%
60.0%
54.5%
100.0%
10%
0%
0%
7.7%
23.1%
65.4%
4%
10
-------
What aspects of the early engagement process worked
well and should be continued?
STATES
Continue to keep states apprised early on of significant changes in work plan priorities.
EPA was very receptive to suggestions for improvements. The final product was a good start in
improving communications streamlining the process.
Opportunities given to states to provide feedback. Early indications of major items within
guidance.
The process was good and I believe should be retained for the FY 2018-2019 guidance cycle.
Appreciated the timelines being posted & EPA's effort to discuss with states & tribes early in
the process.
EPA REGIONS
Outreach process for early involvement was good.
The early communication of engagement process and timelines allowed for the possibility of
actually conferring with state and tribal partners (versus scrambling to get a number from
them to just react to short planning deadlines).
As I understand it, OW and its Regions have always used slightly earlier engagement than
other programs. Thus, I didn't really see a real impact from the Regional perspective.
All elements and aspects of the process should be continued, particularly the multiyear
aspects.
I believe Headquarters (AIEO) was more involved in the process than the regions. It was not
clear what our (regions, at least my office) roles were.
The earlier the better for all participants. Gives people time to plan.
Starting discussions early with the states and tribes on the activities within the NPM guidance.
Approach: NPMs posting schedule and planned approach.
EPAHQ
The early engagement itself - the requirement to speak with the states/tribes early before the
draft NPM guidance is developed. While extra work, it provided a neutral framework for
dialogue.
Collaborative engagement, OCFO tribal outreach and summaries.
Early and continual discussions with NTC and ECOS.
The process appeared more efficient than in years past. I was able to provide my edits or
recommended changes only once. In years past, I would have to review multiple iterations to
ensure our edits were incorporated, which took a considerable amount of time.
Having needed data on time.
OECA sent requests for early input via email and asked for feedback by early August; this
11
-------
seemed to be effective. Responding by email is efficient and asking for the input by August
allows EPA plenty of time to consider the early input up-front prior to developing the draft
NPM Guidance. In addition, OECA identified the questions on which we wanted feedback.
(NPMs should continue to have the option to do so; it should not be mandatory to do so.)
Commenters can still share whatever input they would like even if it goes beyond the
questions posed. Lastly, because OECA provided responses to the early input at the same time
that the draft NPM Guidance was posted for public review this allowed states, tribes and
associations who submitted early input to understand how their comments were considered,
as well as EPA's position on the comment, at the same time that they were reviewing the
external draft of the NPM Guidance.
It was good to have individual meetings with states and tribes specifically for discussing their
input in the next NPM Guidance.
OCFO and OCIR established NPM Guidance/NEPPS workgroup and SharePoint website.
It was useful to develop a framework for likely opportunities for early engagement. In
consultation with NPMs, suggest continuing something like this though keeping it flexible for
NPMs to tailor to their own needs.
Structured format. Required, early formalized outreach (vs each Goal doing it whenever).
What didn't work and should be changed? What are your
recommendations to improve the early engagement
process?
STATES
There will continue to be challenges linking to multiyear grants (PPGS)
EPA REGIONS
Follow up to the outreach and feedback was not transparent or not done well - at least for
regional staff. States did ask the region what came out of the outreach sessions so it appeared
they did not get feedback on that either.
The communications seem broad, vague, and attachments hard to navigate. The action items,
the "ask" is often unclear. The attachments, spreadsheets of commitments, contain so much
information they are confusing.
Some have commented that it would nice if our internal comment period could be a little
longer. Especially those who were used to the previous process' longer comment period.
Communications about the process and the practical aspects of subsequent implementation
were uneven across the NPMs. By extensions, there were gaps and misinformation among
Regional programs
Clear communications from AIEO. Establish clear communication about the roles and
responsibilities about the process.
The Regions never have enough time to review the guidance.
Not having all the information available within the NPM guidance to provide to the states and
tribes (especially the grant information). Recommendation is to have at a minimum a
summary of the information that will be provided in all areas within the NPM guidance so
discussions can be had with the states and tribes.
12
-------
More advanced notice of NPM early engagement with states and tribes.
Consider using SharePoint and allowing regional planners and others to see feedback as it is
developed throughout the process.
Summaries of Engagement: disconnect between how state and tribal feedback was considered
by NPMs; and actually used to revise next version of the NPM Guidance documents. Is it
possible to have an EPA senior leadership discussion on this after it has been compiled? Or
some other check point with Regions, States and Tribes to show how it was considered and
might be used to revise some of the program priorities or measures in the next NPM Guidance
docs?"
EPAHQ
There was not as much "discussion" - it was viewed more as a report out - not sure what it
changed in the final draft. It might have been because it was the first time and this may take
time to take root, so I don't think this is the reason to give up.
Posting the approach timeline was fairly prescriptive but did not seem to increase involvement
in our outreach. Not averse to providing this in the future if others found it beneficial.
OITA is uniquely situated in that we continue to engage the NTC on regular and continual
basis. Many of the concepts of the 2-year Guidance has been implemented and is an integral
part of our relationship
Better Regional/HQ coordination
OECA has no recommendations for changes at this time.
The early engagement may have been a little too early for specific input.
Early on, EPA should expand and reach out to groups which EPA has not traditionally worked
with. Communicate and partner with stakeholders, including industry, non-governmental
organizations, academia, the research community, international organizations, local agencies,
agricultural and manufacturing sectors, small businesses, communities with environmental
justice concerns, citizens, and other partners, both urban and rural, including those who have
been underrepresented, to achieve more innovative and sustainable outcomes. In keeping
with our objective to strengthen partnerships, EPA will emphasize transparency and clarity in
its communications, including environmental education outreach. Through collaboration and
research, we will improve our ability to drive innovation and expand the conversation on
environmentalism to address related social and economic issues, especially in communities
with vulnerable populations or environmental justice concerns.
I think we have a solid approach in place and should continue in a similar manner as we did for
FY 16/17.
13
-------
2. Early Engagement: Substance
Overall, how well do you feel the early engagement
process informed national areas of focus, program-specific
guidance, and areas for improved cross- program
coordination?
14
12
10
Not successful
Extremely
successful
IHQ Region State
Total
Number of Responses
HQ
Region
State
Total
Not at all
informative
1
2
1
1
2
3
3
5
4
12
4
4
4
1
9
Extremely
informative
5
2
1
3
Total
10
11
5
26
Average
3.7
3.5
3.2
3.5
Percent of Responses within Organization Type
1234
HQ
Region
State
10.0%
9.1%
0.0%
30.0%
45.5%
80.0%
40.0%
36.4%
20.0%
20%
9%
0%
7.7%
46.2%
34.6%
12%
14
-------
2A. National Areas of Focus and Program-Specific Guidance
What worked well in early engagement with states and
tribes to identify the most important areas of environmental
and human health work?
STATES
I am not aware of specific changes as a result of early engagement with states/tribes.
The engagement and focus at the HQs level informed national areas of focus.
This seemed to work quite well, although the areas of work did not change enough that it
seemed to generate a lot of true engagement.
Appreciate EPA's effort to view cross-program areas. Also appreciate the program-specific
calls.
EPA REGIONS
Engagement with state associations was good - the media associations are often left out.
Talking to these groups should have better informed EPA on state priorities on a broader
basis.
New efforts to eliminate obsolete measures and focus on outcomes have been helpful in
dealing with states.
From the ACS/NPM Guidance perspective, we always encourage our POCs to share pertinent
information (e.g. Public Comment Period Info) with our State counterparts. The state POCs
seem to be happy to hear from them (even if they get the same info more than once).
The organization (straightforward, not complex) and communication of potential topics for
national areas of priority and focus (again -these seemed clearly laid out, not cluttered or
obfuscated by excessive verbiage or by multiple concurrent processes).
The process was proactive and anticipated the need for collaboration/coordination across
various timeline's where there had been both substantive and process conflicts.
Once we received information, we were able to forward information to the tribes.
The discussion went well with the information available.
While I don't participate in the engagement process, I know the region finds it very helpful to
meet with states early and discuss investments and disinvestment areas.
EPAHQ
The concept of engagement was the correct first step. The structure/report back mechanism
was useful to ensure it occurred.
Collaborative engagement. Using existing venues to seek early input on priorities and
summarizing the results.
The early engagement resulted in fewer questions when the draft went out for public
comment. So the upfront work paid off.
Starting the conversation earlier gave us more time to identify issues.
OECA provided standard letters/emails to all 10 regions and asked the regions to use the
15
-------
standard request to solicit early input from the states and tribes in their regions; this had a
two-fold benefit. First, the states and tribes received essentially the same request for
information from all 10 regions. Secondly, this approach involved the regions, as partners, up-
front in soliciting early input for OECA's NPM Guidance.
OEI doesn't have much engagement with states/tribes.
OCFO and OCIR established NPM Guidance/NEPPS workgroup, and SharePoint website. EPA
worked effectively by collaborating early with ECOS for states and AIEO for tribes.
Providing the NPMs the flexibility to engage with states in ways that are most meaningful to
them.
More time for collaboration and consideration.
What did not work and should be changed? What are your
recommendations for improving how early engagement
informs national areas of focus and program-specific
guidance?
STATES
Not sure of how much the national conversation translates to actual Regional expectations
and commitment negotiations.
I think continuing to emphasize changes that will affect the work that states need to commit
to and do that are different from previous years is key. Also, continuing to keep descriptions
as short and concise as possible is helpful.
I understand the national areas of focus, but I wonder if more could be done with regional
areas of focus.
EPA REGIONS
Not familiar enough with how the process went since regions were not inform at least at the
NEPPS coordination level although our individual programs may have been involved in their
respective forums more, so cannot comment.
Early engagement helps determine programmatic shifts and/or potential areas of concern
(e.g. Regions missing specific targets across the agency). This increases the potential for
collaborative work and innovation.
The "Collaborative Engagement with State and Tribes...to identify the most important areas of
focus for protecting the environment and human health, including topics/priorities of interest
that involve more than one NPM" remains less than integrated in routine business operations
in that the engagement sometimes occurs ceremonially in staged events. While these are of
course important events, more can be done to weave engagement into routines throughout
the cycle so that joint planning and evaluation is ongoing at all levels of state/EPA partnership
work.
Communication about the process.
Some discussions had to be postponed because not enough information was available in the
NPM guidance.
"Suggest more communication (two-way dialogue/feedback) on how NPMs actually used the
16
-------
early engagement feedback and what changed, if anything, in next NPM Guidance docs.
Summaries of Engagement: disconnect between how state and tribal feedback was considered
by NPMs; and actually used to revise next version of the NPM Guidance documents. Is it
possible to have an EPA senior leadership discussion on this after it has been compiled? Or
some other check point with Regions, States and Tribes to show how it was considered and
might be used to revise some of the program priorities or measures in the next NPM Guidance
docs?"
EPAHQ
Since we do have conversations with the states more regularly, building in the timing of this
opportunity and education on what the NPM guidance is used for may be useful. I don't
believe many think the NPM guidance does much for setting the policy objectives.
Framing our approach and posting it did not appear to increase participation.
Some staff not ready yet.
OEI doesn't have much engagement with states/tribes.
Technical Guidance should describe the history of sustainability from 1970 to present day, and
underscore why achieving sustainability is urgent. The Technical Guidance should advance
understanding of sustainability and EPA's role/mandate in achieving sustainability. As the NPM
Guidance is developed, NPMs and Regions should understand the major decision making tools
and approaches that are available and those that are needed to enhance sustainable
outcomes as we preserve the environment for the next generation. To integrate sustainability
into the Agency's day-to-day operations, all headquarters and regional offices will routinely
consider the following principles in their decisions and actions, as appropriate:
1. Conserve, protect, restore, and improve the supply and quality of natural resources
and environmental media (energy, water, materials, ecosystems, land, and air) over
the long term;
2. Align and integrate programs, tools, incentives, and indicators to achieve as many
positive outcomes as possible in environmental, economic, and social systems; and,
3. Consider the full life cycles of multiple natural resources, processes, and pollutants in
order to prevent pollution, reduce waste, and create a sustainable future.
Largely speaking, the national areas of focus are fairly well set going into the guidance
development process. Where the engagement is important is to provide an opportunity for
states to weigh in on state specific or region specific priorities for consideration.
2B. Cross-Program Coordination
What aspects of early engagement mechanisms with
states and tribes worked well for identifying topics/priorities
of interest that involve more than one NPM?
STATES
Keep communicating with multiple agencies to facilitate cross-program coordination.
This is important so that regions and states are not getting confusing or conflicting messages
about priorities that involve more than one NPM.
17
-------
EPA REGIONS
The organization and communication of topics and/or priorities was straightforward and
simply stated.
There was higher level attention to topics where multi media/multi programs could benefit
from collaboration.
Having the NPM guidance and the Strategic Goals available were valuable in discussions.
I polled regional staff who work on programmatic workplans that are rolled into the PPG.
They didn't report any significant changes related to early engagement, but I'm not sure if
they're the correct audience for these questions. It seems like their conversations with state
counterparts are ongoing throughout the year and don't really hinge on the NPM guidance
timeline-especially given the unpredictable schedule for changes to funding allocations.
While I don't participate in the engagement process, I know the region finds it very helpful to
meet with states early and discuss investments and disinvestment areas.
EPAHQ
This did not work particularly well. Program offices focused on their programs. Without a
forced effort, it would not be raised.
To my knowledge, cross-program coordination has never been a concern for OLEM. Whenever
cross-program issues are raised, either through early engagement or through the external
comment period, OLEM shares with the appropriate NPMs (and vice versa).
Early and continued discussion and collaboration between NPMs (i.e., One EPA) is key in
identifying key priorities.
There are challenges: some annual, some specific, but both benefit from more time to address
these issues.
OCFO has been helpful serving as a central point for coordinating across-Agency issues.
Coordinated conference calls worked well.
EPA worked effectively by collaborating early with ECOS for states and AIEO for tribes.
NPMs largely ran this process themselves with relevant NPMs to ensure that we were
capturing cross-program issues. And, by including this as a topic area during the engagement
process with states, we were also able to solicit cross-program topics or areas of interest from
states.
Having all NPMs conduct outreach early and within same timeframe allows for easier
coordination and timing of dialogue.
What should be changed? What are your
recommendations for improving how early engagement
with states and tribes can identify opportunities for
cross-program coordination?
STATES
The main thing from my perspective is a need to further integrate the enforcement priorities
into the broader planning process. That office could coordinate better with others, and could
18
-------
work to set enforcement priorities that better align with the two-year guidance cycle.
EPA REGIONS
State and EPA programs are still stove-piped when it comes to program measures.
See above response.
I don't know.
Same as above.
EPA needs staff and management training on "systems" issue identification, problem solving,
communications and coordination. Linear management approaches are counter intuitive
when the systems associated with environmental services delivery and human health are in
fact complex, connected systems. The point is that occasionally EPA and the states do
themselves a disservice by always trying to simplify and streamline processes when there are
highly trained and competent human resources/knowledge management up to the challenges
of joint governance, partnerships and the "s" in NEPPS.
Better communication.
Having HQ offices and Regions coordinate together the discussions for the states and tribes so
that consistent information is received and consolidated.
At the regional level, I could do more outreach during key parts of the process to encourage
more communication. Because we only have 2 PPGs and no active PPAs in R3 currently, I don't
interact with a wide range of regional or state staff on planning issues and may not be the
most reliable source of information.
Consider using a collaboration tool like SharePoint site for greater efficiency and transparency
so that more can view feedback; common themes; identify where have agreement or support
for these common themes.
Suggest more communication (two-way dialogue/feedback) on how NPMs actually used the
early engagement feedback and what changed, if anything, in next NPM Guidance docs.
Summaries of Engagement: disconnect between how state and tribal feedback was considered
by NPMs and actually used to revise next version of the NPM Guidance documents. Is it
possible to have an EPA senior leadership discussion on this after it has been compiled? Or
some other check point with Regions, States and Tribes to show how it was considered and
might be used to revise some of the program priorities or measures in the next NPM Guidance
docs?
EPAHQ
Use ECOS as the forum. There are not many opportunities for a cross-program section to be
together.
The process outlined for the FY 2016-2017 was unobtrusive. If this provides some level of
assurance to other programs or to our partners, we are OK with retaining it.
More internal information about what we are trying to accomplish.
Potential cross-program topics should be identified ahead of time and assigned an office lead,
i.e., E-Enterprise-who should engage the states and tribes on E-Enterprise and ensure that the
input is deposited in the right NPM Guidance?
This section is basically a place-holder, since it refers to the NPM program-specific section
19
-------
above. Cross program coordination is used interchangeably with cross-agency strategies, and
implied multi-media programs. If an NPM or Region do not consider their program to fit in the
cross-program approach, they may tend to skip this section. Traditional approaches to risk
reduction and pollution control can only go so far to deliver the long-term and broad
environmental quality we seek. The interplay between different environmental statutes and
programs also requires renewed attention to improve "synergy" and long-term solutions. The
NPM Guidance should make it program-specific to consider sustainability.
20
-------
3. Incorporating Flexibility
To what extent was the NPM Guidance process successful
in providing support for EPA, states, and tribes to pursue
flexibilities that advance the most important environmental
and human health work?
Not successful
Extremely
successful
IHQ Region State
Number of Responses
HQ
Region
State
Not
successful
1
2
1
2
1
3
6
4
4
3
2
4
Extremely
successful
5
Total
10
8
5
Average
3.2
3.0
3.6
Total 4 10
Percent of Responses within Organization Type
1 234
23
3.2
HQ
Region
State
10.0%
25.0%
20.0%
60.0%
50.0%
0.0%
30.0%
25.0%
80.0%
Total
17.4%
43.5%
39.1%
21
-------
What aspects of the NPM Guidance were most successful
in articulating support for flexibility (e.g., expressions of
support, parameters for flexibility, how to elevate issues)?
STATES
Continue to articulate support for flexibility at the regional, tribal and state level.
Having guidance that supports flexibility is huge!
I'm personally not aware of any flexibility that resulted from NPM guidance for us. It may be
the case in Region 1.
I think the issue of flexibility is much more clearly articulated now than in the past. While I am
not aware of a large number of instances that it was utilized, I believe the option to pursue it
is now reasonably clear.
All references to support flexibility are appreciated. Having this document in the NPM
guidances sends a positive message to the regions, states, and tribes.
EPA REGIONS
I felt the effort did help programs and NPM staff understand the need for flexibility, but now
sure the process for realizing the program flexibility has been changed much so far or if so that
is not being communicated with regional programs.
Streamlining and formatting the guidances in general helped to make them much more "user
friendly" and easier to read through. This in turned helped them better articulate things like
support for flexibility and elevating issues.
The pre-budget target-setting, commitments, and mid-year adjustments seemed to carry a
message of flexibility (with justification) that wasn't as evident in years past.
Issue elevation and the promise of authentic multi-year work plans with increased focus on
results and outcomes.
This is first I'm hearing about "support for flexibility."
If the NPM guidance was finalized by March 30, it would facilitate more discussions during the
first quarter of the fiscal year and the Final NPM guidance could be available by the end of the
second quarter.
Introductory language.
EPAHQ
This assumes the NPM guidance is used by Regions in their decisions. There should be a more
holistic look at the value and purpose of the NPM guidance. It is developed as a cursory
activity and most regional folks don't actually read or pay attention to it. The state/tribal
engagement actually served to call it into focus more than usual, which provided value.
The process adopted for the FY 2016-2017 to affirm flexibility and identify parameters.
For the very narrow role that I play, I'm not sure that I fully understand this question.
Flexibility in what?
Some staff was more prepared and more informed.
In the NPM Guidance, OECA included language on how to seek flexibility. In addition, in
several different media-specific sections of the Guidance, OECA included language to take into
22
-------
account resource constraints and promote further flexibility. The flexibility language resulted
from internal OECA discussions as well as taking into account regional input. This result was
facilitated by the NPM Guidance drafting and review process.
In an era of decreasing budgets, the Technical Guidance emphasized the importance of
leveraging resources. Language in the Introduction affirmed the intent to address flexibility.
Language on parameters for flexibilities was included in sections on National Areas of Focus
and Program Specific Guidance.
OAR appreciated that you left that largely up to NPMs on how best to reflect opportunities for
flexibility as opposed to imposing language and governance practice where none was needed.
Addendum process allows for some adjustment
What was unsuccessful and should be changed? What are
your recommendations for improving how the NPM
Guidances support flexibilities for EPA, states, and tribes
that advance the most important environmental and human
health work?
STATES
Communication and keeping flexibilities available over time will be challenging.
Again, I think there is somewhat of a disconnect between the high-level AA discussions and
the reality of the trenches negotiations with Regional programs. Perhaps this is exacerbated
because we do not have a PPG.
I am probably biased by having been very involved with the guidance related to state
flexibility, but I have no further suggestions at present other than pursing efforts to document
instances where flexibility has been utilized.
EPA REGIONS
Not sure all NPMs are working with regional programs to encourage (or support?) flexibility
(or maybe some regional staff do not understand what is being communicated to them.
The chronic conflict between the legitimate needs for accountability versus the promotion for
flexibility must be addressed in ways where states and EPA have latitude to negotiate
common sense work plans.
If the NPM guidance was finalized during the second quarter of the fiscal year, it would allow
EPA to facilitate more discussions during work plan and Performance Partnership Agreement
discussions.
Still a bit of a disconnect and lack of understanding by programs.
Consider creating a compendium (and including in OCFO Technical Guidance) of past examples
of EPA, state or tribal trade-offs in work could help make more programs aware of concrete
examples; and consider these during their ACS and/or grant workplan negotiations.
EPAHQ
Re-education on the purpose of the NPM guidance -this concept is not translated beyond the
NPM/planning universe.
23
-------
OCIR could consider additional outreach to states and tribes concerning flexibilities available
through performance partnerships.
Not everybody was on board.
Incorporate sustainability principles into regulatory, enforcement, incentive-based, and
partnership programs. Use available incentives, education, information, and disclosure to
enhance the ability of markets to reward sustainability.
o Coordinate grants, contracts, and technical assistance to promote sustainable
outcomes;
o Advance sustainability science, indicators, and tools;
o Promote new ways to encourage technology-focused innovation that supports Agency
priorities for sustainability. Use EPA's Technology Innovation Roadmap to guide EPA in
stimulating and supporting technology innovation around key environmental
challenges; and
o Use systems-based approaches that account for linkages between different
environmental systems.
The NPM Guidances are documents. It's the people to people conversations about the work
and tradeoffs that actually support potential opportunities for flexibilities.
Aligning Grant processes within the various program offices (OECA, OW, Lead, etc.). They vary
from 2 year cycles, 3 year cycles, etc.
24
-------
4. Aligning Programmatic Grant Guidances with NPM
Guidance Cycle
To what extent did alignment of grant guidances with the
two-year NPM Guidance cycle reduce the workload
burdens on regions, state and tribes and help streamline
the grant work planning process?
No effect on
workloads/burden
Significant reduction in
workloads/burden
IHQ Region State
Total
Number of Responses
HQ
Region
State
No effect
on
workloads
-burden
1
2
1
1
2
1
2
3
5
3
4
1
2
3
Significant
reduction
in
workloads
-burden
5
1
Total
10
8
4
Average
2.8
2.8
3.3
Total 4386
Percent of Responses within Organization Type
1234
HQ
Region
State
20%
13%
25%
10.0%
25.0%
0.0%
50.0%
37.5%
0.0%
10.0%
25.0%
75.0%
10%
0%
0%
18%
13.6%
36.4%
27.3%
5%
22
2.9
25
-------
What aspects of aligning the grant guidances with the two-
year NPM Guidance cycle worked well and should be
continued?
STATES
Facilitated the application of multi-year grants.
Aligning workplans to 2 years.
Making it two-year and reducing the changes in the mid-cycle year.
I think the work planning currently in process for FY 2017 will best answer this question, as it is
this second year of the two-year cycle that should show the greatest reduction in effort. So, it
is a bit premature to understand the real impact.
Appreciate EPA's efforts to keep changes in the addendum to a minimum. Also appreciate the
ability to make pen & ink changes to workplans.
EPA REGIONS
Some states and tribes are taking the opportunity to provide two-year work plans for multi-
year grants, which significantly reduces workload every other year and this will decrease even
more as regions and states/tribes get more experience with true two-year guidances. So
multi-year guidances and grants and work plans should continue.
This was not apparent to me, so cannot comment.
The roll out provided useful materials.
Our work appear to be the same - did not see any changes.
I didn't see a change.
By only having the second year NPM guidance address what changed from the previous year,
cuts down on the review of the document and allows EPA to facilitate more efficient
discussions with the states and tribes on upcoming changes for work plan commitments.
EPAHQ
Too early to tell. Not sure it will be adopted for our programs.
Concur with thought of aligning grant and NPM guidances, though GAP grant guidance may
not change as frequently as NPM guidance production schedule.
For the Regions, aligning the work, I have heard is key.
Guidance documents are complicated, more time helps.
This is not yet applicable to OECA, but we are committed to work with states to further
explore implications of a 2-year guidance on grant work planning.
Several grants were identified for alignment in the NPM Guidance.
There are still other grants that are off. Some grant guidances run a 3-year cycle.
26
-------
What did not work well and should be changed? What are
your recommendations for how to align the NPM Guidance
and grant work planning process better, to further reduce
the workload burdens on regions, state and tribes and help
streamline the grant work planning process?
STATES
Should look more holistically over the two-year time frame to build more effective work plans
instead of just duplicating first and second year, which seemed to occur at our state level.
Involve the regions more. Provide concrete examples of how Regions have worked with states
to streamline and reduce the workload burden.
Nothing until the impacts on effort in the current work planning process is reviewed.
Hard to fully answer at this time as we are in the process of aligning our work plans (PPG) with
the NPM guidance cycle. I expect this alignment to be a benefit.
EPA REGIONS
Regional programs often are more tied to previous year's work plan than to NPM Guidances.
More emphasis by NPMs on the Guidance may help perhaps even more regional program
involvement in developing the guidances might be beneficial.
Communications varied and consequently not all programs were on the same page with
regard to expectations and latitude to work with counterparts. There was significant cynicism
and some skeptics in the programs.
The NPM guidance is finalized too late in the fiscal year. It would be better if the NPM
guidance was finalized during the second quarter, so that discussions would occur earlier
regarding any changes to work plan commitments. This would also reduce the work load
during the end of the 3d and during the 4th quarter of the fiscal year, when everyone is the
busiest.
I can only speak to changes for the PPGs that I manage. For both of the PPGs, we are at
midpoint and the rollout for this process didn't match up with the start of a new planning
cycle. So, we won't really know anything until we start working on our new PPG workplans for
Maryland in the fall-you may want to check back on progress in another year or two. I doubt
that I'm the only person who will have a bit of a delay.
Ask all NPMs to create a summary of key changes for states and tribes from previous version.
This helps regions and states save time on deciding what changes, if any, should a state make
to its next grant workplan. Our recent Lean event with Wisconsin amplified this as a value-
added product that NPMs or OCFO can create as soon as possible and prior to Guidance docs
being final.
o Consistent timing of NPM grant program allocations being shared with regions and
states. If most or all grant allocations were shared in late Winter/early spring, then
we'd gain greater administrative efficiencies by states being able to prepare a single
workplan for a PPG. When final NPM grant allocations are shared throughout a year,
regions and states have to create separate workplans and make separate awards;
which is counter to the burden reduction that PPGs are supposed to provide."
27
-------
EPAHQ
Has there been a review of whether this is actually taking place?
We need to use more Plain English and better inform everybody in the process.
Incorporate green grants policy. Collaborate with Green Grants workgroup. Require that
Project Officers talk to grantee about how they can integrate sustainability into grants (e.g.,
green meetings, EPP, track participation). Make green grants policy a national policy. Put into
RFPs. Have national data collection systems.
28
-------
5. Earlier and More Meaningful Engagement with States and
Tribes on Commitment-Setting
How effective was early engagement with states and tribes
on commitment- setting in facilitating state grant work
planning?
12
10
2
0
Ineffective
Extremely
effective
IHQ Region State
Total
Number of Responses
HQ
Region
State
Ineffective
1
1
1
2
1
1
3
4
4
2
4
2
5
2
Extremely
effective
5
Total
8
11
4
Average
2.9
3.2
3.5
Total 2 2 10 9
Percent of Responses within Organization Type
1234
9%
8.7%
43.5%
39.1%
23
3.1
HQ
Region
State
13%
9%
0%
12.5%
9.1%
0.0%
50.0%
36.4%
50.0%
25.0%
45.5%
50.0%
29
-------
What aspects of early engagement on commitment-setting
worked well and should be continued?
STATES
Conference Calls.
Communication has improved.
I think that the most significant impact was an improved understanding by states of what
commitments were expected. I believe that shortening the NPM guidance documents, and
focusing more on what is new or different has the largest impact.
EPA REGIONS
Not directly involved with this process for 6 of 8 states, so limited experience to base
comments on. For the other two, I observe longer term goals and priorities are driving forces
rather than NPM Guidances (as I said more focus on total strategic plan goals divided by 5
(years) than on new priorities that might be in guidances (like E-Enterprise initiative).
The early engagement has helped reduce the antagonism of short deadline driven
commitment setting.
Simply, the advance notice and more generous time frame for conferring with states and
tribes.
OW and at least the Region 5 Water Division have always engaged the states early on in
commitment setting. No big impact seen by this process being instituted across the board.
The timeline worked well.
There are no discussions of commitment-setting at the regional level with tribes (at least not
in R5).
I didn't see a change.
We were able to start discussions based on the draft NPM guidance activities, however, not all
information regarding grants and monitoring was available during the draft comment period.
I think it's too early to tell this year.
EPAHQ
This does not play a large role in our NPM guidance - maybe it should, but it currently does
not. Which gets to the purpose of the NPM guidance as mentioned earlier.
The commitment-setting process still needs a significant amount of work. Often there is a
disconnect between what is sent to the Regions and what the HQ office knows. For FY 2017,
we were excited and pleased to see that OW is moving away from "spring targets." I would
rather have commitments due once (fall) because even June is still too early.
More information available.
Collaboration with ECOS for states and AIEO for tribes.
I'm not sure what this question is asking. The earlier engagement likely did not have an
impact on what the Air Program was already doing.
Early guidance and initiating agency-wide outreach to formalize the process
30
-------
What did not work and should be changed? What are your
recommendations for improving early engagement with
states and tribes on commitment setting?
STATES
Still need some work tying the commitment system together with states.
My main suggestion is to continue the process, and look for opportunities to further shorten
the actual guidance documents. In the end, there is a lot in the NPM documents that is
directed to regions more than to states, so the parts that states really focus on or engage with
is somewhat limited.
This will be easier when our state planning cycle is synchronized with EPA's NPM guidance
cycle.
EPA REGIONS
Not enough involvement in early engagement and planning efforts to comment.
There was not enough emphasis on outcomes and results so some of the commitments
became merely the arithmetic exercise of taking 12 month commitments times two.
Not having all information about the programs (especially grants and monitoring) in the draft
NPM guidance so that discussions could occur with the states and tribes. Most, if not all,
information should be provided in the draft NPM guidance to facilitate more discussions with
the states and tribes.
Summaries of key changes for states is a valuable tool that allows for easier revisions to
workplans; and thus, helps expedite negotiations for next year's commitments.
EPAHQ
Still lack of easier and friendlier use of this information.
OECA's ACS measures mostly require commitments from our regions. A few measures require
commitments from states but for the most part, those numbers are negotiated between the
region and the state.
In addition to states and tribes, EPA should expand and reach out to other groups which EPA
has not traditionally worked with early on. Because these groups were not invited to
participate in the priority setting, they may not have been as engaged in the commitment-
setting.
Communicate and partner with stakeholders, including small businesses, industry, non-
governmental organizations, academia, the research community, communities with
environmental justice concerns, international organizations, local agencies, agricultural and
manufacturing sectors, citizens, and other partners, both urban and rural, including those who
have been underrepresented, to achieve more innovative and sustainable outcomes.
31
-------
6. Reviews
Overall, how effective were the internal and external
reviews for eliciting meaningful input on draft NPM
Guidances?
Ineffective
4 5
Extremely
effective
IHQ Region State
Number of Responses
HQ
Region
State
Ineffective
1
1
2
2
3
4
4
2
4
3
3
3
Extremely
effective
5
2
1
Total
11
9
5
Average
3.5
3.3
3.6
Total 1 2 10
Percent of Responses within Organization Type
HQ
Region
State
Total
1
0%
11%
0%
4%
2
18.2%
0.0%
0.0%
8.0%
3
36.4%
44.4%
40.0%
40.0%
4
27.3%
33.3%
60.0%
36.0%
5
18.2%
11.1%
0.0%
12.0%
25
3.4
32
-------
What aspects of internal and external reviews worked well
for eliciting meaningful input and should be continued?
STATES
Continue as conducted.
I think the reviews were effective at the high-levels of the states and EPA. We struggle
internally with getting divisions truly engaged in the reviews.
I think the most effective aspect was the preparing the summary of key changes for each
NPM, and being clear what is new or different for states to focus on. This certainly should
continue.
I appreciate the opportunity to engage during the review period. I'm not sure how best to
elicit meaningful input, but appreciate EPA's effort in this area.
EPA REGIONS
Not enough involvement to comment.
The internal review is a good way for Regions to provide input to NPMs on areas/items to
address prior to the external comment period. The external comment period is a good time
for states to weigh in on areas that might affect how they do business (positively/negatively)
prior to setting commitments.
I don't know how much meaningful input was garnered from other than HQ NPM offices? In
general, it seems that NPM guidance is still a largely HQ NPM influenced product. Perhaps by
the nature of it - being NPM Guidance, it will remain so.
To the extent internal and external partners review and provided comments, I think the
reviews are meaningful. While comments don't always result in change, it at least allows for
input that may trigger discussion and, in some cases, affect the final guidance.
There appeared to be more horse power behind the effort this time...that should continue
insofar as both state and EPA leadership should offer consistent support around this work as
opposed to a one-off declaration.
Once the NPM guidances are developed, we provide it to the tribes for comments. We never
know if any comments are provided. The process has not worked well for the tribes, as they
are trying to affect the system/process of budgeting and funding without knowledge of the
entire process. I believe most often they miss the mark on when and where to affect the
process.
Having the NPM guidance available using various media (Share Point, internet, intranet).
Giving 3 to 4 weeks for each review period seems adequate.
EPAHQ
OLEM welcomes both internal and external input on our guidance. From my experience, this
only improves our guidance.
Sharing early drafts were helpful from OITA perspective
I definitely agree with the external review; however, it seems odd that we have a round of
33
-------
internal review. Seems like we should all be on the same page "one EPA".
Creates incentives to prioritize issues and programs
The regional review and public comment process have provided meaningful input.
Internal review was more meaningful for OEI as our NPM Guidance serves primarily as a
communication vehicle to the rest of the Agency, rather than to states and tribes.
OCFO and OCIR established NPM Guidance / NEPPS workgroup, reached out to ECOS for
states and AIEO for tribes.
This is a difficult and broad question to answer as I did not have another process to compare
those reviews- with that said, I think the process needs improvement.
OAR has built in processes for the Air Program prior to releasing the guidance for internal
agency review. Generally speaking, that extra review step is a double check for us. Typically in
the external review process, we receive policy feedback.
What did not work and should be changed? What are your
recommendations for improving internal and external
reviews?
STATES
Considering further streamlining of this process in the mid-cycle addendums to the two-year
guidance.
EPA REGIONS
Would be nice to have a longer internal comment period (especially for two-year guidance
versus addendums) and to see NPM's do a response to comments (or high level summary) so
Regions know what things were considered/changed.
Too much variability in program enthusiasm for, and knowledge about the process
Keep posting the NPM guidance on various sites to facilitate comments.
EPAHQ
I do not find this particularly useful as I don't know how many external partners actual review
the guidance, nor do I believe much is done to change the draft when comments are received.
Not all staff and management understood the main objective of review process.
The conference calls with ECOS and tribes have not generated any input or feedback. Written
comments did provide meaningful input.
When reviewing Addendums, we received comments on the initial two-year Guidance (long
after it was final). There should be better clarification on what is up for review during this
process.
Many employees do not participate on the NPM Guidance/NEPPS workgroup, so the NPM
Guidance may not have been reviewed by everyone. Technical Guidance could provide
reference to webinars on sustainability. Educate employees. Help employees to understand
that sustainability is all of our responsibility. Help employees to see practical ways that
sustainability can be incorporated into principles and programs. On LAN banner, post request
for employee review of NPM Guidance. Externally, invite participation and review from other
34
-------
federal agencies with sustainability experts such as HUD, DOT, HHS.
[Three recommendations:}
o Recommendation 1: Allow the NPM to be the point of contact or provide a common
area in which the NPM can retrieve the comments; last year we received external and
internal comments that were lost in the shuffle a couple of days after the deadline.
o Recommendation 2: Advertise the external review process and define a limit to the
number of comments an organization can provide. I was expecting a more diverse
pool of participants during the external review process. Instead, we received plenty of
comments from 6 organizations.
o Recommendation 3: Provide an improved tool for stakeholders to give their
comments; a word document is difficult to manage. Some organizations provided
letters, which OW had to move to the response to comments template.
35
-------
7. Collaboration/Communication Tools
Overall, how effective were the collaboration tools in
facilitating work, communication and information sharing
(e.g., NPM Guidance website, SharePoint, OCFO
guidances)?
Ineffective
Extremely
effective
IHQ Region State
Total
Total
Number of Responses
HQ
Region
State
Ineffective
1
2
1
1
3
3
4
1
4
6
4
2
Extremely
effective
5
1
1
1
Total
11
10
4
Average
3.6
3.5
4.0
8
12
Percent of Responses within Organization Type
1234
HQ
Region
State
9.1%
10.0%
0.0%
27.3%
40.0%
25.0%
54.5%
40.0%
50.0%
9.1%
10.0%
25.0%
8.0%
32.0%
48.0%
12.0%
25
3.6
36
-------
Which collaboration/communication tools worked well for
facilitating work and communication and should be
continued?
STATES
Website for easy access to all the information throughout the process.
SharePoint.
I'm not familiar with the SharePoint effort. I think the Executive Summaries of most critical
content were effective.
This answer probably varies depending on what role one plays in the process. As a state, the
NPM Guidance Website and the conference calls were probably the most helpful. That said, I
think all of these tools are needed to maintain consistency in approach among the NPMs, and
in the messaging to states.
I generally use the NPM guidance website.
EPA REGIONS
SharePoint was especially valuable and will increase as states and regions become more
proficient in using it. Webinars/video conferencing are excellent tools for engaging more
states and tribes.
OneDrive has great potential for integrating input but many are still not comfortable with the
applications.
The technical guidances and memorandum are always helpful/clearly outline process. The
SharePoint site was also helpful.
SharePoint worked well with my program and HQ NPM counterpart (OW/OWOW).
NPM Guidance website and SharePoint effective from the perspective that email is not
inundated with what is sometimes large (or multiple) attachments.
Any of the tools associated with promoting coordination around multi media/multi program
issues as a starting point for joint planning.
Possibly not very effective since it remains the same.
Share Point, intranet, internet, email.
I think the region uses SharePoint and I believe it is effective but I don't know for certain.
NPM Guidance website is good for referencing draft and final docs.
OCFO guidance: include an appendix of "program trade-offs" that have been approved in the
past -as a way to help more regions/states understand and decide if they want to pursue
flexibility.
EPAHQ
NPM Guidance website, agency overview, OCFO guidance.
I appreciate the SharePoint site that our measures contact has created. A more streamlined
and efficient way to provide comments. The NPM guidance website is not really a good source
of information, rather something that I refer to, but not really others.
Staff and management that learned how to use these tools helped the process.
37
-------
The NPM Guidance website and OCFO conference calls with NPMs have been useful tools for
communication. OCFO can use these calls to float new ideas and NPMs can provide timely
feedback before new ideas are finalized.
OCFO Guidance online with hyperlinks included and SharePoint.
OCFO and OCIR worked with NPM Guidance and NEPPS workgroup.
I learned about your Sharepoint site, days after we published our NWPG. I am sure it would
have been helpful.
Use of SharePoint.
Which collaboration/communication tools did not work and
should be changed? What are your recommendations for
improving collaboration/communication tools?
STATES
The mid-cycle conference calls: I'm not sure a conference call is necessary mid-cycle, especially
if the NPM complied with the requirements of not making many significant changes.
For the addendum year, I think the process could be streamlined further.
EPA REGIONS
All worked and I don't have expertise or knowledge to recommend others.
Would like to see the NPM Guidance website updated (and reorganized) so it is more "user
friendly." Some of the materials are out of date (ACS User Manuals) and it is sometimes hard
to find NPM Guidances from previous years.
The default to the silo culture or the bunker mentality of preventing substantive
disinvestments.
If you had a way for people to provide feedback to the NPM guidance similar to this survey,
then the information would be efficiently submitted for review.
Use of SharePoint for helping communicate early engagement feedback; and possibly being
used to show how it was considered, used or not used.
EPAHQ
Staff and management that do not understand the basic concept of how and why we should
use these tools hurt the process.
The SharePoint site was often difficult to use because it contained too many documents/
guidance materials. Also, many of the documents were found to be blank when
opened/selected. If OCFO plans to continue using the site, OECA recommends the better
management/oversight of the documents that are posted.
Many employees do not participate on the NPM Guidance/NEPPS workgroup, but
sustainability is all of our responsibility. Internally, EPA can promote internal short videos, LAN
banners, posting webinars on SharePoint for background to help employees to understand
practical ways to incorporate sustainability into day-to-day work. Review new and key existing
regulations to examine sustainable enhancements. Integrate efforts with a new commitment
to innovation and greater and more strategic ("high level") use of sustainability-related data
38
-------
and information. Work within and across programs, use all available tools, and implement
innovative approaches. Build on our wide range of existing sustainability-related activities,
including community-based sustainability activities. Use incentive-based efforts to
complement our foundation of regulations. Encourage technology-based innovation through
challenges and partnerships. Externally, we can use more social media.
The OCFO NPM guidance was too broad, which is understandable when the audience is all
NPMs. Nonetheless, I found myself calling your coordinator often. Thanks for the support!
With that said, it would be helpful to have a training for newly hired folks working on an NPM
guidance.
39
-------
8. Other-
Are there any other comments or suggestions you would
like to share on how to improve the NPM Guidance
Process?
STATES
Providing more concrete examples from the Regions on how they have used the guidance to
reduce the workload burden or provide true flexibility in an actual grant commitments
negotiation.
I appreciate EPA's engagement on making improvements to this overall process. Significant
progress has been made, and likely can be improved further.
EPA REGIONS
Continue to seek ways to keep the complexity of the process and the complexity of guidance
itself to a moderate threshold; keep it easy to read and understand.
Communication, Collaboration and Coordination with EPA regions and tribes.
Have the NPM guidance review process electronic (similar to this survey) so that people within
EPA and outside of EPA can respond quickly and efficiently. Also, have a way to respond to the
comments electronically. If you ask questions and then have a comment area for each section
of the NPM guidance that would allow people to provide comments quickly and efficiently.
Process seems to have been improved with early engagement forums. Still opportunities for
continuous improvement in areas like:
o Use of SharePoint to show comments; discuss comments and how might be addressed
or used in next NPM Guidance doc.
o Getting subset of "end users" to provide feedback on draft OCFO Technical Guidance
before it is shared with larger audience of end users.
o Having some type of "multi-partner" corporate discussion after all regional, state,
tribal feedback has been shared and considered; to explain why certain ideas were not
used; and why certain priorities are being pursued over others.
o Getting NPM grant allocations to be calculated and final allocations shared around the
same time frame and before start of EPA/State grant work-planning cycle (e.g., early
spring). Could further reduce administrative burdens and meet the true vision of using
a PPG.
EPAHQ
Kudos to transitioning to a two-year process. I would encourage folks to look at the entire
process and figure out which items return the most value to the end user. There are still some
pretty significant ways that we can become more efficient (commitment setting process).
Also, ACS is such an old and antiquated system that it is very difficult to work with. Would
welcome a new tool to support this piece of our work.
The information and the communication has to be expanded, instead of being a "privileged"
information. The more people know about it, the better use it will have.
40
-------
All EPA offices should avoid making any last minute requests to other offices with regard to
including new information in their NPM Guidance. Also, OCFO should continue to check-in
with NPMs as new issues or ideas arise. OECA's OCFO desk officer has been helpful by
checking-in at key points during the process. Also, preliminary reviews for 508 compliance by
OCFO would be very helpful in addressing issues/errors well before final versions are due.
Put sustainability resources (FTE) into megatrend-focused efforts such as resiliency efforts
instead of program-to-program integration to work better toward getting ahead of the trends
or better preparing for them.
Change time scale to more long-term when conducting project planning. Look ahead at least
ten years or 30-40 years when conducting strategic planning. Build on staff knowledge of and
experience with sustainability and innovation through multiple forms of in-reach, education,
and guidance for incorporating sustainability principles into Agency work in a multi-
disciplinary way. Develop clear Agency leadership expectations for training at all levels to help
equip employees with necessary data and tools to identify appropriate opportunities, network
internally and externally, establish governance and accountability structures, provide everyday
encouragement and recognition, and lead by example in our own operations. These efforts
will improve the ability of all staff to be effective environmental stewards and to help secure a
healthy, just, and flourishing quality of life for current and future generations. Engage and
empower EPA staff.
41
------- |