r/EPA
United States
Environmental Protection
Agency
Office of
The Administrator
(1601F)
EPA 100-R-99-005A
July 1999
National Advisory
Council for Environmental
Policy and Technology
(NACEPT)
Past & Future
A Decade of Stakeholder Advice
-------
Table of Contents
Executive Summary
Findings
Recommendations .
ES-1
17
Recommendations to the NACEPT Council
Recommendations to Standing Committees
Recommendations to OCEM
Conclusion
28
Appendices
Appendix A:
Appendix B:
Appendix C:
Appendix D:
Appendix E:
Appendix F:
Appendix G:
Appendix H:
Appendix I:
Appendix J:
Appendix K:
Appendix L: *
Appendix M:
NACEPT Standing Committee Acronyms
History and Overview of NACEPT
Organizational Chart of NACEPT
Key Aspects of Charters
Bibliography of NACEPT Publications
Overview of the Study Methodology
Technical Details of the Survey Methodology
Survey Instrument
Background Materials Mailed with Survey
Survey Results
Technical Details of the Interview Methodology
Interview Questions
Chronology and History Report (Under Separate Cover)
-------
Executive Summary
Abstract of Study
The National Advisory Council for Environmental Policy and Technology (NACEPT) was
established in 1988 to provide advice to the U.S. Environmental Protection Agency (EPA) on
issues related to environmental management and policy. NACEPT provides a forum for public
discussion and the development of independent advice and counsel by taking advantage of the
respective experiences, strengths, knowledge, and responsibilities of a broad range of Agency
constituents and stakeholders. Since 1988, the Council has convened and directed the work of 26
standing committees, each established to address a specific issue. Over the last decade, these
standing committees, comprised of over 700 stakeholder representatives, have addressed Agency
issues related to information management, program activities, and general management and
policy. The Council, in turn, has approved and published over 50 major reports containing over
1,000 recommendations to the EPA Administrator. In recognition of the 10-year anniversary of
NACEPT, the Council undertook a study to evaluate its past performance and to chart a course
for its future by identifying ways to better serve the Agency.
This study found that NACEPT has undoubtably been a success. NACEPT's standing
committees have produced hundreds of timely and relevant recommendations responding to
requests made by the EPA Administrator. Many of these recommendations have influenced or
been directly responsible for subsequent EPA decisions and actions. In addition, the volume and
range of topics addressed by NACEPT has increased during this time, reflecting the value placed
on the Council by EPA's leadership.
This report, entitled NACEPT: Past and Future, presents the results of this study. Other key
findings of the study are:
» NACEPT recommendations have had significant impacts on Agency decision-
making as demonstrated by the creation of new programs such as the U.S.
Environmental Training Institute, the establishment of new Agency offices such
as the Technology Innovation Office, and incorporation into formal Agency
policy such as the EPA's IRM Strategic Plan.
»- NACEPT's recommendations have fulfilled the requests for advice made to
Council by the EPA Administrator.
» Membership on NACEPT is balanced and representative of diverse points of
view. Feedback to standing committees on the impact of their recommendations
and their implementation has been limited.
+ Standing committees have adequate direction, support, and resources to complete
their work although enhancements in communication and facilitation support are
desirable.
NACEPT: Past and Future ES-1
-------
Based on these findings, NACEPT has undertaken several strategic planning initiatives including
the development of a strategic action plan, which identified future and emerging issues relevant
to environmental decision-making. In addition, this report includes recommendations related to
NACEPT's operations, its standing committees, and EPA's Office of Cooperative Environmental
Management, which provides management and administrative support to the Council, These
recommendations range from implementing a structured evaluation of each standing committee
on completion of its work to requesting a formal response from the Agency to all standing
committee reports. The full list of study findings and recommendations is presented in the
following Exhibits ES-l and ES-2.
Exhibit ES-l.
PRINCIPAL FINDINGS
1. NACEPT provides valuable input and advice to the Agency from a wide variety of
stakeholders,
2. Recommendations are timely for Agency decision-making and fulfill standing
committee charters.
3. Standing committee membership is a balanced representation of points of view.
4. Standing committees are given adequate direction to fulfill their missions; early
agreement on purpose and goals may help to improve efficiency.
5. Standing committee recommendations are developed in a timely, inclusive fashion.
6. Most respondents have not received feedback from the Agency on the impact of their
standing committee's recommendations.
7. Communication between standing committee members is adequate but improvements
are needed for communications between meetings and to the Council.
8. Standing committee meetings are generally well-planned and structured yet
improvements can be made in defining the consensus process, ensuring equitable
participation, and keeping decisions on track.
9. Standing committee members were generally positive about receiving timely and
useful background and technical materials to make informed decisions.
10, More frequent meetings and improved communication between meetings could
improve standing committee effectiveness.
11. Better meeting support can be achieved through increased use of technology and
facilitation.
NACEPT: Past and Future * ES-2
-------
Exhibit ES-2.
RECOMMENDATIONS
The NACEPT Council should:
Do more strategic planning to identify the policy issues which NACEPT standing
committees address.
Better publicize itself and its work to all parts of the Agency and beyond.
Streamline the process of developing and delivering recommendations.
Conduct an evaluation of standing committee processes upon the completion of the
standing committee's work.
Take responsibility for maintaining contact with its past members.
NACEPT Standing Committees should:
Prioritize their recommendations and include suggested schedules and performance
targets for implementation of each recommendation.
Request a formal response from the Agency to all standing committee reports at an
appropriate interval.
The Office of Cooperative Environmental Management should:
Ensure productive interaction directly between NACEPT standing committees and
relevant Agency program offices.
Ensure that standing committee work is adequately planned and managed by the
Designated Federal Officer (DFO) and standing committee chairperson to achieve
committee goals in an efficient manner.
Make clear to the program offices the qualifications needed to be a DFO and provide
training to appointed DFOs.
Develop better ways for NACEPT members to communicate between meetings.
Establish an enhanced formal method for the establishment of standing committees.
Improve the NACEPT and standing committee orientation process for new members.
Develop a fprmal facilitation program to ensure proper support for each standing
committee.
NACEPT: Past and Future ES-3
-------
Summary of Report
Overview of NACEPT
In 1988, the U.S. Environmental Protection Agency (EPA) founded the National Advisory
Council for Environmental Policy and Technology (NACEPT) [previously known as the
National Advisory Council for Environmental Technology Transfer (NACETT)], NACEPT was
established to provide an ongoing stakeholder advisory group to recommend ways the Agency
could encourage technology transfer through cooperative activities with industry, academia, and
non-federal government agencies.
In its first decade of operation, NACEPT has involved over 700 stakeholder representatives from
a variety of sectors including business and industry, state, local, and tribal governments, and
academia, who have participated in 26 standing committees in addition to the Council. Each
standing committee is established to address a specific charge within a set timeframe. As such,
the Council serves as a steering committee, reviewing and approving the reports and
recommendations of the standing committees.
The number of NACEPT standing committee investigations has increased over the past decade as
NACEPT has increased the scope of issues on which it provides the Agency advice, as shown in
the Exhibit ES-3 below. These committees, identified in Exhibit ES-4, have held hundreds of
open meetings in accordance with the Federal Advisory Committee Act (FACA), produced over
50 major, published reports, and put forth over 1,000 recommendations related to information
management, program activities, and general Agency policies.
Exhibit ES-3. Number of Standing Committees by Topic Area
12
1111 HI! lift 1111 !t«2 1*»J 11»* 11*5 tilt IliT 1111
| General Agency policy or management
. Proflfam/rntdii ip«eific
KJ Information management/information technology
NACEPT: Past and Future ES-4
-------
Exhibit ES-4. Timeline of NACEPT Standing Committees by Topic Area
ec
Em Education and Training Cnue
[Pollution Prevention EduciUion Cnue |
Stare and Loc^l Program:) Cnue
J
Technolo£\ Innovation and Ecouomjcs Cmle
nternational Env Cmte [Trade and Em Cmte.
IF.FAB
IESEC
IEITC
IEIET
Administrator Thomas
establishes NACEPT in
1988
ICBEP
[Reinvention Cnlena Cnue
[TITLE VI
IECMC
I
tt
a
1
2L
Env. Measures/Chemical Arndent Prevention C'mlt.
Renamed NACEPT in July
1990 lo reflect ihe scope of
us mission and activities.
Administrator Reilly asks
NACEPT lo help the Agency
promote pollution prcvenlion.
=
~ =
£ I
\Env. Statistics Cmte
Adminisiraior Browner asks
NACEPT lo expand its charier
to incorporate a wider range of
ideas and issues.
\Env. Info, and Assessments Cmte^
\IRM ~~\
\lnfo. Impacts Cmte.
\EIFAC
Note: Appendix A: NACEPT Standing Committee Acronyms presents the full name and dates of operation for
each committee.
Purpose of this Study
The purpose of this study was to assess NACEPT's effectiveness and impact over its first decade
of operations and to develop recommendations to enhance its value in the future. The study
examined NACEPT brganizational processes, products, and impacts to assess both the efficiency
and effectiveness of NACEPT and its standing committees from the perspectives of those serving
on the Council and EPA officials who are the Council's clients. The study also examined the
interaction between the Council and its standing committees and between the Council and EPA,
especially the Office of Cooperative Environmental Management (OCEM) which supports the
Council on behalf of the Administrator.
The study was conducted in three stages of data collection, involving: (1) extensive review of
NACEPT and OCEM records and external data sources; (2) a written survey targeting all past
NACEPT: Past and Future ES-5
-------
and current NACEPT members; and (3) detailed one-on-one interviews with a selected sample of
NACEPT members and EPA officials. Additional details on the study methodology are included
Appendices F, G, H, I, K, and L.
Principal Findings
Several tests of effectiveness were used in this study to assess NACEPT's impact, including the
extent to which the Council and its standing committees had:
* fulfilled their charges;
* brought new, outside perspectives to EPA;
* provided timely and relevant advice and recommendations; and
influenced the outcome of Agency policy decisions and/or course of program
activities.
By all of these standards, NACEPT has been a success. Principal findings of the study are
summarized in Exhibit ES-l on page ES-2.
Through its standing committees, NACEPT has produced over 1,000 recommendations presented
in over 50 major reports. In addition to informal advice, counsel, and insight were provided
directly to EPA officials in the course of NACEPT meetings. Both NACEPT members and
Agency officials interviewed attest to the new perspectives that NACEPT has brought to issues
put before it and to the quality and timeliness of its recommendations on these issues. By a
significant margin, past and current NACEPT members valued their service on NACEPT and
rated their standing committee's work as valuable to EPA, For example,
* Survey respondents were very positive about the value of the NACEPT process
and the advice NACEPT provides EPA decision-makers.
* Respondents particularly value the diversity of perspectives captured within the
NACEPT process.
* Over three-quarters of respondents indicated that they would serve on NACEPT
again if asked,1
Agency officials echoed these views and, as shown in Exhibit ES-5, there is an extensive list of
Agency decisions and actions that can be traced back to recommendations made by NACEPT.
This list, which is only a partial list of NACEPT's impact, indicates the considerable influence of
NACEPT on Agency policies and actions over the last decade.
1 NACEPT members volunteer their expertise and time and are not compensated although travel costs are
reimbursed.
NACEPT: Past and Future ES-6
-------
Exhibit ES-5. Impacts of NACEPT by Topic Area
INFORMATION MANAGEMENT AND TECHNOLOGY
Standing Committees
Impacts on EPA Policy
Environmental Information
and Assessments Committee
Environmental Statistics
Committee
Information Resources
Management Strategic
Planning Task Force
Environmental Information
and Public Access
Committee
Information Impacts
Committee
OIRM management incorporated NACEPT recommendation into the Agency's
Strategic Management Plan.
EPA launched the Facility Identification Initiative to streamline access and
reporting by establishing a uniform set of facility identification data.
EPA's IRM Strategic Plan incorporated much of the language contained in the
IRM Task Force recommendations.
Consistent with NACEPT advice, the Agency has created a Chief Information
Officer to oversee the Agency's information management.
Advice of the Information Impacts Committee was cited in a June 1998 Agency
audit of the Office of Water's Data Integration Efforts.
Recommendations of the Environmental Information and Assessment Committee
influenced the Agency's Office of Research and Development's management
strategy for scientific data.
EPA established a Center for Environmental Information and Statistics as
recommended by the Environmental Statistics Committee.
PROGRAM/MEDIA SPECIFIC POLICY
Standing Committees
Impacts on EPA Policy
Environmental
Measures/Chemical
Accident Prevention
Committee
Superfund Evaluation
Committee
Food Safety Advisory
Committee
Total Maximum Daily Load
Committee
Effluent Guidelines Task
Force
Waste Isolation Pilot Piartt
Review Committee
Toxic Data Reporting
Committee
Tolerance Reassessment
Advisory Committee
The advice of EMCAP has been used in the measurement process in the
Agency's pollution prevention program.
EPA's Superfund Administrative Reforms adopted many of the concepts
embodied in the SEC's recommendations.
The Integrative Environmental Justice Model Demonstration Approach developed
by SEC was incorporated into the OSWER Environmental Justice Action Agenda
developed by the National Environmental Justice Advisory Council (NEJAC).
EGTF recommendations have led to limitations on the use of synthetic-based
drilling fluids, an examination of rules addressing coal mining operations, and
revisions to the feedlot category as well as a commitment from the Agency to
write regulations for dams.
EPA now recommends that states publish their methodology for TMDL listings
and establish related data quality assurance measures.
The Agency has incorporated earlier stakeholder participation in the development
of specific Effluent Guideline Rules.
EPA has used approaches developed by FSAC to make FQPA regulatory
decisions.
EPA is currently utilizing the framework developed by TRAC to investigate
science policy areas related to FQPA and tolerance reassessment.
NACEPT: Past and Future ES-7
-------
Input from the TDR committee has led to revisions to Form A reporting
requirements and resulting procedures,
NACEPT advice was incorporated into the Agency's compliance criteria for
WIPP.
EPA now recommends the establishment of community groups to increase public
involvement in the remediation of radiation contaminated sites.
GENERAL AGENCY/ POLICY/MANAGEMENT
Standing Committees
Impacts on EPA Policy
Environmental Education
and Training Committee
Pollution Prevention
Education Committee
State and Local Programs
Committee
Technology Innovation and
Economics Committee
Trade and Environment
Committee
International Environmental
Committee
Environmental Financial
Advisory Board
Ecosystems Sustainable
Economies Committee
Ecosystems Implementation
Tools Committee
Environmental Information,
Economics, and
Technology
Community Based
Environmental Protection
Committee
Reinvention Criteria
Committee
Environmental Capital
Markets Committee ,
Title VI Implementation
Advisory Committee
* EPA developed of a guidebook to help develop more sustainable economic
systems.
* EPA created of a new U.S. Environmental Training Institute.
* EPA formed of EPA's Office of Environmental Education.
4 EPA created of non-profit organizations addressing environmental education and
needs of industry.
* EPA improved delivery of environmental information to college students and
young adults.
4 EPA established of a Technology Innovation Office (TIO) to facilitate the
transfer of technologies developed in the Superfund Innovative Technology
Evaluation program.
* EPA has addressed "environmental education" in a proactive manner.
* EPA developed programs which focus on educating businesses on how to
implement environmental programs.
4 EPA developed Enviro$en$e, an electronic library of information on pollution
prevention, technical assistance, and environmental compliance.
4 The Agency has supported of projects to build state and local capacity for risk-
based planning.
4 EPA established of a the Clean Air Act Advisory Committee by the Office of Air
and Radiation.
4 EPA implemented of comparative risk and strategic planning in EPA Regions.
4 EPA increased cooperative agreements.
4 EPA increased use of performance evaluation based on outcomes.
4 EPA conducted pre-congressional consultations on the Agency's GPRA plan.
4 EPA incorporated technology incentives into the Agency's pollution prevention
strategy.
4 EPA developed techniques for ecosystem valuation.
4 EPA developed final guidance for implementation of the data elements required by
the Pollution Prevention Act.
NACEPT: Past and Future ES-8
-------
At the same time, although largely satisfied with the process, many past and current NACEPT
members recommended improvements in certain aspects of NACEPT and standing committee
operations. Most importantly, few NACEPT members indicated knowing what EPA had done
with their standing committee's recommendations, which impedes the Council's ability to
provide continuing advice and counsel on that topic. Nearly one-quarter (24%) of respondents
did not know whether the Agency had taken actions as a result of the standing committee's
advice. The other principal findings are as follows:
» While adequate direction is given to guide the work of a standing committee, an
initial agreement between the standing committee and relevant Agency offices on
the specific purpose and goals of the standing committee would improve
efficiency of the standing committee's work.
» Standing committees would benefit from more support (e.g., background
materials) on technical issues. In addition, in the case of a few standing
committees, respondents stated that agendas and meeting materials were not
distributed in a timely way.
* The efficiency of standing committee work could be improved by more frequent
meetings of the committee and improved communication between meetings to
allow the committee to advance its work between meetings.
* Better meeting management is often needed. Specifically, clear and agreed on
processes for reaching consensus need to be identified, equitable participation
must be ensured, and facilitation is needed to keep discussions on-track.
" Deliberations could be improved by having technical advisors on hand at all
meetings as well as better clerical support and equipment for real-time
collaborative group work (e.g., laptop computers to draft recommendations).
» While communication is effective between standing committee members,
communication between the Council and standing committees is limited.
Recommendations
Based on the findings summarized above and described in greater detail in the remainder of this
report, recommendations were developed in three areas related to: (I) Council operations; (2)
standing committee activities; and (3) OCEM support. These recommendations are presented in
Exhibit ES-2 on page ES-3 and described briefly below.
Recommendations For improving the NACEPT Council
» NACEPT should engage in a strategic planning effort to identify pressing or
emerging policy issues which standing committees might address. The results of
these efforts should be transmitted to the EPA Administrator on an annual basis.
NACEPT; Past and Future ES-9
-------
» NACEPT should streamline the recommendation review process to ensure that the
advice of standing committees is approved by the Council and transmitted to the
Agency in a timely fashion.
* NACEPT should conduct post-committee evaluations. Such evaluations would be
led by the standing committee chairperson and DFO and would attempt to identify
which aspects of the process worked well and where improvement or change is
needed.
+ NACEPT should better publicize itself and its work to all parts of the Agency and
to external audiences,
* NACEPT should make a concerted effort to maintain contact with its past
members. This contact should include communication regarding the actions
which have been taken by the Agency as the result of NACEPT's advice.
* For cases in which the Agency has committed to implement NACEPT
recommendations, NACEPT should request formal updates on the status of
implementation of those recommendations.
Recommendations for improving the NACEPT Standing Committees
* Standing committees should be encouraged to prioritize their recommendations
and include, where appropriate, implementation schedules and milestones for each
recommendation.
* Standing committees should request that the Agency provide a formal response to
NACEPT on the Agency's disposition toward standing committee
recommendations at a mutually acceptable interval. (For most standing
committees, the schedule of 60 days after transmittal seems reasonable.)
Recommendations for OCEM's management of NACEPT
* OCEM should maximize direct interaction between standing committees and
Agency program offices. This could be accomplished by establishing a program
office-standing committee liaison. Such a liaison would be encouraged to attend
standing committee meetings, contribute to the development of agendas and
background materials, and serve as the key intermediary on technical issues.
t
* OCEM should ensure that standing committee work is adequately planned and
managed by the Designated Federal Official (DFO) and chairperson to achieve the
standing committee goals in an efficient manner. Such planning would include
the initial development of clear objectives and timelines to guide the
investigation, as well as milestones and performance objectives by which to assess
progress.
* OCEM should develop and implement a formal facilitation program to ensure
proper support for each standing committee.
NACEPT: Past and Future ES-10
-------
OCEM should develop better ways for standing committee members to
communicate between meetings. Options provided by the Internet and
telecommunication systems should be considered.
OCEM should review and enhance the method by which new standing
committees are established. Specifically, better definition of the purpose and a
more rigorous membership selection process are needed.
OCEM should improve the orientation process for new members. Improvements
could include enhanced focus on past NACEPT work to provide committee-
specific background, as well as the development of new ways to provide training
on FACA guidelines such as online tutorials or a brief video.
Conclusion
Over its first decade, NACEPT has addressed an extensive and varied set of issues at the request
of the EPA Administrator and provided valuable recommendations, advice, and counsel on these
issues and topics. Equally important, NACEPT's recommendations have had a significant and
lasting impact on the Agency's decision making, policies, and program activities. These impacts
attest to the value of the expertise and perspectives that NACEPT is able to bring to an issue. In
addition, the usefulness of NACEPT to the Agency is demonstrated by the number of standing
committees and range of issues addressed by NACEPT at the Administrator's request in the last
decade.
NACEPT has demonstrated that it is EPA's most unique federal advisory committee. This study
has identified NACEPT as having developed a niche for providing valuable advice on broad,
cross-media issues. At the same time, NACEPT has displayed the flexibility to address specific
programmatic issues that are of a high priority or urgent in nature. This flexibility has enabled
NACEPT to be responsive to EPA, even as issues arid priorities change.
At present, NACEPT is taking the strategic initiative to reinvent itself, thereby increasing its
value to EPA. Evidence of this includes this study, the creation of the NACEPT Council's
strategic plan, customer focus and partnering, and improved processes. NACEPT's future is
filled with possibilities and promise, based on the past 10 years' exemplary record of service to
EPA and .the citizens of the United States.
NACEPT is indebted to Nancy Tosta, Gerard Bulanowski, Bill Sonntag, Tom Davis, and Patricia
Bauman who, as the members of the Study team, directed this study and developed its findings
and recommendations. Their insights, efforts, and enthusiasm were instrumental to the success
of this study.
NACEPT: Past and Future ES-11
-------
Findings
This chapter presents the findings of the study, based on both the completed written surveys and
the interviews with NACEPT members and Agency officials. These findings address all aspects
of NACEPT's operations, effectiveness, and impact as described in greater detail in Appendix F,
Overview of the Study Methodology,
The findings presented in this chapter are based on 198 completed surveys and 37 interviews.
The response rate on the written survey of past and current NACEPT members was 34%, which
is good for a survey of this type. For more complete survey results, see Appendix J. Twenty-two
of the follow-up interviews were conducted with NACEPT members, and the other 15 were with
Agency officials. In general, there was noticeable agreement between the NACEPT members
and Agency officials as to the strengths of NACEPT and areas for improvement.
Overall, NACEPT and its standing committees were found to have fulfilled their charges,
brought new, outside perspectives to EPA, provided timely and relevant advice and
recommendations to the Agency, and influenced the outcome of Agency decisions, policies, and
program activities. At the same time, opportunities for improvements were noted in certain
aspects of meeting management and support, communications, and feedback to NACEPT on the
impact and use of its advice and recommendations. The principal findings of the study are
presented below in order of importance.
Principal Findings
Overall assessment of NACEPT. NACEPT provides valuable input and advice to the Agency
from a wide variety of stakeholders.
Respondents were very positive about the NACEPT process as a whole. Seventy-six percent
agreed that NACEPT provides EPA decision-makers with valuable input from a broad range of
interested parties, and 77% agreed that they would serve on a NACEPT committee again if
asked. Further, respondents regarded the NACEPT process as an effective means for the Agency
to get advice (77% agree), a way to gather valuable input from a broad range of interested parties
(76% agree), and helpful to the Agency in identifying and addressing key issues in a timely
manner (63% agree).* On the value of their participation and NACEPT in general, comments
include:
+ As an independent advocate, NACEPT committees can serve as "the corporate
memory" for the Agency between political administrations and can help to keep
important initiatives on-track.
« Standing committees discuss stakeholder views of important issues in a structured
and timely way.
NACEPT: Past and Future !
-------
NACEPT brings people in from around the country to share their expertise and
viewpoints with policy-makers, which helps to enhance the Agency's credibility
beyond the Beltway.
Standing committees provide new perspectives to Agency considerations.
Standing committees add significant value as compared to the Federal Register
method of commenting on draft regulations since NACEPT provides stakeholder
input early in policy development, allowing the Agency to act proactively.
Since 1988, NACEPT has convened 26 standing committees at the request of the Agency to
provide stakeholder advice on specific issues. Over 700 stakeholders have served as
NACEPT members producing over 50 major reports and 1,000 recommendations to the EPA
on a variety of information management, program specific, and policy issues.
Other respondents suggested ways in which NACEPT could improve its service to the Agency:
» Request that each Agency program office share with NACEPT its top-priority
issues for the corning year. This would allow either the program office, or
NACEPT, to identify areas in which NACEPT might provide timely and relevant
advice.
* Focus on increasing NACEPT support to Regional offices on relevant issues.
» Encourage the Agency to establish grant programs to provide the financial support
to implement standing committee recommendations.
Establish greater visibility within the Agency, including establishing direct lines
of communication to the appropriate Agency offices.
* Reorganize NACEPT around four or five core topics (e.g., Information Resources
Management, media-specific issues, Agency management issues) and treat topics
in a broader, more integrated, and proactive way.
* Forge a stronger relationship between NACEPT and EPA's Science Advisory
Board.
Desigrj and implement an orientation system which includes an explanation of
how each standing committee fits into the overall NACEPT process.
» Allow senior EPA staff to serve as voting or ex-officio standing committee
members.
Continue its commitment to process improvement by conducting a follow-up
study in a few years to determine the effects of this Study.
NACEPT: Past and Future *2
-------
Impact of advice and reports. Recommendations are timely for Agency decision-making and
fulfil! standing committee charters.
The majority of respondents believed that the standing committees recommendations were timely
to the Agency's decision-making process, addressed decision-makers' needs, and introduced
new perspectives into the Agency's considerations (54%, 51%, and 57% agreed respectively).
These views are supported by the documentary research conducted to develop the standing
committee briefing papers as well as by specific written responses on the survey and in the
follow-up interviews. These impacts are presented in Exhibit 1 on the following page.
Examples cited in survey and interview responses include:
* As a result of participation at standing committee meetings, Agency staff are more
aware of the issues of concern to stakeholders and are thus better prepared to
consider the political ramifications and impacts that Agency actions might have.
» As recommended by the Total Maximum Daily Load (TMDL) Committee, EPA
now recommends that States develop a methodology for TMDL water listings
which includes measures of assurance to the quality of data.
* As recommended by TMDL, EPA committed to writing regulations for dams.
» As suggested by the Environmental Education Training Committee/Pollution
Prevention Education Committee, EPA has expanded its mission beyond end-of-
pipe considerations to include "environmental education" in a proactive manner.
* The advice of the Emergency Management Chemical Accident Prevention
committee has been used directly in the measurement process which EPA has
established for its prevention program.
* As proposed by the Effluent Guidelines Task Force (EGTF), the Agency has
revised the effluent guidelines process to incorporate very early stakeholder
participation (up to a year earlier than traditionally has been sought) as
demonstrated in the development of the Iron/Steel Effluent Guideline Rule. As
such, the EGTF has helped to effect culture changes about the role of stakeholder
participation in the rulemaking process in both EPA and certain industries.
* In accord with the advice of the Information Impacts Committee, the Agency has
strengthened the role of the Chief Information Officer (CIO) by making the
position a focal point of the information management process and establishing a
chain of responsibility that leads directly to the CIO.
* The discussions of the State and Local Committee helped to develop ideas within
the Agency which led to the creation of programs to educate businesses on how to
implement environmental programs.
The State and Local Committee's recommendations contributed to the Agency's
implementation of cooperative agreements considering all stakeholders and the
increased use of performance evaluation based on outcomes.
NACEPT: Past and Future -3
-------
Exhibit 1. Impacts of NACEPT by Topic Area
INFORMATION MANAGEMENT AND TECHNOLOGY
Standing Committees
Impacts on EPA Policy
Environmental Information
and Assessments Committee
Environmental Statistics
Committee
Information Resources
Management Strategic
Planning Task Force
Environmental Information
and Public Access
Committee
Information Impacts
Committee
Environmental Information
and Assessments Committee
Environmental Statistics
Committee
Information Resources
Management Strategic
Planning Task Force
Environmental Information
and Public Access
Committee
Information Impacts
Committee
OIRM management incorporated NACEPT recommendation into the
Agency's Strategic Management Plan.
EPA launched the Facility Identification Initiative to streamline access
and reporting by establishing a uniform set of facility identification data,
EPA's IRM Strategic Plan incorporated much of the language contained
in the IRM Task Force recommendations.
Consistent with NACEPT advice, the Agency has created a Chief
Information Officer to oversee the Agency's information management.
Advice of the Information Impacts Committee was cited in a June 1998
Agency audit of the Office of Water's Data Integration Efforts.
Recommendations of the Environmental Information and Assessment
Committee influenced the Agency's Office of Research and
Development's management strategy for scientific data.
EPA established a Center for Environmental Statistics as recommended
by the Environmental Statistics Committee.
OIRM management incorporated NACEPT recommendation into the
Agency's Strategic Management Plan.
EPA launched the Facility Identification Initiative to streamline access
and reporting by establishing a uniform set of facility identification data.
EPA's IRM Strategic Plan incorporated much of the language contained
in the IRM Task Force recommendations.
Consistent with NACEPT advice, the Agency has created a Chief
Information Officer to oversee the Agency's information management.
Advice of the Information Impacts Committee was cited in a June 1998
Agency audit of the Office of Water's Data Integration Efforts.
Recommendations of the Environmental Information and Assessment
Committee influenced the Agency's Office of Research and
Development's management strategy for scientific data.
EPA established a Center for Environmental Statistics as recommended
by the Environmental Statistics Committee.
NACEPT: Past and Future -4
-------
PROGRAM/MEDIA SPECIFIC POLICY
Standing Committees
Impacts on EPA Policy
Environmental
Measures/Chemical
Accident Prevention
Committee
Superfund Evaluation
Committee
Food Safety Advisory
Committee
Total Maximum Daily Load
Committee
Effluent Guidelines Task
Force
Waste Isolation Pilot Plant
Review Committee
Toxic Data Reporting
Committee
Tolerance Reassessment
Advisory Committee
* The advice of EMCAP has been used in the measurement process in the
Agency's pollution prevention program.
* EPA's Superfund Administrative Reforms adopted many of the concepts
embodied in the SEC's recommendations.
* The Integrative Environmental Justice Model Demonstration Approach
developed by SEC was incorporated into the OSWER Environmental
Justice Action Agenda developed by the National Environmental Justice
Advisory Council (NEJAC).
* EGTF recommendations have led to limitations on the use of synthetic-
based drilling fluids, an examination of rules addressing coal mining
operations, and revisions to the feedlot category as well as a commitment
from the Agency to write regulations for dams.
4 EPA now recommends that states publish their methodology for TMDL
listings and establish related daia quality assurance measures.
* The Agency has incorporated earlier stakeholder participation in the
development of specific Effluent Guideline Rules.
* EPA has used approaches developed by FSAC to make FQPA regulatory
decisions.
* EPA is currently utilizing the framework developed by TRAC to
investigate science policy areas related to FQPA and tolerance
reassessment.
* Input from the TDR committee has led to revisions to Form A reporting
requirements and resulting procedures.
* NACEPT advice was incorporated into the Agency's compliance criteria
for WIPP.
* EPA now recommends the establishment of community groups to
increase public involvement in the remediation of radiation contaminated
sites.
NACEPT: Past and Future «5
-------
GENERAL AGENCY/ POLICY/MANAGEMENT
Standing Committees
Impacts on EPA Policy
Environmental Education
and Training Committee
Pollution Prevention
Education Committee
State and Local Programs
Committee
Technology Innovation and
Economics Committee
Trade and Environment
Committee
International Environmental
Committee
Environmental Financial
Advisory Board
Ecosystems Sustainable
Economies Committee
Ecosystems Implementation
Tools Committee
Environmental Information,
Economics, and
Technology
Community Based
Environmental Protection
Committee
Reinvention Criteria
Committee
Environmental Capital
Markets Committee
Title VI Implementation
Advisory Committee
* EPA developed of a guidebook to help develop more sustainable
economic systems.
* EPA created of a new U.S. Environmental Training Institute.
* EPA formed of EPA's Office of Environmental Education.
* EPA created of non-profit organizations addressing environmental
education and needs of industry.
* EPA improved delivery of environmental information to college students
and young adults.
* EPA established of a Technology Innovation Office (TIO) to facilitate
the transfer of technologies developed in the Superfund Innovative
Technology Evaluation program.
* EPA. has addressed "environmental education" in a proactive manner.
* EPA developed programs which focus on educating businesses on how
to implement environmental programs.
* EPA developed EnviroSenSe, an electronic library of information on
pollution prevention, technical assistance, and environmental
compliance.
* The Agency has supported of projects to build state and local capacity
for risk-based planning.
* EPA established of a the Clean Air Act Advisory Committee by the
Office of Air and Radiation.
* EPA implemented of comparative risk and strategic planning in EPA
Regions.
* EPA increased cooperative agreements.
* EPA increased use of performance evaluation based on outcomes.
*
* EPA conducted pre-congressional consultations on the Agency's GPRA
plan.
* EPA incorporated technology incentives into the Agency's pollution
prevention strategy.
* EPA developed techniques for ecosystem valuation.
* EPA developed final guidance for implementation of the data elements
required by the Pollution Prevention Act.
NACEPT: Past and Future «6
-------
Other respondents provided written responses on the survey on the obstacles which may have
limited the impact of standing committee recommendations. These include:
There is no obligation for the Agency to implement standing committee
recommendations since standing committees have no formal authority and most
recommendations have no measures of progress for implementation.
* The standing committee did not have interaction with the appropriate level of
Agency staff where the culture changes that were sought by the recommendations
needed to occur (i.e., the standing committee's interactions were with staff too
senior to affect the day-to-day culture).
* ' The standing committee's recommendations were too vague to have an impact
because the standing committee's agenda was too large to encourage focus on
specific implementation issues.
» The effectiveness of the standing committee was limited because the Agency was
too concerned with defending its current positions. (Other respondents wrote that
the Agency perceived the standing committee's function as that of a "rubber
stamp".)
* The standing committee was unable to set its own course because of the undue
influence of Agency decision-makers in steering the standing committee's agenda
in specific directions.
* A protocol needs to be established for interaction between standing committees
and Agency offices. This should include measures to ensure that Agency
representatives have the necessary interpersonal skills to listen to a standing
committee and act as an effective liaison during the standing committee's
operations.
* The Council should proactively communicate with the Agency to ensure that the
appropriate decision-makers are aware of and have access to the standing
committee recommendations relevant to their programs.
NACEPT: Past and Future !
-------
Membership. NACEPT and standing committee membership is a balanced representation of
points of view.
Overall, respondents agreed that standing
committee membership was balanced in terms of
the points of view represented (76% agree). See
Figures 1, 2, and 3 for the Council's composition
by sector.
Figure 2.1995 Council Composition
Figure 1, 1992 Council Composition
Figure 3.1999 Council Composition
In narrative comments, many respondents indicated that the exposure to other stakeholders'
viewpoints was the greatest benefit to both themselves and the Agency. Nonetheless, some
respondents felt that their standing committee could have been more effective had more attention
been paid in the membership selection process to;
* The technical level and experience of members on the topic to be addressed.
* Recruiting members who were willing to put in the time to prepare for, actively
participate in, and work outside the standing committee's meetings,
Bringing in, when possible, representatives of organizations who have previously
reviewed the topic to be addressed.
NACEPT: Past and Future «8
-------
Seeking out and encouraging participation from individuals "outside the
Beltway".
Ensuring that each member is willing to work toward achieving consensus.
Limiting standing committee size to encourage full engagement in a true "give
and take" process.
For example, the Environmental Capital Markets Committee is charged with identifying
practical ways for the financial services industry to include the environmental performance
of its clients as an integral part of its core credit, investment, and underwriting processes.
One respondent wrote that this was difficult for the committee to do given the limited number
of committee members with relevant experience and the lack of participation from the
commercial banking and asset management sectors.
ECMC Member
Standing committee mission and purpose. Standing committees are given adequate direction
to fulfill their missions; early agreement on purpose and goals may help to improve efficiency.
Respondents generally agreed that the standing committee was provided with adequate direction
on its purpose and goals (64% agree), that the standing committee understood its purpose
(62% agree), and that the standing committee was able to determine its own direction and lines of
inquiry (63% agree). However, some respondents saw a lack of a defined purpose which limited
efficiency. Specific comments include:
* The Agency's expectations for the standing committee should be made as clear as
possible. These expectations should include:
specific statements of what the standing committee is expected to do;
clear goals of what products and advice the Agency would like from the
standing committee;
an explanation of what the Agency has already done relevant to the topic
and how the standing committee's activities will be integrated with these
activities;
, an explanation of the reasons why recommendations are needed and why
the particular standing committee membership has been selected;
guidelines for limitations to the standing committee's deliberations
including what types of data the Agency will accept; and
identification of the key issues and priorities on which the standing
committee should focus its resources and activities.
NACEPT: Past and Future «9
-------
The Agency had already conducted a significant amount of analysis and assessment of
options for streamlining the Effluent Guidelines process. As a result, the Task Force wasn 't
sure what its job was since EPA had left few loose ends.
EGTF Member
Some respondents advocated a more interactive development of the standing committee's charge
by the Agency and the standing committee. Specific suggestions include:
* what the standing committee's work products should look like;
» what would be a reasonable timetable for the production of work products and
Agency feedback; and
» what the feedback from the Agency to the standing committee should be.
Such negotiations would allow a reasonable degree of involvement by the standing committee to
determine its own course of inquiry yet incorporate sufficient Agency involvement to ensure that
the standing committee's work meets the Agency's specific needs.
Formulation of advice and reports. Standing committee recommendations are developed in a
timely, inclusive fashion.
Overall, respondents agreed that the recommendations developed by the standing committee
fulfilled the standing committee's purpose (56% agree1). Further, respondents agreed that the
recommendations were developed through a productive dialogue with the appropriate Agency
staff (67% agree); included considerations of practicality (58% agree); were developed in a
timely manner (52% agree); and were presented in a clear and specific way (52% agree).
Nonetheless, narrative responses and interview responses provided useful suggestions for
improvement, including:
» Conduct more case studies and identify real-world examples to support
recommendations. This approach would allow the Agency to better consider how
effective implementation of recommendations is likely to be.
Have EPA staff from relevant program offices present at standing committee
meetings to hear all aspects of the standing committee's discussions, some of
whicH may not be included in final recommendations.
+ Make feasibility a key consideration in the development of recommendations.
Reference in standing committee recommendations the reports and documents
which have been used in deliberations and include as many of these as appendices
as needed.
Adjusted percentage excluding "Not Applicable" and "Skipped" responses.
NACEPT: Past and Future «10
-------
* Encourage participation and feedback from relevant Agency staff to discuss the
feasibility of standing committee proposals (e.g., statutory and legal limitations).
Facilitate communication between the standing committee and relevant Agency
managers during the early stages of the development of recommendations. These
discussions should address the willingness of Agency managers to carry such
recommendations forward.
* Include dissenting opinions in standing committee recommendations.
» Streamline NACEPTs recommendation review process to minimize delays in
approving and transmitting recommendations to the Administrator.
Feedback on impact of advice and reports. Most respondents have not received feedback from
the Agency on the impact of their standing committee's recommendations.
In both the survey and interviews respondents stated they received little feedback regarding the
disposition of their recommendations. Only 27% of respondents agreed that the standing
committee had received any feedback from the Agency as a result of the standing committee's
advice or even that the Agency valued the advice as a positive contribution to its decision-
making process (24% and 20% "Do Not Know" responses respectively). Suggestions on ways to
improve the feedback to the standing committee include that NACEPT should:
» Establish a process by which an Agency response to standing committee
recommendations is required after a reasonable period for review by the
Administrator and program offices. This response should come from appropriate
Agency officials and provide specific feedback on the Agency's intent to
implement the recommendations. For those respondents who identified a specific
timeframe, six months was the most commonly suggested review period. Further
aspects of such suggestions include:
*
send quarterly e-mails and conduct annual phone calls to provide updates
on the status of the implementation of standing committee
recommendations;
send standing committee members a copy of newly issued regulations with
a cover note providing an explanation of the actions which the Agency
t took; and
send a short note reiterating the committee's recommendations and
detailing what the Agency is doing in response.
» Encourage EPA to invite a few standing committee members to serve as a
monitoring group during the implementation of recommendations and to provide
further advice on unresolved issues or new problems that emerge.
NACEPT: Past and Future 11
-------
> Ensure its past members know that the Agency appreciates and is using their
recommendations through an annual newsletter or other formal feedback
mechanism.
> Ensure that any Agency feedback received by a standing committee chairperson is
promptly distributed to all standing committee members.
» Establish and maintain a list of the current status (e.g., adopted, under
consideration, rejected) of all NACEPT recommendations.
While many past participants had not received feedback from the Agency, other respondents
wrote examples of feedback methods which had worked for their committee. These examples
include:
* For the State and Local Programs Standing committee, some program offices sent
a representative to standing committee meetings to give feedback. These
representatives reported on the implementation of standing committee
recommendations, asked clarifying questions on other recommendations, and
explained why certain recommendations could not be implemented. Other offices
sent written memos. Feedback was generally given six months after advice was
issued by the committee, which worked well with the Agency's timeframe.
» One Agency official contacted the Total Maximum Daily Load standing
committee members since its completion to discuss the standing committee's
recommendations.
Communication within NACEPT. Communication between standing committee members is
adequate but improvements are needed for communications between meetings and to the
Council.
While quantitative data showed that respondents felt communication among standing committee
members was effective (71% agree), fewer than one-third agreed that there was effective
communication between the standing committee and the Council (31% agree; Note: 30%
expressed no opinion). Additionally, written comments and interview responses further
indicated that respondents saw opportunities for improvement in communication among
members through both technical and procedural changes. Specific comments include:
> Provide better technological tools to communicate such as conference calls and
on-line posting of meeting agendas and background materials to improve the
timeliness of communication and reduce the volume of paper generated.
» Establish a way to for standing committee members to communicate between
standing committee meetings (and possibly with other standing committees and
the Council as well). Specific suggestions for such methods include:
NACEPT: Past and Future »12
-------
Send standing committee members meeting summaries that include what
the Council would like to see next from each standing committee.
Create a NACEPT web page for NACEPT members to use as a working
tool. This page could provide relevant documents for members to read,
progress updates on NACEPT activities and relevant agency activities, and
a forum for members to share views and materials.
+ Encourage more communication between the standing committee and the Council
by appointing a standing committee member to serve as a "Council Liaison."
+ Facilitate communication across standing committees to promote cross-
fertilization of ideas and avoid duplication of effort.
Meeting management. Standing committee meetings are generally well-planned and structured
yet improvements can be made in defining the consensus process, ensuring equitable
participation, and keeping decisions on track.
While the majority of respondents agreed that standing committee meetings were well-planned
and structured to achieve the standing committee's goals (55% agree), nearly one-quarter of
respondents disagreed (23% disagree). These respondents offered a wealth of suggestions to
improve meeting management, including:
" Ensure that a clear focus is set for each meeting, which keeps the work focused on
achieving the standing committee's goals.
» Establish and enforce meeting ground rules.
» Identify at the outset of the standing committee the standards for standing
committee decision-making (i.e., process for seeking consensus).
» Define parameters (both in terms of time and scope of issues) for standing
committee deliberations.
Clearly assign tasks so hardest part of the process was keeping
that everyone understands everyone focused and happyt as committee
what is expected of
whom, and by when.
Form Breakout groups to
focus on specific issues
and develop preliminary
recommendations.
members with different competing interests
started t
A standing committee chairperson
Some respondents suggested that to improve the meetings process, each standing committee
should have a professional facilitator assigned to the standing committee. By having an
independent, objective procedural leader to keep discussions on track, maintain order, and ensure
equitable participation, less burden would be placed on the chairperson, and all members could
NACEPT: Past and Future -13
-------
focus on the issues at hand. Other respondents added that the facilitator should be well-versed in
the topic area and that the same individual should work with the standing committee throughout
the entire process.
TMDL used a color scheme to vote on recommendations. Red meant you could not live with
including the recommendation, beige meant you could live with it, and green meant you
thought it should definitely be included. WJien the final report was developed, a paragraph
was usually included that addressed the other perspectives raised during deliberations.
TMDL Member
Technical support and background materials. Standing committee members were generally
positive about receiving timely and useful background and technical materials to make informed
decisions,
Most respondents agreed that materials provided to the standing committee were effective for use
in standing committee discussions (68% agree) and adequate to support informed standing
committee decision-making (67% agree). In addition, respondents were positive regarding the
timeliness, usefulness, and thoroughness of the Agency's response to requests for information
(78%, 70%, and 61% agree respectively2). A minority of respondents were critical of the
information provided to the committee. Specific comments include:
* Adequate time to review background materials was often not provided
materials were frequently received during the week before meetings.
» Some background materials could have been better focused to the specific needs
of the standing committee.
*
> It took several meetings for new standing committee members to "get up to
speed," which limited the effectiveness of standing committee discussions.
» For some standing committees, too much time was spent debating the definition
of key terms rather than addressing the issues.
> The information made available to the standing committee was not technical
enough to support the task the standing committee had been given.
* Some standing committees were not kept aware of relevant initiatives within the
Agency.
* Real time technical advice was needed to answer specific questions raised during
standing committee discussions.
Adjusted percentage based on exclusion of "Not Applicable" responses.
NACEPT: Past and Future -14
-------
» More involvement by Agency staff to prepare materials, perform analysis, and
assist the standing committee is desirable.
» EPA's response to requests for information could have been more timely and
more thorough.
Meeting schedule and location. More frequent meetings and improved communication bet\veen
meetings could improve standing committee effectiveness.
Respondents generally agreed that sufficient notice was given of meetings and for most
committees, that the frequency of standing committee meetings were sufficient (77% and 61%
agree respectively). However, for some standing committees, respondents reported that the
meeting schedule limited the standing committee's effectiveness and efficiency. Specific
comments on standing committee meeting schedules include:
* The schedule left too much time between meetings and resulted in inefficiency
"catching up" during meetings on developments and activities that occurred
between meetings.
Meetings were too infrequent to address the evolving nature of the standing
committee's topic (i.e., information management/information technology issues).
* Quarterly meetings were the most commonly suggested schedule by respondents
who suggested more frequent meetings.
* Standing committee meetings should include more opportunities for public
participation to determine how other stakeholders view standing committee
proposals.
* Local/regional committees could develop preliminary recommendations. Then, a
national standing committee could integrate, review, and modify these preliminary
suggestions into a formal report.
Meeting support. Better meeting support can be achieved through use of technology.
In addition to better meeting management, several respondents suggested that the efficiency of
meetings could be improved through better meeting support and resources such as:
4
> More administrative support before, during, and after meetings to ensure that
materials are prepared and distributed in a timely manner.
* Providing standing committees computers to use during meetings to facilitate
efficient drafting and revision of recommendations. If the standing committee is
to be working as a whole, the ability to project the computer display on a screen
would be necessary.
NACEPT: Past and Future »15
-------
Conclusion
Overall, NACEPT and its standing committees were found to have:
> fulfilled their charges;
* brought new, outside perspectives to EPA;
» provided timely and relevant advice and recommendations to the Agency;
and
» influenced the outcome of Agency decisions, policies, and program
activities.
At the same time, the effectiveness and impact of NACEPT would be increased with
improvements in the following areas: meeting management and support, communications and
feedback from EPA on the impact and use of its advice and recommendations, as summarized in
Exhibit 2 below. Specific recommendations related to these findings are presented in the next
chapter.
Exhibit 2. Summary of Principal Findings
NACEPT has many strengths:
Provides valuable input and advice to the Agency from a wide variety of stakeholders.
Provides a balanced representation of points of view through its membership.
Provides timely recommendations for Agency decision-making consistent with the charges of the
standing committees.
Provides standing committees with adequate direction to fulfill their missions.
Operates through its standing committees in generally well-planned and structured meetings with
sufficient technical support and background materials. ,
but also opportunities for improvements in the following areas:
Feedback from the Agency on the impact and use of recommendations and advice.
Early agreement on goals and purposes among standing committee members to ensure efficient
progress.
Communication among standing committee members between meetings and between committees and
the Council.
Planning for standing committee activities including defining the consensus process, methods for
ensuring equitable participation, and providing for sufficient numbers of meetings to accomplish the
committee's charge.
Facilitation and meeting support.
NACEPT: Past and Future -16
-------
Recommendations
This chapter presents the Study team's recommendations based on its analysis and interpretation
of the survey and interview data. These findings confirm NACEPT's importance and impact
over the last 10 years and validate and support its continuance as an independent source of
stakeholder involvement. The findings also indicate opportunities for improvement in the
Operating effectiveness of the NACEPT and its standing committees, which could translate into
greater impact on and value to EPA. These specific opportunities for improvement are presented
below organized as follows:
recommendations to the NACEPT Council;
recommendations to standing committees; and
* recommendations to OCEM.
Recommendations to the NACEPT Council
I. NACEPT should more actively engage in strategic planning to identify the policy
issues which NACEPT standing committees address.
The Study team found that NACEPT has provided valuable advice and introduced new
perspectives to the Agency. This advice has covered a wide range of topics and expanded to
new topic areas with success. Due to this demonstrated success in expanded areas, it is likely
that NACEPT could further add value by identifying additional areas in which NACEPT believes
stakeholder input would aid Agency decision-makers. Specific ways in which NACEPT can be
more proactive and strategic in the advice it provides include:
> Survey Agency offices to determine what the major issues for the Agency will be
in the coming year. Based on the results, have the Council (or its Executive
Committee) identify the committees currently operating which might be able to
address those issues. Where the issues are beyond the scope and expertise of
extarrt committees, recommendations for new standing committees could be
developed to be presented to the Administrator.
Engage key Agency stakeholders (such as the Science Advisory Board and non-
governmental organizations) in identifying Agency policies which might be
improved through review by the NACEPT process. Have the NACEPT Council
and (or its Executive Committee) review these suggestions and make
recommendations to the EPA Administrator.
NACEPT: Past and Future -17
-------
II. NACEPT should better publicize itself and its work to all parts of the Agency and
beyond.
While the Study team found that many Agency officials are well-informed of NACEPT and its
advisory role, others were not aware of NACEPT activities relevant to their office. To maximize
the value of NACEPT to the Agency, NACEPT must publicize its works through activities such
as:
* An enhanced Internet presence. NACEPT's website should be integrated with
other Agency sites relevant to the topics which NACEPT is investigating. OCEM
should partner with such offices to highlight NACEPT's role in the office's
decision-making.
Partnering with other Agency stakeholder involvement programs to create and
maintain a comprehensive list of stakeholder advice reports. To ensure the utility
of such a database, advertise its usefulness to EPA staff (both upper management
and line staff)-
* Utilizing existing Agency communication mechanisms (e.g., Enviro-Newsbrief)
to inform Agency staff about the availability and content of NACEPT reports and
products.
III. NACEPT should streamline the process of developing and delivering
recommendations.
The Study found that standing committee members were often frustrated by the delay in the
delivery of advice to the Administrator caused by the Council's review and approval of
recommendations. These findings suggest that NACEPT should re-evaluate its recommendation
approval process to provide advice more quickly and directly. Potential changes to the
recommendation process might include:
* De-emphasize the final report as the primary output of a standing committee.
Instead, seek ways in which to involve program offices and Agency officials more
in the standing committee discussions and the development of recommendations.
Whert appropriate, standing committee reports might even include reactions from
or descriptions of program office involvement.
* Invite program office officials to comment on the standing committee's draft
recommendations prior to the preparation of the final report.
* Investigate technological options such as Internet-based collaborative groupware
to expedite the Council review process.
NACEPT: Past and Future -18
-------
Incorporate Council review earlier in the recommendation development process to
comment on interim recommendations.
Implement techniques within the Council to ensure the thorough yet timely review
of standing committee reports. One such technique would be the establishment of
Council review subcommittees which would be responsible for framing and
presenting any issues which require Council discussion to the larger Council body
and shepherding the report through such review.
IV. NACEPT should conduct an evaluation of standing committee processes upon the
completion of the standing committee's work.
This study was the first opportunity for many NACEPT members to provide direct feedback to
the Council on their view of how well the NACEPT standing committee process worked. The
findings revealed a wealth of information which will be used to improve future NACEPT efforts.
To ensure the continuous improvement of NACEPT, it is recommended that similar self-
assessments be conducted at the close of each standing committee. To efficiently conduct such
an evaluation, NACEPT might:
> Conduct a short written survey of all standing committee members on standing
committee processes and other topics similar to those addressed in this study.
* Conduct telephone interviews with a random sample of standing committee
participants to gain further insight on the NACEPT process.
* Encourage each DFO to conduct "close-out" interviews with the standing
committee-program office liaison and standing committee chairperson to discuss
the management of the standing committee and assess whether all issues laid out
in the standing committee's work plan were efficiently and comprehensively
addressed. (The DFO would use performance measlires set at the outset to guide
this process and develop a final assessment including lessons learned.)
» Provide NACEPT members with a specific point of contact within OCEM who
would be available for discussion of any aspect of the NACEPT process on which
the members has a concern or question.
V. NACEPT should take responsibility for maintaining contact with its past
members.
Perhaps the most surprising aspect of the Study was the extent to which past participants were
unaware of NACEPT's and the Agency's actions after the end of their standing committee.
NACEPT's alumni represent a vast network of talented and committed individuals, many of
whom continue to participate extensively in the public debate on environmental protection and
NACEPT: Past and Future -19
-------
management at the federal, state, tribal, and local level. They also represent a body of supporters
for the value of NACEPT and its advice. As a long-term institutional body, it is the Council
which is in the position to monitor and communicale to past and present members. These
communications should highlight what NACEPT is currently addressing as well as recent
activities the Agency has taken in the areas in which NACEPT standing committees have
worked. Specifically, NACEPT could:
» Develop an internal tracking system of standing committee recommendations and
resultant Agency activities. The system would be updated as the Agency
implements activities based on NACEPT recommendations (potentially identified
by standing committee-program office liaison) and the results summarized in a
periodic newsletter.
* Establish an e-mail list-serve on the NACEPT website which would send bulleted
e-mail updates on current NACEPT and Agency activities to those interested.
These short e-mails would not be comprehensive, but focused on directing
recipients to specific websites should further information be desired.
Recommendations to Standing Committees
L Standing committees should prioritize their recommendations and include
suggested schedules and performance targets for implementation of each
recommendation.
While both NACEPT participants and Agency officials generally viewed standing committee
recommendations as valuable to Agency decision-making, several.participants noted that
standing committee recommendations were too vague, too unrealistic, or too conceptual to allow
adequate assessment of Agency implementation to be made. As a result, it is recommended that
standing committees:
+ Consider the practicality of implementation and attempt to avoid any
recommendation with technical, statutory, and regulatory barriers which might
hinder successful implementation.
Prioritize their recommendations in order of importance.
* Limit the overall volume of recommendations to heighten focus on the most
important few.
* Include, where possible, implementation goals and performance measures as part
of the recommendations. (Such goals and measures could also serve as ways in
which to assess the Agency's use of standing committee recommendations.)
NACEPT: Past and Future »20
-------
II. Standing committees should request a formal response from the Agency to all
standing committee reports,
The least positive aspect of the Study findings was the extremely limited extent to which
members had received feedback as to the Agency's use of the standing committee's development
of each recommendation. To improve this lack of feedback, it is recommended that:
* Appropriate Agency officials be encouraged to attend and participate in the
standing committee's recommendations to provide immediate feedback on
specific issues when possible.
» An appropriate time frame be agreed during the establishment of the standing
committee for Agency review and comment on recommendations.
» Standing committee members be informed of the Administrator's or program
office's decisions on the standing committee recommendations. When possible,
implementation schedules should be created and included to allow subsequent
follow-up by interested standing committee members.
» At an appropriate time after completion of the standing committee's work, the
program office liaison or other program office official should communicate to
standing committee members updates on the disposition of standing committee's
recommendations.
Recommendations to the Office of Cooperative Environmental
Management (OCEM)
I. OCEM should ensure productive interaction between NACEPT standing
committees and relevant Agency program offices.
While NACEPT is formally chartered to provide advice to the EPA Administrator, several
standing committees and program offices have worked intensively and directly together very
successfully. Such direct interaction seems to have benefitted both the standing committee and
the program office. The standing committee gained valuable technical assistance. The program
office, which developed a better understanding of the diversity of individual stakeholder
perspectives, received input in a much quicker timeframe, and was exposed to the standing
committee's discussions beyond the final consensus recommendations. Based on such successes,
it is suggested that OCEM actively foster more direct collaboration between NACEPT standing
committees and relevant Agency offices. To encourage better cooperation between these groups,
OCEM could:
NACEPT: Past and Future "21
-------
Work closely with the appropriate program office managers during the
establishment of the standing committee to ensure that the standing committee has
the right membership composition and charge to provide the advice which the
program office needs.
Require the program office to designate a "standing committee-program office
liaison" and allow for the requirements of this role in budgeting staff time.
Guidelines for such a liaison should include technical background, attendance at
meetings, and post-committee responsibilities (e.g., ensuring a response to the
recommendations).
Sponsor a meeting of the standing committee chairperson, DFO, standing
committee-program office liaison, and program office managers in advance of the
first committee meeting. The goal of this meeting would be to build consensus
regarding the priorities and goals for the standing committee, establish a sense of
teamwork, and develop shared understanding of the time line of activities for the
standing committee.
II. OCEM should ensure that standing committee work is adequately planned and
managed by the DFO and standing committee chairperson to achieve the
committee's goals in an efficient manner.
While survey respondents and interviewees reported that, in general, standing committees
provided the Agency with valuable recommendations in a timely fashion, several noted that the
recommendation development process could be improved. Specifically, few standing
committees utilized project management techniques (such as activity workplans) to guide the
standing committee's investigation. To improve the efficiency of NACEPT's standing
committees, it is recommended that OCEM effectuate better standing committee management by
methods such as:
Develop a set of standing committee management guidelines which identify
effective project management techniques for standing committee work. Such
guidelines should include techniques to:
develop an workplan and timeline;
<, define interim and endpoint goals;
set performance measures and methods to measure performance toward
goals; and
keep the standing committee's work on track.
> Require that the DFO, standing committee chairperson, and relevant program
office staff develop a workplan, timeline, and product definition for the standing
committee's activities. (These documents could then be used for scheduling
NACEPT: Past and Future »22
-------
Council review of standing committee products, to assess the standing
committee's degree of operational success, and by OCEM to determine whether a
standing committee has completed its work.)
III. OCEM should inform program offices of the qualifications needed to be a DFO
and provide training to appointed DFOs.
Due to the need for NACEPT's operations to be compliant to Federal Advisory Committee Act
regulations, the DFO and OCEM are responsible for ensuring proper standing committee
operations. To ensure efficient management and oversight of this compliance, OCEM should
take additional steps to ensure that every standing committee DFO is adequately trained and
conversant in FACA requirements. Findings from the survey and interviews suggest that:
> OCEM should ensure all DFOs are well trained on the NACEPT process. The
most effective method would be that the DFO preferentially be from OCEM. If it
is necessary that the DFO be from another office, it is important that this
individual has specific FACA training prior to the establishment of the standing
committee. In addition, program office DFOs should be provided a contact within
OCEM to act as a mentor throughout the process. This mentor would be a
resource to ensure that the standing committee is always in compliance with the
requirements of FACA.
» At the first standing committee meeting, the DFO should ensure that each
standing committee member is aware of the relevant FACA requirements as well
as the way in which the NACEPT process works (i.e., explain the Council review
process).
IV. OCEM should develop better ways for NACEPT members to communicate
between meetings.
The Study team found that the majority of communication within NACEPT occurs during
standing committee meetings. While such communication has generally been adequate to
support standing committee decision-making, it is evident that increased communication with
other NACEPT standing committees, the Council, and between meetings might increase the
efficiency of the standing committee's work and allow consensus to be reached more quickly.
Based on these findings, it is suggested that OCEM:
» Develop a website for standing committees to use as a resource and tool to
accomplish their work. Such a website should include:
security features which protect the integrity of committee communications
but allows access from universal locations;
NACEPT: Past and Future «23
-------
- a designated area for each standing committee containing its activity
workplan, background materials, and agendas; and
user-friendly features to allow appropriate standing committee members to
post comments or materials (for example, templates could be provided to
allow quick posting of meeting summaries).
Provide DFOs with technical tools and training to enhance the use of electronic
communication to standing committee members (for example, Adobe Acrobat
software and training might enhance e-mail distribution of documents; a group
voice-mail system could be used to alert members of newly released documents
available on the Internet).
Develop an electronic newsletter to be issued after Council meetings detailing the
events of the last meeting, upcoming key events of various standing committees,
and other NACEPT announcements.
V. OCEM should establish an enhanced formal method for the establishment of
standing committees.
In the case of a few standing committees, the Study team found that standing committee
members were uncertain as to the standing committee's purpose and the expectations of the
Agency. In addition, some respondents expressed concerns that their standing committee's
membership lacked the necessary technical and legal knowledge to provide specific
recommendations. To address these findings, it is recommended that OCEM re-evaluate the
process by which new standing committees are established. To improve the process, OCEM
might:
* Develop a brochure which details what a program office needs to do to have a
NACEPT standing committee established. The process should include both
practical considerations, such as the paperwork required by OCEM, as well as the
conceptual needs, such as framing an issue requiring advice.
» Involve one or more Council representatives with experience in the proposed
topic area to review the program office's request for a standing committee. As
individuals familiar with both the NACEPT process and the area of study, these
individuals are in a unique position to help clarify the standing committee's
purpose, provide a preliminary stakeholder's perspective on the topic, and assess
the timefrarne necessary to address the scope of the investigation.
> Work collaboratively with the program office and the Council to identify
potential members who have a willingness to work toward consensus and the
required expertise.
NACEPT: Past and Future *24
-------
VI. OCEM should improve the NACEPT and standing committee orientation process
for new members.
The Study team found that on some issues (such as the organizational structure and roles of
NACEPT) members did not fully understand how NACEPT works or recognize that previous
standing committees had addressed very similar topics. Several respondents wrote that such lack
of awareness was detrimental to standing committee efficiency. To address this issue, it is
suggested that OCEM re-evaluate its current orientation process. Suggestions for improving the
orientation program include:
* Develop an orientation process which can be used effectively both at the
establishment of a standing committee as well as for members who join later in
the standing committee process.
> Send new standing committee members a briefing paper on the committee. This
paper would present the standing committee's charge, activities to date, and prior
relevant NACEPT recommendations and reports in this topic area.
* Produce an "Introduction to NACEPT" video which provides new members a
brief history and overview of NACEPT and the relevant aspects of the Federal
Advisory Committee Act. This video could be sent to new participants and
program office liaisons in advance of meeting attendance, shown at a pre-meeting
coffee, or posted on the website as a digital video so that valuable meeting time is
not consumed by the need to orient new members.
> Create an informational online briefing which, using simple graphics and text,
which allows a new NACEPT member to review the NACEPT process and FACA
background at a convenient time through the Internet. (If desired, a paper copy
could be developed as well and mailed to individuals at their request.)
VII. OCEM should develop a formal facilitation program to ensure proper support for
each standing committee.
While survey respondents stated that the NACEPT process worked adequately, the Study team
found that the efficiency of standing committee meeting management can be substantially
improved. To take advantage of this opportunity, it is recommended that a facilitation program
be implemented for all standing committees. Specific aspects of such a facilitation program
might include:
* Provide each DFO and standing committee chairperson with a guidebook which
details how to host efficient and effective FACA meetings and outlines the
potential role and use of trained facilitators to assist in the committee's work.
Specific topics within the manual should include guidelines for distribution of
meeting agenda and advance materials; techniques and options for standing
NACEPT: Past and Future «25
-------
committee decision-making processes; and guidance for how to determine if
professional facilitation is required.
Develop a roster of qualified facilitators which NACEPT can call upon for
support to standing committees. The roster should focus on facilitators who have
subject manner knowledge in the environmental field, FACA, and appropriate
experience with senior-level managers within both private industry and
government.
Make resources available for meeting facilitators and meeting support staff (e.g.,
note-takers, laptop computers, etc.) especially for working meetings during which
recommendations will be developed.
Conclusion
Although the Study team found that overall NACEPT has been an effective source of stakeholder
involvement and has had a significant positive influence on EPA's decision making, policies, and
program activities, it also found areas for improvement. These recommendations, addressed to
the Council, its standing committees, and OCEM, concern both the process of NACEPT's
operations and the substance of its work. Exhibit 3 presents a summary of the recommendations.
Exhibit 3. Recommendations
The NACEPT Council should:
» Do more strategic planning to identify the policy issues which NACEPT standing committees
address.
* Better publicize itself and its work to all parts of the Agency and beyond.
« Streamline the process of developing and delivering recommendations.
Conduct an evaluation of standing committee processes after the completion of the standing
committee's work.
Take responsibility for maintaining contact with its past members.
NACEPT Standing Committees should:
Prioritize their recommendations and include suggested schedules and performance targets
for implementation for each recommendation.
Request a formal response from the Agency to all standing committee reports.
NACEPT: Past and Future *26
-------
The Office of Cooperative Environmental Management should:
Ensure productive interaction directly between NACEPT standing committees and relevant
Agency program offices.
* Ensure that standing committee work is adequately planned and managed by the DFO and
standing committee chairperson to achieve committee goals in an efficient manner,
* Make clear to the program offices the qualifications needed to be a DFO and provide
training to appointed DFOs.
* Develop better ways for NACEPT members to communicate between meetings.
Establish an enhanced formal method for the establishment of standing committees.
* Improve the NACEPT and standing committee orientation process for new members.
Develop a formal facilitation program to ensure proper support for each standing
committee.
NACEPT: Past and Future »27
-------
Conclusion
Since its establishment in 1988, NACEPT has served as an important and independent source of
stakeholder advice to EPA across a wide range of subjects. Through its 26 standing committees,
NACEPT has involved over 700 past and current members in meaningful dialogue and
deliberation and has produced over 50 major reports containing over 1,000 recommendations.
The value of these recommendations, advice, and counsel is confirmed not only by the views of
NACEPT past members and Agency officials, but also in the numerous documented examples of
the recommendations influencing the Agency's decisions, policies, and program activities. As a
result, the number of NACEPT standing committees and the range of issues that they have
addressed has grown over the last years.
From this rich heritage of success, NACEPT is preparing for its second decade and making
adjustment to ensure and enhance its continued effectiveness. Among these changes are the
continued expansion in the breadth and variety of issues on its agenda and a greater focus on
strategic planning, including efforts to identify emerging issues of importance to EPA and its
constituents.
This study continues and contributes to these improvement efforts by providing findings and
recommendations related to NACEPT's operations and effectiveness. Collectively, these
findings and recommendations will help to guide NACEPT, its standing committees, and OCEM
in ensuring the ongoing vitality and importance of NACEPT into the future.
NACEPT: Past and Future »28
-------
Appendix A NACEPT Standing Committee Acronyms
EiggmCTJii
I^SffiK
CBEP
ECMC
EETC
EFAB
EGTF
E1AC
EIET
E1PAC
EITC
EMCAP
ESEC
ESTATS
FSAC
11C
IRM
PPEC
RCC
S&L
SEC
TOR
TEC
TIE
TITLEVI
TMDL
TRAC
WIPP
Communily Based Environmenial Proteclion
Committee
Environmental Capital Markets Committee
Environmental Education and Training
Committee
Environmental Financial Advisory Board
Effluent Guidelines Task Force
Environmental Information and Assessments
Committee
Environmental Information Economics and
Technology Committee
Environmental Information and Public
Access Committee
Ecosystems Implementation Tools Committee
Environmental Measures/Chemical Accident
Prevention Committee
Ecosystems Sustainable Economies
Committee
Environmental Statistics Committee
Food Safety Advisory Committee
Information Impacts Committee
Information Resources Management
Strategic Planning Task Force
Pollution Prevention Education Committee
Reinvention Criteria Committee
State and Local Programs Committee
Superfund Evaluation Committee
Toxic Data Reporting Committee
Trade and Environment
Committee/International Environmental
Committee
Technology Innovation and Economics
Committee
Title VI Implementation Advisory Committee
Total Maximum Daily Load Committee
Tolerance Reassessment Advisory Committee
Waste Isolation Pilot Plant Review Committee
!"»'«; Sj^p.*^r
1996-1997
1998-Ongoing
1988-1990
1989-1990
1992-Ongoing
1994-1996
1995-1996
1998-Ongoing
1994-1995
1990-1996
1994-1996
1992-1997
1996
1996-1997
1994
1991-1993
1996-Ongoing
1988-1993
1993-1994
1993-Ongoing
1989-1993
1989-1993
1998-Ongoing
19%- 1998
1998
1992-Ongoing
General Agency Policy or Management
General Agency Policy or Management
General Agency Policy or Management
General Agency Policy or Management
Program/Media-Specific
Information Management and Technology
General Agency Policy or Management
Information Management and Technology
General Agency Policy or Management
Program/Media-Specific
General Agency Policy or Management
Information Management and Technology
Program/Media-Specific
Information Management and Technology
Information Management and Technology
General Agency Policy or Management
General Agency Policy or Management
General Agency Policy or Management
Program/Media-Specific
Program/Media-Specific
General Agency Policy or Management
General Agency Policy or Management
General Agency Policy or Management
Program/Media-Specific
Program/Media-Specific
ProgranVMedia-Specific
NACEPT: Past and Future Appendix A-l
-------
Appendix B History and Overview of NACEPT
Reasons for Establishment
During the 1980s, a variety of factors led to the realization by the EPA that in the future, both
non-federal government agencies and the private sector would have a larger role in
environmental protection. In addition, federal legislation such as the Federal Technology
Transfer Act of 1986 encouraged joint government-industry collaboration to develop new
technologies. In accord with these trends, in March 1987, EPA Administrator Thomas created
the Task Force on Technology Transfer. This Task Force was charged to develop
recommendations as to how EPA could most effectively leverage its resources to support and
encourage technology transfer and training through cooperative activities with industry,
academia, and non-federal government agencies. As a result of these events, the National
Advisory Council for Environmental Technology Transfer (NACETT), was established in July
1988.
Original Charge
The National Council for Environmental Technology Transfer (NACETT) was charged to
consult with and make recommendations to the EPA Administrator on a continuing basis on
technology transfer issues associated with:
the management of environmental problems;
activities, functions, and policies relevant to the Agency under the Federal
Technology Transfer Act of 1986; and
other statutes, executive orders, and regulations affecting the conduct of
technology transfer activities within EPA.
NACETT's activities were designed to:
promote continuing consultation and debate to ensure mutual understanding of the
differing perspectives, concerns and needs among the institutions involved in
environmental management;
» maximize the extent to which each institutional participant understands, accepts,
and fulfills its environmental management responsibilities;
facilitate broad public sharing of information on environmental problems as well
as alternative approaches and implementation strategies for addressing them; and
* promote consideration of alternative strategies for leveraging resources to address
environmental needs.
NACEPT: Past and Future Appendix * B-l
-------
Evolution of the Charge
1988-1990
During its first two years, NACETT standing committees investigated various aspects of
technology transfer delivering recommendations on environmental education and training, state
and local programs, and technology innovation issues. In addition, NACETT sponsored projects
that resulted in the development of new institutions which demonstrated real-world cooperative
environmental management, assisted in the formulation of an Agency position on reauthorization
of the Resource Conservation and Recovery Act (RCRA), and addressed other issues in response
to internal Agency requests (such as requests from the Office of Solid Waste and Emergency
Response to provide input to chemical accident prevention and hazardous waste remediation
issues.) It became clear that the Council's initial technology transfer title did not fit the broad
environmental policy formulation role to which the Council had evolved and as a result, the
Council was renamed from NACETT to the National Advisory Council on Environmental Policy
and Technology (NACEFT) in July 1990. To guide NACEPT, a Mission Statement was
adopted:
NACEPT
Bridging the gap from problem identification to environmental solutions through successful
program implementation, cooperation, and consensus-building by business, government,
educational institutions, and private organizations.
1990-1993
In November 1990, Administrator Reilly met with the NACEPT Executive Committee and asked
the Council to address broad, cross-cutting issues related to pollution prevention to provide input
for the Agency's Congressional reports regarding the then-new Pollution Prevention Act.
Consequently, from 1990 though 1993, NACEPT standing committees investigated Chemical
Accident Prevention, Effluent Guidelines, the use of environmental statistics in pollution
prevention, and education issues related to pollution prevention.
1993 - Present
In 1993, Administrator Browner continued NACEPT's central stakeholder advisory role and
expanded NACEPT's charge to provide advice on an even wider range of issues including
information management and technology. Since 1993, NACEPT standing committees have
addressed topics sucn as community based environmental protection, Agency reinvention, and
information resource management. In 1997, a collaborative effort between the Administrator's
office, the Office of Reinvention, and Office of Cooperative Environmental Management
(OCEM) resulted in a realignment of NACEPT creating a revised NACEPT structure increasing
the role of program offices in the NACEPT process through standing committee management
with OCEM oversight. Today, NACEPT works collaboratively with the Office of the
Administrator, EPA program offices, and other federal agencies to investigate a breadth of
NACEPT: Past and Future Appendix B-2
-------
environmental protection related issues including food safety regulations, environmental justice
policies, and public access to environmental information.
NACEFFs Structure
NACEPT is organized by a three-tiered committee structure of the NACEPT Council, an
Executive Committee, and standing committees. Each type of committee has a specific role in
the NACEPT process. An organization chart of NACEPT is presented in Appendix C.
NACEPT Council
The NACEPT Council is the formal long-term established body which is responsible for
providing advice across a breadth of issues to the Administrator. To accomplish this, the Council
operates as a steering committee, coordinating, overseeing, and reviewing the work of the
standing committees which are established under the Council's auspices.
The Council is comprised of senior-level representatives from a broad range of interests
including government, business and industry, academia, professional associations, and labor,
environmental advocacy, and community groups who serve renewable terms (usually two years).
Council members generally also serve as a member of one or more NACEPT standing
committee,
NACEPT Executive Committee
NACEPT's Executive Committee, with the assistance of OCEM, is responsible for the overall
planning for the Council and coordination between standing committees. It is comprised of the
Chair and Vice-Chair of the Council and standing committee chairpersons.
NACEPT Standing Committees
The majority of NACEPT's operations are accomplished through standing committees, which are
Council subcommittees established and charged to address specific issues. Standing committees
are comprised of qualified professionals experienced in the area of the standing committee's
focus. Membership includes representatives from government, business, industry, academia, and
relevant non-governmental organizations to assure balanced consideration of the range of
perspectives. While, all Council members serve on at least one standing committee, standing
committees also include issue-specific experts who are not Council members.
Standing committees operate with independent courses of action and standing committee
chairpersons. Standing committees hold public meetings, establish workgroups or
subcommittees to look at issues in depth, and interact with relevant Agency program offices.
Committees develop advice to fulfill their charges and present draft reports to the Council for
review, approval, and transmittal to the Administrator or other Agency customer.
NACEPT: Past and Future Appendix B-3
-------
Overview of the NACEPT Process
NACEPT is a stakeholder-input process which is utilized for a variety of reasons to accomplish a
great range of goals. The reasons why NACEPT standing committees are established and the
way in which specific standing committees operate may vary considerably. In that sense, there is
not a single "NACEPT process". Nevertheless, the process can be broadly characterized through
four steps:
1) NACEPT is asked to provide advice.
2) A standing committee is charged to consider issues and develop draft advice.
3) The standing committee's draft advice is reviewed and approved by the Council.
4) NACEPT advice is provided, possibly including dissenting opinions and
recommendations for further study.
Management of NACEPT
NACEPT is a federal advisory committee, governed by federal legislation such as the Federal
Advisory Committee Act (FACA) of 1972. FACA requires that advisory groups be:
(1) chartered; (2) balanced and diverse in terms of perspective, professional qualifications, and
experience; and (3) kept to a minimum, established only when "determined to be essential" and
disestablished upon completion of work. Further, the business of these groups is to be:
(1) announced in the Federal Register; (2) open to public attendance and comment as agency
guidelines permit; and (3) conducted promptly.
FACA legislation includes specific management guidelines for federal advisory committees.
OCEM is responsible for the operations and management of the NACEPT Council and Executive
Council. In addition, FACA requires that each standing committee has a Designated Federal
Official (DFO) to oversee its operations and FACA compliance. DFOs assist the standing
committee chairperson, the Executive Council, and OCEM in the planning of the standing
committee's work and act as a communications conduit within the standing committee and
between the standing committee and relevant EPA program offices. In most cases, the DFO is a
member of the OCEM staff, although it is possible for the DFO to be an Agency program office
staffperson working under the oversight of OCEM.
NACEPT: Pas! and Future Appendix B-4
-------
Appendix C 1999 Organizational Chart of NACEPT, OCEM, and
the Administrator's Office
EPA Administrator
Carol M. Browner
Deputy Administrator
Peter D.Robertson
Office of Cooperative Environmental Management
Director
Clarence Hardy
I
EPA
Regional and Program Offices
NACEPT
Chairman
Robert L Rhodes, Jr.
Executive Committee
Composed of Comrniit/e Chairpersons
| General Agency Policy or Management
i Standing Committees
Program or Media Specific
Standing Committees
information Management and Technology
i Standing Committees
NACEPT: Past and Future Appendix C-l
-------
Appendix D - Key Aspects of NACEPT Charters
1988 - Original Charter
j
Purpose and Authority
The purpose of the Advisory Council is to provide advice and counsel to the Administrator of EPA
on technology transfer associated with the management of environmental problems. The Advisory
Council is being established in accordance with the provisions of the Federal Advisory Committee
Act 5, U.S.C. (App.I)9(c). The Advisory Council provides independent advice and counsel to the
Administrator on such specific technology transfer activities, issues and needs as:
identifying the barriers impeding environmental technology transfer and training
efforts and possible approaches for reducing these barriers;
creating a positive institutional climate within EPA with respect to technology
transfer and training activities;
promoting cooperative, mutually-supportive EPA-state relationships aimed at
establishing more effective environmental management at federal, state and local
levels;
increasing and institutionalizing communication among all levels of government, the
business community, the academic, educational, and training community and the
international community, with the aim of increasing non-federal resources and
improving the effectiveness of federal and non-federal resources directed at solving
environmental problems, and establishing direct links between these resources and
those who need assistance to solve environmental problems;
developing and applying an appropriate array of existing and new delivery
mechanisms for meeting technology transfer and training needs;
implementing the Federal Technology Transfer Act of 1986 which facilitates access
to science and technology, and other related legislation, executive orders and
regulations previously enacted or which may be enacted in the future;
&
reviewing any periodic EPA reports describing the Agency's progress in
implementing statutes, executive orders and regulations on technology transfer; and
assessing alternative approaches for measuring the environmental benefits of
technology transfer activities.
NACEPT: Past and Future Appendix D-l
-------
Scope of the Activity
The Advisory Council advises, consults with, and makes recommendations on a continuing basis to
the Administrator on technology transfer issues associated with the management of environmental
problems generally and in matters relating to activities, functions, and policies under the Federal
Technology Transfer Act of 1986 and other statutes, executive orders and regulations affecting the
conduct the technology transfer activities within EPA. The Advisory Council will analyze problems,
present findings, make recommendations, conduct meetings and perform other activities necessary
for the attainment of its objectives. Environmental technology transfer consists of the purposeful
transfer of technical and environmental management information and "know how" from one
individual or organization to one or more others where it is needed to achieve environmental
protection objectives. Such technology transfer may take the form of training, technical assistance or
targeted information dissemination. It includes such transfers between and among interstate, state,
regional and local agencies with environmental responsibilities, EPA regional offices, EPA
headquarters and EPA laboratories. It also includes such transfers between and among businesses;
academic, educational and training institutions; federal, state and local government organizations;
international organizations; and governmental organizations in other countries, especially such
transfers undertaken to facilitate or accelerate the development, commercialization or use of needed
new environmental technology or skills.
Composition
The Advisory Council consists of a group of independent experts drawn from industry and business,
academic, educational and training institutions; federal, state and local government agencies;
international organizations; environmental groups; and non-profit entities. The group shall be of
sufficient size and diversity to provide the range of perspectives required to assess each element of
the implementation of the Federal Technology Transfer Act and related statutes, executive orders and
regulations and, generally, the technology transfer issues associated with the management of
environmental problems. The Advisory Council may constitute itself into such specialized
committees on an ad hoc or standing basis as it finds necessary to carry out its responsibilities.
Duration
The Advisory Council shall be needed on a continuing basis and may be renewed beyond its initial
two-year period, as authorized in accordance with Section 14 of the Federal Advisory Committee
Act.
NACEPT: Past and Future Appendix D-2
-------
1990 Charter Renewal (and renaming)
Purpose and Authority
This Charter is to renew the National Advisory Council for Environmental Policy and Technology
(NACEPT) which was previously established as the National Advisory Council for Environmental
Technology Transfer (NACETT) on July 7, 1988, for an additional two-year period in accordance
with the requirements of the Federal Advisory Committee Act, 5 U.S.C. (App.I) 9(c). The purpose of
the Advisory Council is to provide advice and counsel to the Administrator of EPA on issues
associated with the management of environmental problems. The Advisory Council provides
independent advice and counsel to the Administrator on such specific activities, issues and needs as:
identifying the barriers impeding environmental technology transfer and training efforts and possible
approaches for reducing these barriers; creating a positive institutional climate within EPA with
respect to technology transfer and training activities; promoting cooperative, mutually-supportive
EPA-state relationships aimed at establishing more effective environmental management at federal,
state and local levels; increasing and institutionalizing communication among all levels of
government, the business community, the academic, educational and training community and the
international community, with the aim of increasing non-federal resources and improving the
effectiveness of federal and non-federal resources directed at solving environmental problems, and
establishing direct links between these resources and those who need assistance to solve
environmental problems; developing and applying an appropriate array of existing and new delivery
mechanisms for meeting technology transfer and training needs; implementing the Federal
Technology Transfer Act of 1986, Executive Order 12591, which facilitates access to science and
technology, and other related legislation, executive orders and regulations previously enacted or
which may be enacted in the future; reviewing any periodic EPA reports describing the Agency's
progress in implementing statutes, executive orders and regulations; and assessing alternative
approaches for measuring the environmental benefits of technology transfer and related activities.
Scope of the Activity
The Advisory Council advises, consults with and makes recommendations on a continuing basis to
the Administrator on issues associated with the management of environmental problems generally
and on matters relating to activities, functions and policies under the Federal Technology Transfer
Act of 1986 and other statutes, executive orders, and regulations affecting the conduct of technology
transfer activities within EPA. The Advisory Council will analyze problems, present findings, make
recommendations, conduct meetings and perform other activities necessary for the attainment of its
objectives. Environmental technology transfer consist of the purposeful transfer of technical and
environmental management information and "know how" from one individual or organization to one
or more others where it is needed to achieve environmental protection objectives. Such activity may
take the form of training, technical assistance or targeted information dissemination. It includes such
transfers between and among interstate, state, regional and local agencies with environmental
NACEPT: Past and Future Appendix D-3
-------
responsibilities, EPA regional offices, EPA headquarters and EPA laboratories. It also includes such
transfers between and among businesses; academic, educational and training institutions; federal,
state and local governmental organizations; international organizations and governmental
organizations in other countries, especially such transfers undertaken to facilitate or accelerate the
development, commercialization or use of needed new environmental technology or skills.
1994 Charter Renewal
Purpose and Authority
This Charter renews the National Advisory Council for Environmental Policy and Technology
(NACEPT) which was originally established on July 7, 1988, for an additional two-year period in
accordance with the requirements of the Federal Advisory Committee Act (FACA), 5 U.S.C. App. 2
section 9(c). The purpose of NACEPT is to provide advice and counsel to the Administrator of EPA
on issues associated with environmental management and policy. It is determined that NACEPT is
in the public interest in connection with the performance of duties imposed on the agency by law.
Objectives
NACEPT provides independent advice and counsel to the Administrator on issues such as;
» developing approaches for reducing barriers to environmental technology
development and transfer, institutionalizing public and private pollution prevention
programs, ecosystems management and environmental sustainability and community
empowerment;
fostering improved global environmental management, and increasing the focus on
environment in international trade and contributions to U.S. competitiveness;
*.
increasing communication and understanding among all levels of government,
business, non-governmental organizations and academia, with the goal of increasing
non-federal resources and improving the effectiveness of federal and non-federal
resources directed at solving environmental economic problems;
» implementing statutes, executive orders and regulations previously enacted or which
may be enacted in the future;
» reviewing progress in implementing statutes, executive orders and regulations; and
« assessing approaches for measuring the environmental benefits of technology transfer
and alternative approaches to environmental protection.
NACEPT: Past and Future Appendix » D-4
-------
Scope of the Activity
NACEPT advises, consults with, and makes recommendations on a continuing basis to the
Administrator on issues associated with environmental management generally and on matters
relating to activities, functions and policies under the federal environmental statutes, executive
orders, regulations, and policies affecting environmental management responsibilities of EPA.
NACEPT conducts meetings, analyzes problems, presents findings, makes recommendations,
and performs other activities as necessary for the attainment of its objectives. It advises the
Administrator on ways to improve development and implementation of domestic and
international environmental management policies, programs, and technologies. Ecosystems
protection, NAFTA implementation and Information Resources Management Strategic Planning
are some of the issues currently under review. NACEPT also provides external input to Assistant
Administrators on selected program topics where appropriate. NACEPT, working with other
EPA organizations, advises the Administrator on broad, cross-cutting environmental policy and
technology issues and priorities.
NACEPT: Past and Future Appendix D-5
-------
Appendix E Bibliography of NACEPT Publications
General Reports of the Council
National Advisory Council for Environmental Policy and Technology: An Overview
September 1990,
* Progressive Environmental Management: Leveraging Regulator)' and Voluntary Action
March 1993.
Promoting Innovative Approaches to Environmental Protection: A Summary of
Recommendations from the National Advisory Council for Environmental Policy and
Technology June 1996.
General Agency Policy or Management
Technology Innovation and Economics Committee
* Report and Recommendations of the Technology Innovation and Economics Committee
January 1990.
* Permitting and Compliance Policy: Barriers to U.S. Environmental Technology
Innovation: Report and Recommendations of the Technology Innovation and Economics
Committee January 1991.
* Improving Technology Diffusion for Environmental Protection: Report and
Recommendations of the Technology Innovation and Economics Committee October 1992.
* How Best to Promote Industrial Pollution Prevention Through the Effluent Guidelines
Process: Report of the Technology Innovation and Economics Committee/Industrial
Pollution Prevention Project Focus Group February 1993.
* Transforming Environmental Permitting and Compliance Policies to Promote Pollution
Prevention: Removing Barriers and Providing Incentives to Foster Technology Innovation,
Economic Productivity, and Environmental Protection: Report and Recommendations of
the Technology Innovation and Economics Committee April 1993.
» Report and Recommendations for Action: "EPA's Technology Innovation Strategy and
Program Plans for the Environmental Initiative" August 1994.
State and Local Programs Committee
« Report and Recommendations of the State and Local Programs Committee February 1990.
Implementation of Recommendations October 1990.
State and Local Programs Committee Recommendations March 1991.
NACEPT: Past and Future Appendix E-l
-------
Building State and Local Pollution Prevention Programs December 1992.
Environmental Financial Advisory Board
* Environmental Tax Policy Statement Draft Recommendations March 1990.
* Small Communities Financing Strategies Workgroup Draft Recommendations March 1990
Public Financing Options Workgroup Draft Recommendations March 1990.
* Private Sector Incentives Workgroup Draft Recommendations March 1990.
Environmental Education and Training/Pollution Prevention Education
Committee
* National Advisory Council for Environmental Technology Transfer: Report and
Recommendations of the Environmental Education and Training Committee 1990.
» National Advisory Council for Environmental Policy and Technology: The Urban
Environmental Education Report December 1990.
» Pollution Prevention Education and Training for an Environmentally Sustainable Future:
Report and Recommendations of the Academic Focus Group of the Pollution Prevention
Education Committee October 1992.
* Partnership-Building to Promote Pollution Prevention: Industry Focus Group Report
October 1992.
Partnerships for Pollution Prevention Education and Training December 1992.
Trade and Environment/International Environmental Committee
The Greening of World Trade February 1993.
Ecosystems Implementation Tools Committee
» Interim Report of the NACEPT Implementation Tools Committee on EPA's Place-Based
Approach to Ecosystem Management January 1995.
Ecosystems Sustainable Economies Committee
« NACEPT Ecosystems Sustainable Economies Committee FY 1995 Activities and
Recommendations June 1996.
Environmental Information, Economics, and Technology
« Peer Review Of Analysis of Cost-Based Environmental Technology Gaps June 1996.
Peer Review Of Resource-Based Method For Identifying Environmental Technology
Priorities July 1996.
NACEPT: Past and Future Appendix » E-2
-------
Reinvention Criteria Committee
Letter to the Deputy Administrator: Preliminary findings and recommendations October 21
1996.
Letter to the Deputy Administrator: Preliminary findings and recommendations April 18,
1997.
Recommendations on EPA's Draft Strategic Plan July 1997.
Interim Report of the Reinvention Criteria Committee March 1998.
Community Based Environmental Protection Committee
Report and Recommendations of the Community-Based Environmental Protection
Committee 1997.
Program/Media Specific Policy
Environmental Measures/Chemical Accident Prevention
National Environmental Information Goals and Objectives for the 2V Century: Draft
Interim Recommendations of the Environmental Statistics Subcommittee April 1992.
Report of the Pollution Prevention Measurements Subcommittee June 1992.
Establishment of a Center for Environmental Statistics at EPA: Interim Recommendations
1992.
Measuring Progress in Chemical Accident Prevention: Recommendations of the Chemical
Accident Prevention Subcommittee September 1992.
Superfund Evaluation Committee
Superfund Liability Workgroup Recommendations Nove/nber 1993.
Remedy Selection Workgroup Recommendations November 1993.
State Role Workgroup Recommendations November 1993.
Municipal Liability Workgroup Recommendations November 1993.
Effluent Guidelines Task Force
s
The Effluent Guidelines Program: Selection Criteria for Preliminary Industry Studies July
1994.
Effluent Guidelines Task Force Workgroup 1 - Issue Paper: Design of Preliminary Studies
September 1996.
Fostering Pollution Prevention and Incorporating Multi-Media Considerations into
Effluent Guidelines Development September 1996.
Effluent Guidelines Task Force: Design of Preliminary Studies September 1996.
NACEPT: Past and Future Appendix E-3
-------
Removing the Bottlenecks from the Effluent Guidelines Process October 1996.
Recommendations on Streamlining the Effluent Guidelines Development Process: Draft
Report May 1998.
Food Safety Advisory Committee
Summary Report of Food Safety Advisory Committee December 1996.
Total Maximum Daily Load Committee
Report of the Federal Advisory Committee on the Total Maximum Daily Load (TMDL)
Program July 1998.
Tolerance Reassessment Advisory Committee
Framework for Addressing Key Science Issues Presented by the Food Quality Protection
Act (FQPA) as Developed Through the Tolerance Reassessment Advisory Committee
(TRAC) October 1998.
Framework for Refining FQPA Science Policy October, 1998.
Schedule for Release of Guidance on Science Policy Issues October, 1998.
Toxics Data Reporting Committee
Issues and Concerns for the Definitions and Guidance for the Requirements of 6607 of the
Pollution Prevention Act; Summary of Discussion of the Toxics Data Reporting
Subcommittee of the National Advisory Council for Environmental Policy and Technology
January 1994.
Information Management and Technology
Information Resources Management Strategic Planning Task Force
Using Information Strategically to Protect Human Health and the Environment:
Recommendations for Comprehensive Information Resources Management August 1994.
Environmental Statistics Committee
*
Fiscal Year 1995 Recommendations of the Environmental Statistics Subcommittee 1995.
Environmental Information and Assessments Committee
Findings and Recommendations of the Ecosystems Information and Assessments
Committee June 1996.
NACEPT: Past and Future Appendix E-4
-------
Information Impacts Committee
» Interim Report January 1997.
Managing Information as a Strategic Resource: Final Report and Recommendations of the
Information Impacts Committee January 1998.
NACEPT: Past and Future Appendix E-5
-------
Appendix F Overview of the Study Methodology
A key methodological challenge of this study was defining what "effectiveness" means in the
context of a stakeholder advisory process such as NACEPT. Viewed narrowly, NACEPT
provides a structured process of providing advice to the EPA Administrator in accordance with
the requirements of the Federal Advisory Committee Act (FACA). From this perspective, testing
the effectiveness of NACEPT is relatively straightforward and can be measured and assessed in
terms of the number of standing committees constituted and the reports and recommendations of
the Council.
Another, broader definition of effectiveness would be to assess the extent to which NACEPT's
recommendations had influenced or changed EPA's decision making, policies, or actions. Such
a definition poses greater methodological challenges because EPA may already be addressing
these issues and establishing cause-and-effect relationships between NACEPT's
recommendations and subsequent Agency actions may be difficult. Still another definition of
effectiveness would include the extent to which NACEPT had created new dialogue on issues or
added new perspectives, which might have influenced decisions and activities outside of EPA,
such as at the state and local level.
Based on input from current NACEPT members leading this study, no single definition was
exclusively used. Instead, the Study team identified the following four areas of focus to
determine NACEPT's effectiveness:
* Did NACEPT standing committees fulfill their charters?
» Did NACEPT bring outside perspectives to EPA?
» Did NACEPT provide useful and timely recommendations?
* Did NACEPT have observable effects on Agency policy?
These areas of focus represent different tests of effectiveness and provide a tiered set of standards
for assessing NACEPT's performance over the last 10 years. The methodology described below
reflects the need to collect data on NACEPT related to each of these four areas of focus.
To investigate these questions, the Study team adopted a two-prong approach: (1) to reach as
many past and present NACETT-NACEPT participants as possible; and (2) to interview a
representative sample of Agency officials. In this manner, the Study was intended to gather data
on both the way in which NACEPT functions internally as well as how it is perceived by and
impacts Agency decision-makers.
NACEPT: Past and Future Appendix « F-l
-------
A three-stage process was developed to conduct the Study:
Stage 1: NACEPT and OCEM records and other external data sources (i.e..
Internet) were reviewed to document and chronicle the activities of
NACEPT;
Stage 2: Past and current NACEPT members were surveyed to gather participants'
views of the quality of the NACEPT experience and standing committee
effectiveness; and
Stage 3: Selected survey respondents and Agency officials were interviewed to
validate the survey data and to gather additional data on NACEPT's
impacts.
Stage 1: Review of Relevant Documentary Records and Reports
NACEPT operates through a central management council and standing committees established to
provide advice to the Agency on specific topics. Over its 10-year history, 26 such standing
committees have been constituted. The first step in the Study was to document the various
activities of these standing committees and identify addresses for their members. This research
effort included the review of OCEM, NACEPT, and other EPA records, and Internet research to
identify the charge, activities, recommendations, and impacts (where directly attributed to a
committee) of each standing committee. For each, a briefing paper was developed for providing
a short explanation of why the standing committee was established, a summary of its charge, a
review of its activities, and a listing of its reports and recommendations. These briefing papers
aided the Study team in developing the survey questionnaire and were included in the survey
mailing to provide respondents with a contextual reminder of the standing committee's purpose,
activities, and recommendations. (Based on information gathered during the Study, these
briefing papers have been updated and further developed to include the impacts of standing
committee recommendations on the Agency.) The briefing papers are collected in a report
entitled, NACEPT: Chronology and History Report.
Stage 2: Study Survey
680 written surveys were sent to past and present NACEPT members to evaluate their standing
committee experiences. The survey consisted of 44 closed-ended questions (Likert-scale and
multiple choice); fivfe open-ended narrative questions; and five demographic questions (gender,
race/ethnicity, age, sector represented, and number of standing committees on which the
respondent served). Questions were organized in four sections:
committee charge, processes, & procedures;
committee recommendations and advice;
impact of committee recommendations and advice; and
overall assessment of standing committee and NACEPT experience.
NACEPT: Past and Future Appendix F-2
-------
At the end of each survey, respondents were asked to indicate their willingness to participate in a
follow-up interview. The study, therefore, relied heavily on the opinions of past and current
NACEPT members, who may be viewed as potentially bias sources of information. However,
since NACEPT members do not receive compensation for their services and many past members
no longer have immediate contact or professional relationships with EPA, the Study team
believed that the risk of such bias was modest and could be qualitatively evaluated by comparing
the views expressed to the historical record. For example, opinions related to NACEPT's impact
on specific program and activities could be verified through the interviews with Agency officials
described below and information publicly available about EPA programs. In addition, because a
major focus of the study was to identify improvement in NACEPT's operations, past and current
members were viewed as the most informed sources on the processes and procedures followed by
the Council and its standing committees and their effectiveness.
Stage 3: Study Interviews
To corroborate the findings of the survey and gather additional information, interviews were
conducted with a selected sample of survey respondents, Agency officials, and OCEM staff. To
ensure that these interviews gathered information representative of the variety of NACEPT
standing committee experiences, a subset of six standing committees representing a range of
subject areas and survey response averages was selected by the NACEPT Study team for these
in-depth, follow-up interviews. These six standing committees were:
Total Maximum Daily Load Committee;
» Effluent Guidelines Task Force;
Reinvention Criteria Committee;
Toxic Data Reporting Committee;
State and Local Programs Committee; and
«
Information Impacts Committee.
Interviews were conducted by an independent survey firm via telephone and in person and
covered the same topics as the Study survey with added focus on gathering supporting details and
suggestions for improving future NACEPT efforts. For each standing committee, multiple
interviews were conducted to promote a balanced and comprehensive set of perspectives.
NACEPT: Past and Future Appendix F-3
-------
Timeline and Key Activities
April
May
June
July
August
September
October
November
December
Kick-off meeting at NACEPT plenary session
Researching and drafting of the standing committee briefing papers
Development of the study methodology
First Study team meeting
Development of draft survey and sample
Revisions to the standing committee briefing papers
Second Study team meeting
Survey in the field
Development of interview protocols
Analysis of survey data
Third Study team meeting
Interviews conducted
Interviews conducted
Presentation of preliminary results to NACEPT Council
NACEPT: Past and Future Appendix F-4
-------
Appendix G Technical Details of Survey Methodology
Purpose
The purpose of the survey was to gather the views and opinions of as many past and current
NACEPT members as possible on their NACEPT experience. Specifically, the survey:
» asked their assessment of the quality of their experience participating on NACEPT;
solicited their opinion of the effectiveness of NACEPT; and
» determined their willingness to participate in a follow-up interview.
Approach
The survey was sent to all past and current NACEPT members for whom reliable current mailing
address information was obtained. The survey was sent to NACEPT members rather than
external stakeholders and EPA for several reasons:
» Outside stakeholders are unlikely to offer valuable information on how standing
committees conduct their business.
» Limiting the survey to members allowed all past and present members to be surveyed,
improving the quality and breadth of information gathered.
* It was expected that external stakeholders and EPA officials were less likely than past
members to complete the survey since those with direct personal involvement with an
organization are generally more likely to complete surveys about that organization. This
consideration was significant since the survey addressed the work of NACEPT
committees that existed as far back as 10 years.
Instrument
The survey was admmistered as a written questionnaire. It consisted of 44 multiple choice
Likert-scale questions and five open-ended narrative questions. The format of the instrument
provided boxes in which the respondent could mark their response to close-ended questions.
Text boxes were provided for replies to open-ended questions. Instructions for completing and
returning the survey were included at the beginning of and throughout the survey. Overall, it was
estimated that completion of the survey would take approximately 20 minutes.
In addition, the survey asked five demographic questions. Instructions encouraged respondents
NACEPT: Past and Future Appendix G-l
-------
to answer these questions, but made it clear that responses to gender, race/ethnicity, and age
questions were optional. All respondents had the option of completing the survey anonymously.
At the end of the survey, respondents were asked to indicate if they were willing to participate in
a follow-up interview, and if so, to provide pertinent contact information.
Survey Pre-Test
The draft survey form and instructions were pre-tested by the NACEPT Study team, OCEM, and
the survey contractor to ensure that instructions and questions were clear, logically organized,
and unambiguous.
Survey Distribution and Follow-up
The survey was distributed by U.S. Express Mail to emphasize its importance and to ensure that
it "stood out" from other mail. Each survey package included:
» a cover letter from the NACEPT chairperson explaining the purpose of the study;
» background material about NACEPT, FACA, and the standing committee on which the
individual participated [For respondents who served on multiple standing committees,
multiple chronology reports and survey forms were provided to allow completion of one
survey for each standing committee];
» the written survey pre-coded to indicate the committee in question; and
a pre-addressed postage-paid return envelope. [Note: Instructions were also provided for
return of completed surveys by fax.]
A follow-up postcard was sent approximately one week after the survey distribution, reminding
the recipient of the survey and encouraging the recipient to complete and return the survey.
Follow-up telephone calls were also made selectively to encourage survey response for those
committees with low response rates.
NACEPT: Past and Future Appendix G-2
-------
Appendix H Survey Instrument
National Advisory Council for
Environmental Policy and Technology
Membership Survey
The purpose of this survey is to gather feedback from current and past NACEPT
committee members to support NACEPT's self-assessment of its accomplishments
and progress during its first ten years. The data from this survey will complement
information collected from other sources and will help to assess the efficiency of
NACEPT's internal processes as well as the impact of its committees. Together, these
data will be used to identify opportunities for improving the overall effectiveness of
NACEPT's internal processes and stakeholder service.
INSTRUCTIONS
- Please complete the attached survey for the Committee noted at the top of the next page (page
1). In answering the questions, please consider your personal experience on that specific
Committee and mark (with an "X") the response which best indicates your view.
» Most questions use a standard response set: Strongly Agree; Agree; Neither Agree nor
Disagree; Disagree; Strongly Disagree; Do Not Know; and Not Applicable.
2
<
oc
a
o
L.
D
1!
«
a
2*
DC L.
li
£i
D
D
£
US
a
D
01
M
Q
f
O
VI
D
*«
u
a.
o
£
D
o
£
o
Z
£
D
fti
XI
"o.
o.
o
a
If you sometimes agree
and sometimes
disagree in roughly
equal proportion, mark
"Neither Agree nor
Disagree."
Questions 18, 27, 35, 36, and 41 ask for narrative responses. Please write your response in
the space provided. If you require additional space, you may continue on the back side of the
page. «
The survey is ANONYMOUS if you prefer. Both survey and interview data will be collected and
processed by a non-federal contractor. Results will be presented in cumulative statistics to
ensure the anonymity of individual respondents.
If you are willing to participate in a follow-up interview about your NACEPT experience, please
provide your name and contact information at the end of the survey. Participation in the follow-
up interviews is strictly voluntary.
NACEPT: Past and Future Appendix H-1
-------
Please complete this survey for the following committee: \ ',
< Committee label \vas placed here ;
BACKGROUND INFORMATION
In which of the following sectors did you work when appointed to this Committee?
Q Federal, State, or Local Government Agency a College/University or other Academic
LI Corporate, Private, or Self-Employed Research Organization
Q Trade Union or Labor Organization a Advocacy or Public Interest Non-
a Professional or Trade Association Governmental Organization
Q Other
On how many NACEPT Committees have you served?
Q 1 a 2 as a 4 Q5 or more
Which of the following best characterizes your attendance at Committee meetings?
a Never a Attended less a Attended Q Attended more a Always attended
attended than half about half than half
How would you characterize your experience serving on this Committee?
a. a Worth my time and effort Q A/of worth my time and effort
b. a Committee's work had a Q Committee did not have a Q Do Not
noticeable effect on Agency noticeable effect on Agency Know
decision-making. decision-making.
c. Q In hindsight, I would serve on the Q In hindsight, I would not serve on the same
same Committee again. Committee again.
How would you characterize NACEPT? (Please answer each part)
a. a An effective means for the Agency to collect a An ineffective means for the Agency to collect
advice advice
b. Q An efficient means for the Agency to collect Q An inefficient means for the Agency to collect
advice advice
c. a I would serve on NACEPT again. a I would not serve on NACEPT again.
Please provide your assessment about the contribution of NACEPT to EPA on:
a. Policy Q Excellent Q Good Q Fair Q Poor Q No Opinion
b. Management Q Excellent QGood Q Fair Q Poor Q No Opinion
c. Outreach Q Excellent Q Good Q Fair Q Poor Q No Opinion
d. Technology Q Excellent Q Good Q Fair a Poor Q No Opinion
NACEPT: Past and Future Appendix H-2
-------
Note:
"Committee" refers to the Committee identified at the
top of page 1 .
"Council" refers to NACETT or NACEPT.
v
QJ
OC
on
o
C/3
V
DC
CJ
< w
** u
z z
o
t
oc
o
V)
5
0£
O
LM
c/3
a
£
0
Z
^
a
o
c
4-1
o
Z
0
Q
.c
C3
"5.
a.
c
Z
COMMITTEE CHARGE, PROCESSES, & PROCEDURES
1. The Committee was provided with adequate direction on its
purpose and goals.
2. The Committee understood its purpose.
3. The Committee membership was balanced in terms of the
points of view represented.
4. The Committee was able to determine its own direction and
lines of inquiry.
5. As a Committee member, 1 had access to the information that 1
needed to make an informed decision on the issues.
6. When 1 contacted the EPA program office or Office of
Cooperative Environmental Management to request
background information or other support, the support 1
received was:
a. timely
b. useful
c. thorough
7. There was a productive dialogue between appropriate Agency
managers and the Committee.
8. Communication among Committee members was effective.
9. Differing opinions within the Committee were considered
during Committee discussions.
1 0. Communication between the Committee and the Council was
effective.
1 1 . The frequency and schedule of meetings was sufficient for the
Committee to achieve its purpose.
12. Sufficient notice of meeting times and locations was given to
Committee members to allow adequate preparation for the
meetings.
13. Background materials (e.g., agendas, issue papers) were
effective in helping me to prepare for the meeting.
14. Meetings were well-planned and structured to achieve the
Committee's goals.
15. Travel authorization and vouchers were received in a timely
manner and met my needs.
D
n
D
n
n
n
n
n
n
n
D
n
n
D
n
n
n
n
D
D
n
D
n
n
n
n
n
n
n
n
n
D
D
n
D
n
D
n
n
n
n
D
n
n
n
n
n
n
D
n
D
D
D
D
D
D
D
D
D
D
D
D
D
D
n
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
a
a
D
a
D
a
a
a
n
a
D
D
n
n
n
D
D
a
a
a
a
a
a
D
D
D
a
D
D
D
a
a
a
a
a
D
a
n
a
a
n
a
D
D
n
a
n
NACEPT: Past and Future Appendix H-3
-------
Note:
> "Committee" refers to the Committee identified at the
top of page 1.
" "Council" refers to NA CETT or NA CEPT.
16. The Committee had adequate time to address all relevant
issues.
17. The Committee's recommendations or advice were developed
In a timely manner.
1
DC
<
.i1
"ob
c
o
L.
35
P
P
-I
0
L»
w>
<
n
D
* «
feS
< £
L. vs
|a
'Z o
Z 2
D
D
QJ
£
DC
a
V,
5
n
a
Strongly Disagree
n
a
a
^
-------
Note:
> "Committee" refers to the Committee identified at the top of page
t
"Council" refers to NACETT or NACEPT.
31. I believe that the Agency has taken action consistent with the
Committee's recommendations or advice.
32. I believe that the Agency has taken action as a direct result of
the Committee's recommendations or advice.
33. Based on the Agency's consideration and use of this
Committee's recommendations or advice, this Committee
provided a positive and worthwhile contribution to Agency
decision-making.
34. I believe this Committee's recommendations or advice had an
influence beyond the Agency's decision-making process (e.g.,
at the state and local level).
t
an
»»
Bfi
C
I
Cfl
a
a
a
a
(U
1*
OO
D
D
D
D
ttJ 4^
JT M
< -J
&* 2?
£ Q
4J O
D
D
a
D
o
L.
or
S3
5
a
a
a
a
o
o
M
cs
Cfl
C
>.
M
C
a
en
a
a
a
D
=
i
X
j^.
"Z.
O
C
a
a
a
a
*
o
C
X
^s
Z
o
C
a
a
a
a
|
V
C.
<
o
2
a
a
D
D
35. Please characterize your assessment of the impact of this Committee's recommendations and advice on
the Agency's decision-making and actions.
(Continue on back if necessary)
36. What did you value most about your participation on this Committee?
Continue on back if necessary)
ASSESSMENT OF NACEPT
Note: The following questions ask for your assessment of NACETT/NACEPT as whole (i.e., not
limited to your experience on this Committee).
If you have answered this section on a previous Committee survey, you need not complete
this section again please check this box a
37. NACEPT is an effective way for EPA to collect stakeholder
input for the Agency's decision-making process.
38. NACEPT provides EPA decision-makers with valuable input
from a broad range of interested parties.
39. NACEPT provides timely input to EPA to identify and address
key issues and challenges facing the Agency.
a
D
a
D
D
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
D
a
a
a
40. As a result .of my participation on NACEPT, 1 have become involved in other Agency decision-making
processes. ; a Yes a No a Not Applicable
41. Please use this space to share with the Study team any other comments about this Committee or
NACEPT.
(Continue on back if necessary)
NACEPT: Past and Future Appendix H-5
-------
BACKGROUND INFORMATION (OPTIONAL)
A. Gender:
Q Male
Q Female
B. Race/Ethnicity: a Native American/Alaskan Native
Q Asian or Pacific Islander
Q Caucasian (White), non-Hispanic
Q Hispanic
C3 African American
Q Other
C. Age at time of Committee membership: Q 35 years old or younger
Q 36-45 years old
Q 46-55 years old
Q 56-65 years old
Q 66 years old or older
FOLLOW-UP INTERVIEW (VOLUNTARY)
As part of the NACEPT 10-Year Anniversary Study, case studies of selected committees will be
developed to further examine the impact of NACEPT on Agency policy, management, outreach, and
technology as well as to identify opportunities for improving the operations and influence of NACEPT
and its committees. As part of these case studies, in-depth interviews (approximately 30 minutes in
length) will be conducted over the next two months with NACEPT members, Designated Federal
Officers, EPA senior managers, staff from relevant EPA program offices, and external stakeholders.
If you are willing to be interviewed as part of these case studies, please indicate so below.
NACEPT: Past and Future Appendix H-6
-------
Appendix I Study Survey Background Materials
Background materials on relevant points of FACA legislation, NACETT/NACEPT history, and
the history of the specific committees on which the respondent served were included in the
survey mailing to refresh respondent's recollection of their NACEPT experience. All
respondents received the same FACA and NACETT/NACEPT history background as provided
below. In addition, the respondent received only the committee report(s) on which the he/she
served. All committee reports followed the same format and the report for the IRM Task Force
is included here as an example.
Key Points of the Federal Advisory Committee Act
The Federal Advisory Committee Act (FACA) was passed by Congress in 1972. It provides for a
larger citizen voice in the affairs of the Federal Government through invited committee member
and open public participation in advisory groups. These groups are frequently a useful and
beneficial means of providing expert advice, ideas, and diverse opinions to the federal
government.
FACA requires that such citizen advisory groups be:
chartered;
balanced and diverse in terms of perspective, professional qualifications, and experience;
and
established when "determined to be essential" and disestablished upon completion of
work.
Further, the business of these groups is to be:
announced in the Federal Register prior to the meeting;
open to public attendance and written and/or oral comment as agency guidelines permit;
and
conducted promptly.
NACEPT: Past and Future Appendix 1-1
-------
NACETT/NACEPT Background
In 1988, EPA established the National Council for Environmental Technology Transfer
(NACETT) to provide advice and counsel to the Administrator on technology transfer issues
associated with the management of environmental problems. As the Council evolved,
NACETT's role expanded beyond the initial technology transfer charge to a broader
environmental policy formulation. As a result, the Council was renamed the National Advisory
Council for Environmental Policy and Technology (NACEPT) in September, 1990. Celebrating
its 10-year anniversary, NACEPT has experienced the following evolution in focus:
1988 NACETT established to provide advice and counsel on environmental technology
transfer issues.
1990 NACETT renamed to NACEPT to address broader environmental policy issues and
shifted its primary focus to address cross-cutting major issues related to trade and
the environment and pollution prevention.
1994 NACEPT's charter is expanded again addressing new areas such as environmental
technology development and transfer, ecosystems management and environmental
sustainability, community empowerment, global environmental management, non-
federal resources directed at solving environmental economic problems, benefits of
technology transfer, and alternative approaches to environmental protection.
Today NACEPT's mission is to help EPA to improve implementation of environmental
programs by fostering more effective use of the resources of all public and private
institutions involved in environmental management.
NACEPT is organized into three interactive committee structures:.
Standing Committees are the primary operating units for NACEPT, created and charged
to address specific issues and concerns related to environmental policy and technology.
The Executive Committee, comprised of the Chair and Vice-Chair of the Council as well
as the Chair and Co-Chair of each standing committee, provides for overall planning for
the Council and for coordinating activity among committees.
The Council operates as a steering committee, coordinating, overseeing, and reviewing
the work of all standing committees. It is the Council1 s responsibility to present
recommendations from NACEPT to the EPA Administrator and Deputy Administrator.
The Office of Cooperative Environmental Management (OCEM) is responsible for the
management and oversight of NACEPT. In addition, a few NACEPT standing committees are
managed by EPA program offices under the oversight of OCEM.
NACEPT: Past and Future Appendix 1-2
-------
Information Resources
Management (IRM)
Strategic Planning
Task Force
1994
* Established to provide expert input on how to prioritize the Agency's information resource
management budget in support of the Agency's strategic vision, the integration of
information, and working relationships with external partners.
* Worked closely with EPA's Office of Information Resources Management to conduct an
intense six-month study of key IRM strategic issues.
* Published one report:
> "Using Information Strategically to Protect Human Health and the Environment:
Recommendations for Comprehensive Information Resources Management: Report of
the IRM Strategic Planning Task Force" (August 1994).
A. Why Established
Integrated information provides the basis for a better understanding of environmental issues and is a
key enabler for comprehensive approaches to environmental protection. Over the past several years,
EPA has been implementing more comprehensive approaches for protecting human health and the
environment (e.g., ecosystem protection, environmental justice). These approaches, in turn, require
a new integrated approach to managing EPA's information resources.
Specifically, the Agency was interested in receiving input on how to prioritize its budget for
information resource management (IRM). As a result, the Information Resources Management
Strategic Task Force was established to gather such input.
*
B. Description of Charge
The Information Resources Management (IRM) Strategic Planning Task Force was charged with
providing recommendations on the key IRM strategic issues and IRM capabilities required by the
Agency. The Task Force was asked to focus particularly on how to prioritize the Agenyc's IRM
budget in support of the Agency's strategic vision, the integration of information, and improving the
working relationships with external partners.
NACEPT: Past and Future Appendix 1-3
-------
jpj_ Pgtesjoi ^Activity, j^nd Mode o|. Qperation
The Task Force included members from five states, one local government, three public and
environmental interest groups, two Federal agencies, and one academic institution. The Task Force
was created in January 1994 and held its seven meetings during its one-year existence.
The standing committee's Designated Federal Official (DFO) worked very closely with EPA's
Office of Information Resources Management (OIRM) to delineate the issues that the standing
committee would consider. The standing committee spent six months studying key IRM strategic
issues and then focused on developing its report. EPA staff worked closely with the standing
committee throughout its active life to ensure regular and meaningful Agency-standing committee
collaboration.
P. Recom men dation s and Reports ______________________________________________________
The Task Force published two reports: a set of interim recommendations (March 1994), and a set of
final recommendations (August 1994). For the purposes of this report, only the final
recommendations are presented because they so closely resembled the interim recommendations.
"Using Information Strategically to Protect Human Health and the
Environment: Recommendations for Comprehensive Information
Resources Management: Report of the IRM Strategic Planning Task
Force," August 1994 ;
In this report, the Task Force expressed its belief that the management of EPA's information resources must be aligned
to support the mission of the Agency EPA is in the midst of a profound shift from a media-by-media approach to a
more comprehensive approach to protecting human health and the environment. This new comprehensive approach
includes the following guiding principles:
*
* Ecosystem Protection;
* Environmental J usrice;
* Pollution Prevention;
Strong Science and Data;
Partnerships;
* Reinventing EPA Management; and
* Environmental Accountability.
Implementing these principles will fundamentally alter the Agency's approach and requite new thinking in many areas,
including the management of its information resources. In addition, as the Agency realigns its strategic directions, it is
also challenged by new legislative mandates and Executive Office directions, including the Government Performance
and Results Act (GPRA), The National Performance Review (NPR), and the Pollution Prevention Act (PPA).
Ultimately, EPA's ability to fulfill its mission depends upon how it manages its resources, including information. EPA
historically has managed its information resources in terms of its single-media programs, such as "Air" and "Water-"
This heritage has resulted in a fragmented approach to managing the Agency's information. In addition, many
NACEPT: Past and Future Appendix 1-4
-------
programs do not have sufficient data to measure their progress toward achieving their programmatic goals, and EPA
has not identified and does not collect adequate data to measure environmental quality or trends in environmental
quality.
The Task Force concluded that the Agency's current approach to collection and management of information will not
support the requirements of the GPRA, NPR, PPA, or the Agency's comprehensive approach to environmental
protection, and that EPA will fail to implement its guiding principles unless it moves to a more comprehensive
approach to managing the Agency's information resources.
Specifically, the Task Force made the following four recommendations in its report: EPA Must Use Information
Strategical}) to Achieve the Agency's Mission. Information must be viewed and managed as a fundamental, corporate asset to
move beyond the fragmented use ot information resources. The Agency must realize that information provides the
critical link to integrate programs, empower stakeholders to accurately identify, manage, and prevent environmental
problems, and promote environmental successes. There are several elements to this recommendation, including:
use information strategically to protect human health and the environment;
manage information as an Agency asset; and
manage information as an essential element of programs.
EPA Must Active!)' Use Information to Empower its Partners. Information is a powerful asset. It is one asset that all partners
can share without depleting the asset, and which gains in value as it is used. These attributes make information a critical
asset in partnership building. The establishment of information-based partnerships is one key way information can be
used strategically to protect human health and the environment. Environmental issues can be better defined and more
effectively addressed through partnerships with: local, state, tnbal, and foreign governments; other Federal agencies;
educational, environmental; and community-based organizations; industries; and individuals. There are several elements
to this recommendation, including:
aggressively provide information to the public on environmental issues; and
aggressively pursue information-based partnerships with co-implementors and stakeholders
EPA Must Establish an Integrated Information Infrastructure to Achieve a Comprehensive Approach to Environmental Protection.
Although EPA has begun to implement environmental initiatives in a manner that links and refocuses its traditional
single-media programs, the Agency's investment in and use of its information infrastructure does not yet reflect or
support this change. Instead, the existing infrastructure mirrors the Agency's traditional single-media approach. The
infrastructure comprises a senes of "stovepipe" information systems and databases that were designed solely to support
specific media programs and not to exchange or link information across programs. An integrated information
infrastructure with standardized, accurate information that spans the Agency's organizations and its partners is critical
to implementation of EPA's guiding principles. There are several elements to this recommendation, including:
develop, immediately implement, and enforce data standards;
develop data integration policies and tools;
define data requirements and identify gaps in the data inventor)1, and
reduce the burden on providers of information.
EPA Must Establish a More Effective Organisation for Information Resources Management. The Task Force found that EPA's
existing information resources management structure is fragmented and does not provide sufficient authority to its
senior IRM official to ensure that Agency information needs are met. An appropriate organizational structure must be
created with authority and responsibility clearly aligned to manage the Agency's information resources. The
management of EPA's information resources must be championed at a senior level and receive adequate attention from
all senior managers. There must be a distinct budget for all IRM expenditures to ensure that information resource costs
are accounted for like other corporate assets. Essentially, EPA must change the general culture of its IRM
management. There are several elements to this recommendation, including:
NACEPT: Past and Future Appendix 1-5
-------
establish a Chief Information Officer (CIO) position with mission cnucal responsibilities;
maintain an executive level IR.\1 Steering Committee;
integrate the IRM planning process with the Agency's budget;
resolve the organizational fragmentation; and
strengthen program IRM implementation efforts.
NACEPT: Past and Future Appendix * 1-6
-------
Appendix J Survey Results
All percentages based on 198 respondents unless otherwise noted.
BACKGROUND INFORMATION
In which of the following sectors did you work when appointed to this Committee?
29% Federal, State, or Local Government Agency
31% Corporate, Private, or Self-Employed
1 % Trade Union or Labor Organization
7% Professional or Trade Association
17% College/University or other Academic Research Organization
8% Advocacy or Public Interest Non-Governmental Organization
5% Other
2% Skipped
On how many NACEPT Committees have you served?
60% i 22% 2 8% 3 6%
0%
5 or more
4% Skipped
Gender;
65%
Male
26% Female
Race/Ethnicity:
3% Native American/Alaskan Native
77% Caucasian (White), non-Hispanic
5% African American
9% Skipped
3% Asian or Pacific Islander
1 % Hispanic
1% Other
10% Skipped
Age at time of Committee membership:
6% 35 years old or younger
33% 36-45 years old
39% 46-55 vears old
13%
1%
56-65 years old
66 years old or older
Which of the following best characterizes your attendance at Committee meetings?
1% Never attended 33% Attended more than half
1% Attended less than half 61% Always attended
4% Attended about half 1% Skipped
NACEPT: Past and Future Appendix J-l
-------
OVERALL ASSESSMENT OF COMMITTEE AND NACEPT EXPERIENCE
1. How would you characterize your experience serving on this Committee?
a. 77% Worth my time and effort
16% Not worth my time and effort
7% Skipped
b. 37% Committee's work had a noticeable effect on Agency decision-making.
29% Committee did not have a noticeable effect on Agency decision-making.
29% Do Not Know
5% Skipped
c. 75% In hindsight, I would serve on the same Committee again.
16% In hindsight, I would not serve on the same Committee again.
9% Skipped
3 How would you characterize NACEPT?
a. 77% An effective means for the Agency to collect advice.
14% An ineffective means for the Agency to collect advice.
7% Skipped
b. 62% An efficient means for the Agency to collect advice.
23% An inefficient means for the Agency to collect advice.
14% Skipped
c. 77% I would serve on NACEPT again.
12% I would not serve on NACEPT again.
11% Skipped
4. Please provide your assessment about the contribution of NACEPT to EPA
a. Policy
b. Management
c. Outreach
d. Technology
Excellent
14%
^Excellent
8%
Excellent
14%
ExceUent
10%
Good
38%
Good
26%
Good
38%
Good
24%
Fair
23%
Fair
29%
Fair
23%
Fair
21%
Poor
8%
Poor
13%
Poor
9%
Poor
16%
No Opinion
10%
No Opinion
19%
No Opinion
11%
No Opinion
23%
Skipped
7%
Skipped
6%
Skipped
6%
Skipped
6%
NACEPT: Past and Future Appendix J-2
-------
SA= Strongly Agree
A = Aeree
NA/ND = Neither Agree nor Disagree
D = Disagree
SD = Strongly Disagree
DNR - Do Not Recall
DNK = Do Not Know
NA = Not Applicable
S = Skipped
T = Total
SA
A
NA/
ND
D
SD
D
N
K
COMMITTEE CHARGE, PROCESSES, & PROCEDURES
.
1. The Committee was provided with adequate
direction on its purpose and goals.
2. The Committee understood its purpose.
3. The Committee membership was balanced in
terms of the points of view represented.
4. The Committee was able to determine its own
direction and lines of inquiry.
5. As a Committee member, I had access to the
information that I needed to make an informed
decision on the issues.
6. When I contacted the EPA program office or
Office of Cooperative Environmental
Management to request background
information or other support, the support I
received was: a. timely
b. useful
c. thorough
7. There was a productive dialogue between
appropriate Agency managers and the
Committee.
8. Communication among Committee members
was effective.
9. Differing opinions within the Committee were
considered during Committee discussions.
10. Communication between the Committee and
the Council was effective.
11. The frequency and schedule of meetings was
sufficient for the Committee to achieve its
purpose.
12. Sufficient notice of meeting times and locations
was given to Committee members to allow
adequate preparation for the meetings.
15%
14%
19%
15%
21%
21%
17%
16%
27%
19%
30%
7%
10%
22%
49%
47%
57%
48%
46%
43%
40%
34%
40%
52%
59%
24%
51%
55%
16%
17%
8%
14%
16%
5%
13%
17%
13%
15%
6%
22%
16%
9%
13%
16%
12%
12%
10%
5%
4%
5%
12%
10%
2%
12%
15%
10%
4%
3%
2%
9%
4%
2%
3%
3%
4%
3%
1%
6%
5%
2%
0%
0%
1%
0%
1%
3%
2%
2%
0%
0%
1%
2%
0%
0%
D
N
K
2%
1%
1%
1%
1%
2%
3%
3%
1%
0%
1%
22%
2%
1%
NA
0%
0%
0%
0%
1%
18%
18%
18%
1%
0%
0%
4%
1%
1%
S
2%
2»
2%
1%
2%
2<*
2%
2%
2%
2%
1%
2%
2%
1%
T
198
198
198
198
198
198
198
198
198
198
198
198
198
198
NACEPT: Past and Future Appendix J-3
-------
13. Background materials (e.g., agendas, issue
papers) were effective in helping me to prepare
for the meeting.
14. Meetings were well-planned and structured to
achieve the Committee's goals.
15. Travel authorization and vouchers were
received in a timely manner and met my needs.
16. The Committee had adequate time to address
all relevant issues.
17. The Committee's recommendations or advice
were developed in a timely manner.
SA
17%
18%
18%
8%
10%
A
51*
37%
39%
33%
42%
NA/
ND
15%
20%
8%
23%
20%
D
13%
15%
4%
24%
17%
SD
2%
8%
2%
9%
5%
D
N
R
1%
\%
1%
1%
2%
D
N
K
1%
1%
2%
1%
2%
NA
0%
0%
26%
1%
2%
s
2c/c
\%
2%
2%
i%
T
198
198
198
198
198
18. Narrative Response
COMMITTEE RECOMMENDATIONS AND ADVICE
19. The recommendations or advice developed by
the Committee were clear and specific.
20. The recommendations or advice that the
Committee provided fulfilled the Committee's
purpose/charge.
21. In developing its recommendations or advice,
the Committee discussed the practicality of
implementation.
22. The Committee prioritized recommendations
or advice in order of importance for
implementation.
10%
12%
12%
8%
42%
40%
46%
29%
22%
19%
16%
23%
13%
12%
13%
16%
2%
3%
1%
5%
1%
1%
3%
5%
3%
7%
3%
5%
5%
4%
3%
7%
3%
3%
4%
3%
23. The recommendations or advice developed by the Committee were:
3% Overly detailed 59% Appropriate in detail 23% Too vague 16% Skipped
24. In its reports, the Committee provided
sufficient documentation of the basis for its
recommendations or advice.
25. The Committee's report(sj) of its
recommendations or advice were presented in
a clear, easily understood format.
26. The Committee received feedback from EPA
on the Agency's decisions related to the
Committee's recommendations or advice.
8%
12%
5%
35%
42%
22%
24%
20%
22%
12%
9%
17%
3%
2%
13%
2%
2%
2%
7%
5%
8%
7%
6%
8%
3%
4%
3%
198
198
198
198
198
198
198
198
NACEPT: Past and Future Appendix J-4
-------
SA
A
NA/
ND
D
SD
D
N
R
D
N
K
NA
s
T
| IMPACT OF COMMITTEE RECOMMENDATIONS AND ADVICE
27. Narrative Responses
28. The Committee's recommendations or advice
introduced new perspectives to the Agency's
consideration of the issues.
29. The recommendations or advice offered by this
Committee addressed Agency decision-makers'
needs.
30. The recommendations or advice offered by this
Committee were timely to Agency decision-
makers' needs.
31. I believe that the Agency has taken action
consistent with the Committee's
recommendations or advice.
32. I believe that the Agency has taken action as a
direct result o/the Committee's
recommendations or advice.
33. Based on the Agency's consideration and use of
this Committee's recommendations or advice,
this Committee provided a positive and
worthwhile contribution to Agency decision-
making.
34. I believe this Committee's recommendations or
advice had an influence beyond the Agency's
decision-making process (e.g., at the state and
local level).
10%
11%
12%
3%
5%
7%
6%
47%
40%
42%
22%
22%
32%
24%
14%
21%
19%
22%
22%
17%
19%
10%
5%
4%
12%
10%
10%
12%
4%
3%
3%.
10%
10%
8%
6%
3%
2%
3%
1%
0%
0%
1%
7%
11%
10%
23%
24%
20%
22%
4%
5%
4%
7%
7%
5%
7%
3%
3%
4%
2%
2%
2%
5%
198
198
198
198
198
198
198
35. Narrative response
ASSESSMENT OF NACEPT
Note: The following questions ask for your assessment of NACETT/NACEPT as whole (i.e., not limited to your experience
on this Committee). 34 surveys did not have responses for any questions in this section as the respondents had completed
this portion of the survey on another survey.
36. NACEPT is an effective way for EPA to collect
stakeholder input for the Agency's decision-
making process.
37. NACEPT provides EPA decision-makers with
valuable input from a broad range of
interested parties.
38. NACEPT provides timely input to EPA to
identify and address key issues and challenges
facing the Agency.
15%
23%
17%
52%
53%
46%
16%
8%
16%
9%
7%
9%
3%
2%
2%
0%
0%
0%
4%
7%
10%
0%
0%
0%
164
163
163
NACEPT: Past and Future Appendix J-5
-------
39.
As a
result
of my
participation
on NACEPT,
I have
become involved
in other Agency decision-making
processes.
25%
Yes
49%
No
25%
Not
Applicable
157
Follow-up Interview
58% Yes, I am willing to be interviewed by telephone to discuss my experience with NACEPT.
42% Other
NACEPT: Past and Future Appendix J-6
-------
Appendix K Technical Details of the Interview Methodology
Purpose
As part of this study, interviews were conducted with four groups:
NACEPT members;
« Agency officials involved with specific standing committees;
» Past and current Directors of the Office of Cooperative Environmental
Management; and
* Externalstakeholders
Interviews with NACEPT members and Agency officials focused on specific standing committee
activities and results. Interviews with the Directors of OCEM were intended to provide context
on changes in NACEPT's organization and procedures, as well as changes in EPA's support to
the Council, such as internal management improvement efforts. Interviews with external
stakeholders were intended to provide an outside perspective on the effectiveness of the advisory
process and the influence of NACEPT beyond EPA.
The purpose of the interview phase was to validate and enrich the survey results and to gather
additional information. Interviews were conducted with a sample of survey respondents and
officials from select Agency offices.1 Interviews offer greater flexibility than surveys to pursue
issues specific to the respondent's experience including issues not addressed in a written survey.
In addition, interviews allow for detailed follow-up to questions which is particularly relevant to
a study such as this, which seeks not just to identify issues but also recommendations for change.
In this way, interviews can both confirm and expand the data collected through the survey.
Interview Selection Process and Mix
A sampling plan was developed for the interview phase. Due to the variety of experience among
the numerous standing committees, the sampling plan identified a sub-set of standing
committees for which a series of interviews would take place. (Selecting a limited number of
standing committees for follow-up interviews allowed interviews to be conducted with a greater
number standing committee members thus providing a more representative set of perspectives on
that standing committee.) Six standing committees were selected to provide a range of
experiences based on the following specific criteria:
1 A small number of interviews were also conducted with external stakeholders. Because of the difficultly in
securing the participation of relevant external stakeholders, methodological concerns about representativeness, and
the lack of new information gathered, these interviews were not included in the final data set and did not
meaningfully contribute to the findings of this study.
NACEPT: Past and Future Appendix K-I
-------
> topic area of the standing committee (information management /information
technology, general agency policy or management, program or media-
specific);
» average response scores on the written survey (most positive, average, and
least positive); and
» a response rate to the written survey greater than 40%.
Specific interviewees were selected based on their indicated willingness to be interviewed (from
the written survey) and on their individual survey responses (ranging from negative to positive)
to capture a fair and complete set of opinions.2 Relevant external stakeholders and Agency
officials were identified from standing committee records such as meeting participant lists and
those judged most knowledgeable about the workings and results of the committee were selected.
In addition, three past and the current director of OCEM were interviewed to provide additional
content on changes in EPA and OCEM policy toward initiating and supporting committees.
Interview Protocol
An interview protocol was developed to ensure that all interviews would be conducted in a
standard way. Based on topics for investigation provided by the Study team, the survey
contractor developed a list of potential questions to be used throughout the Study interviews (see
Appendix L).
Using the survey results, a customized interview script was developed. This script included
selected of interview questions based upon the interviewee's survey responses. (For Agency staff
interviewees, standing committee background documents were sent in advance of the interview.
These reports were similar to the report presented in Appendix I.) In addition, interviewers
raised ad hoc questions to clarify responses to pre-selected interview questions. Interviews were
conducted by phone and in-person and lasted approximately 30 miYiutes. All interview
information was kept anonymous to the Study team by the contractor to promote candor.
Interview Data Analysis
To promote objectivity in the analysis of the interview data, each interviewer drafted a summary
of key points immediately after the interview (while the conversation was fresh in his or her
mind). A second analyst reviewed the interview notes and interview summary to ensure
consistency and completeness of the summary.
2 This approach posed potential concerns that voluntary self-identification for follow-up interviews might result
only in the most positive committee members being interviewed. Comparison of those willing to be interviewed and
those not did not, however, show significant differences in overall views regarding NACEPT.
NACEPT: Past and Future Appendix K-2
-------
Appendix L Interview Questions
Note: The following is a list of the questions used to prepare the scripts for the interviews of
NACEPT members. Since each script was customized based on the interviewee's survey
responses, not all questions were asked of all interviewees. Consequently, some of the questions
in this list assume that the interviewee holds a specific opinion on the question topic. Such
questions were only asked as a follow-up to the interviewee's survey response. For that purpose,
the survey question to which each interview question in this, list is denoted below in parenthesis
following the question. Additional questions may have been asked at the discretion of the
interviewer based on issues raised in the interview.
Interview questions for Agency officials involved with specific standing committees were drawn
from a similar list, which more narrowly focused on the products and impact of the standing
committees. These questions primarily addressed the nature, specificity, timeliness, and
relevance of the standing committees' recommendation, as well as the value of dialogue with the
committee during its deliberations. In addition, these Agency officials were asked about the
subsequent effect of the recommendations on Agency actions, including specific examples of
Agency decisions and policies influenced by NACEPT.
Interview questions addressed to the past and current Directors of OCEM focused on changes in
the operations and focus of NACEPT over time, as well as changes in OCEM's support to
NACEPT and its standing committees. These interviews provided additional context to
understand the composition and internal operations of NACEPT as it evolved over the last 10
years.
Standing Committee Charge, Processes, and Procedures
In what way did the standing committee lack direction on its purpose and goals? (1)
Why did the standing committee not understand its purpose? What should have been done to
help foster such an understanding? (2)
Were there any groups or views not represented on the standing committee that you would have
expected or thought should have been? (3)
*
How could the standing committee membership have been better balanced in terms of the points
of view represented? (3)
How did the standing committee decide its own direction and line of inquiry? Was the standing
committee encouraged to do so? (4)
NACEPT: Past and Future Appendix«L-l
-------
What factors impeded the standing committee's ability to determine its own direction/lines of
inquiry? (4)
How would you characterize the information made available to you as a standing committee
member - too little/too much? Well organized? Concise? Overly technical? (5)
Why did you find OCEM support to be less than thorough/timely/useful? Can you provide
examples? What could have been done differently? (6c)
How much contact did you have with EPA program offices? With OCEM? With the Council?
Were these groups responsive to your needs and requests? (6-7)
Would you characterize the standing committee's interactions with EPA program office staff: as
primarily with technical EPA staff, or senior managers with decision-making authority? (7)
What aspects of communication among standing committee members were ineffective? What
could have been done to improve communications? (8)
Would the standing committee's work have benefitted from greater opportunity for
communication among members outside of official meetings, such as conference calls or group
e-mails? Do you think that most standing committee members would have used such
opportunities if available? (8)
What was the amount and nature of communication between the standing committee and the
Council? (10)
Would more communication with the Council have been helpful to the standing committee in
completing its work? (10)
Why do you characterize the communication between the standing'committee and the Council as
ineffective? What could have been done to improve communications? (10)
In hindsight, would you have recommended a different frequency of meetings for your standing
committee? If so, how would the schedule of meetings have been different? (11)
How much notice of meeting times and locations would you consider sufficient to allow
adequate preparation*for the meetings? How much did you actually receive? (12)
How helpful were the background materials for the meetings? What would have made these
materials more helpful to you? Do you think this was true for most of your standing committee
members as well? (13)
How could standing committee meetings have been structured differently to help the standing
committee better fulfill its charter? (14)
NACEPT: Past and Future Appendix«L-2
-------
What aspects of the travel authorization/voucher process did not meet your needs? What could
have been done differently? (15)
Did time constraints affect the quality or thoroughness of the standing committee's work? What
would have been a more appropriate time frame for the standing committee to formulate and
provide advice? What could the standing committee have achieved with more time? What
additional resources or support would have been needed for the standing committee to have
completed its work sooner? (16)
Do you think that other standing committee members shared your sentiments regarding the
operation and function of your standing committee? (1-17)
Are there any recommendations to the Office of Cooperative Environmental Management that
you would suggest to improve the operations of future NACEPT standing committees? (17)
If the chairperson of a newly established NACEPT standing committee called you to ask how
he/she could ensure the success of his/her's standing committee's work, what advice would you
give him or her?
Standing Committee Recommendations and Advice
What characteristics of the recommendations or advice provided by the standing committee led
you to state that they did not fulfill the standing committee's charge? (20)
Did the standing committee's recommendations go beyond the standing committee's charter? If
so, in what way? Did the standing committee do this purposely? (20)
What considerations, such as the practicality or specificity, did the standing committee focus on
in developing its recommendations? Were there any considerations that were intentionally not
considered? (19-23)
Why do you believe the standing committee's recommendations were overly detailed/too vague?
What could have been done differently to have avoided this? (23)
How well documented was the basis for the standing committee's recommendations or advice?
Did the standing committee believe that such documentation was not needed? (24)
Did you and the rest of the standing committee expect feedback on your recommendations and
advice? What feedback did the standing committee receive from the Agency on its
recommendations and advice? What was the form of this feedback (e.g., in a letter from the
Administrator or Deputy Administrator, through the DFO)? When was this feedback received?
(26)
NACEPT: Past and Future Appendix-L-3
-------
In what form would you like the feedback regarding the impact of the standing committee's
recommendations and advice on the Agency's decision-making and actions? How much? How
frequently? How soon after the recommendations are provided? (26)
What specifically hindered the quality and timeliness of developing the recommendations and
advice? (19-26)
How did your actual experience on this standing committee differ from your expectations?
Impact of the Standing Committee's Recommendations and Advice
Do you believe that your standing committee's recommendations and advice served to confirm or
reaffirm the Agency's established positions, or did the advice provide new perspectives and
directions? (28)
What could have been done to help the standing committee more effectively introduce new
perspectives to the Agency's consideration of the issues? (28)
How could the standing committee have improved the quality of its recommendations? (29)
Why do you feel the standing committee's recommendations did/did not address the Agency
decision-makers' needs? What should the recommendations have contained to better meet the
Agency decision-makers' needs? (29)
Why do you believe the standing committee's recommendations were/were not timely to meet
the Agency's needs? When should the recommendations have been submitted to meet the
Agency's needs? (30)
Why do you believe that the Agency has/has not taken action consistent with the standing
committee's recommendations? (31)
Why do you believe that the Agency has/has not taken action as a direct result of the standing
committee's recommendations? (32)
What was the most significant contribution of the standing committee to EPA and why? (33)
s
Why do you believe that the standing committee's recommendations have had an influence
beyond the Agency's decision-making process? Can you provide specific examples? (34)
Do you think that it would have been useful for your standing committee to have continued so
that it could have monitored and provided advice on the implementation of its recommendations?
NACEPT: Past and Future Appendix«L-4
-------
Assessment of NACEPT
In your opinion, why does EPA not always implement NACEPT1 s recommendations? (37-40)
Why do you think that NACEPT does not provide timely input to EPA to identify and address
key issues and challenges facing the Agency? (39)
Why do you consider NACEPT an efficient/inefficient means for the Agency to collect advice?
What changes to the process would improve NACEPT's efficiency? (3b)
Why would you not serve on NACEPT again? What would have to be different for you to
reconsider? (3c - Overall Assessment)
How could NACEPT provide more value to the Agency? Specifically, what could NACEPT do
differently to improve its contributions to issues related to EPA policy, management, outreach,
and technology? (4a-d - Overall Assessment)
Questions for Agency Staff and External Stakeholders
How much interaction did you have with the standing committee? What was the nature of that
interaction made a presentation, provided technical expertise, attended meeting as an
observer, addressed ad hoc inquiries, etc.?
Were you familiar with the standing committee's charge and charter? If yes, did the
recommendations accomplish the goals and purpose of the standing committee?
Did the standing committee provide proper background information and documentation to
substantiate the final recommendations?
Did the standing committee present the final recommendations in'a format which facilitated
understanding the recommendations and implementing them?
Did the recommendations specify who should implement them and by when?
How did this standing committee's recommendations and advice impact the Agency's decision-
making and actions?^
Has the Agency taken action consistent with the standing committee's recommendations or
advice?
Has the Agency taken action as a direct result o/the standing committee's recommendations or
advice? Can you provide specific examples?
NACEPT: Past and Future Appendix«L-5
-------
Based on your knowledge of the standing committee's recommendations or advice, do you think
that this standing committee provided a positive and worthwhile contribution to Agency
decision-making?
Did the standing committee's recommendations or advice have an influence beyond the
Agency's decision-making process (e.g., at the state and local level)? Can you provide specific
examples?
Did you provide any feedback directly to the standing committee regarding the recommendations
or their implementation? If so, when? Did you provide any feedback directly to the Council
regarding the recommendations or their implementation? If so, when?
Based on your experience, is NACEPT an effective way for EPA to collect stakeholder input for
the Agency's decision-making process?
Based on your experience, does NACEPT provide timely input to EPA?
How can NACEPT provide more value to the Agency's decision-making process?
As a result of your experience with NACEPT, are there other topics or issues currently facing
the Agency that might merit the attention of NACEPT?
NACEPT: Past and Future Appendix«L-6
-------
Appendix M Chronology and History Report
The NACEPT Chronology and History Report is available as a separate document.
NACEPT: Past and Future Appendix»M-l
------- |