May 2008
Evaluating the
Effectiveness of the
EPA Pollution
Prevention Information
Network Grant Program
Promoting Environmental Results
t »
Through Evaluation
-------
Acknowledgements
This evaluation was performed by Abt Associates Inc. (Abt), under contract to Industrial
Economics, Incorporated (IEC) for EPA's Office of Policy, Economics and Innovation (OPEI)
under Contract EP-W-04-023 between EPA and lEc. Abt gratefully acknowledges the input
and guidance provided by EPA staff Beth Anderson, Office of Pollution Prevention and
Toxics, and Yvonne Watson, OPEI, who served as the evaluation advisor, throughout the
design and implementation of the evaluation. Additional thanks go to Jean Waters of P2RIC,
Andy Bray of NEWMOA, and Elizabeth Bird of Peaks to Prairies for their contributions
throughout the evaluation process. Abt also appreciates the time made available by all P2Rx
Center staff members to assist the Evaluation Team and provide critical information and
perspectives on P2Rx that would not have otherwise been available.
This report was developed under the Program Evaluation Competition sponsored by OPEI.
To access copies of this or other EPA program evaluations, please go to EPA's Evaluation
Support Division's website at http://www.epa.gov/evaluate.
-------
Table of Contents
Acronyms iii
Executive Summary iv
1 Introduction 1
1.1 Program Description 1
1.2 Purpose and Evaluation Questions 2
1.3 Evaluation Audience 3
1.4 Program Logic Model 3
1.5 Organization of Report 6
2 Methods 8
2.1 Data Collection Methods 8
2.2 Data Analysis Methods 14
3 Findings 17
3.1 Center Operation Models 18
3.2 P2Rx Information Users 26
3.3 Usefulness of P2Rx Products 35
3.4 Outreach to Customers 42
3.5 Attribution of Outcomes 48
3.6 Performance Measurement 57
4 Conclusions and Recommendations 60
4.1 Center Operation Models (Question 1) 60
4.2 P2Rx Information Users (Question 2) 61
4.3 Usefulness of P2Rx Products (Questions 3 and 4) 63
4.4 Adoption of P2 Practices by Businesses (Question 5) 64
4.5 Outreach to Businesses Directly and Through TAPs (Questions 6 and 7) 65
4.6 Performance Measurement (Question 8) 66
Appendix A - Center Logic Models 70
Appendix B - Data Collection Matrix 79
Appendix C - 2007/2008 Behavior Survey Analysis 82
Appendix D - Center-Specific 2007/2008 Behavior Survey Results 92
Appendix E - 2004 Customer Satisfaction Survey Analysis 163
Appendix F- Interview Guide 170
Appendix G - Focus Group Moderator's Guide 176
Appendix H - Center and National P2Rx Documents Reviewed for Analysis 182
Appendix I - Abt Associates Data Analysis Plan 183
List of Tables
Table 1-1: Crosswalk of Evaluation Questions with Logic Model Components 6
Table 2-1: Summary of Data Collection Methods 9
Table 2-2: 2007/2008 Behavior Survey Respondents by Center 10
Table 3-1: Crosswalk of Report Sections and Themes with Evaluation Questions 17
Table 3-2: Selected Web and Activity Measures for Each Center: First Half 2007 20
Table 3-3: Type of Organization (% of responses) 27
Table 3-4: Uses of P2Rx Centers by Interviewed Business Representatives 28
Table 3-5: Business Utilization of P2Rx Web Sites 30
Table 3-6: TAP Utilization of P2Rx Web site 31
Table 3-7: Meetings and Workshops Organized by P2Rx Centers 34
i
-------
Table 3-8: Percent of Active Users Rating Resource as Useful or Very Useful 36
Table 3-9: Interviewed Business Representatives' Reported Needs 38
Table 3-10: Referrers to P2Rx 43
Table 3-11: Center Outreach to Interviewed Businesses 46
Table 3-12: Actions Taken as a Result of P2Rx Information 49
Table 3-13: 2007/2008 Behavior Survey Responses: Improved Awareness/Ability to Provide
Assistance 49
Table 3-14: Pollutant Reduction as a Result of P2Rx Use 50
Table 3-15: Cost Savings to Business- Directly or Through TAPs 51
Table 3-16: Cost Savings to TAPs 51
Table 3-17: Interviewed Businesses' Reported Outcomes Attributable to P2Rx 55
Table 4-1: Recommendations for Existing Measures 67
Table 4-2: Recommended New Measures 68
List of Figures
Figure 1-1: Generic P2Rx Information Dissemination Logic Model 5
Figure 3-1: P2Rx Centers' Clients Served by Type of Assistance 29
Figure 3-2: P2Rx Centers' Direct Contact with Clients, January 2006 - June 2007 33
Figure 3-3: Meetings and Workshops Organized by P2Rx Centers 34
-------
Acronyms
DOE Department of Energy
EPA Environmental Protection Agency
ES&H Environment, safety, and health
GLNPO Great Lakes National Program Office
GLRPPR Great Lakes Regional Pollution Prevention Roundtable
GPRA Government Performance and Results Act
ICR Information Collection Request
lEc Industrial Economics, Incorporated
ISO International Organization for Standardization
MEP Manufacturing Extension Partnership
MPSC Midwest Product Stewardship Council
NEWMOA Northeast Waste Management Officials' Association
OMB Office of Management and Budget
OPEI Office of Policy, Economics, and Innovation
OPPT Office of Pollution Prevention and Toxics
P2 Pollution prevention
P2RIC Pollution Prevention Regional Information Center
P2Rx Pollution Prevention Resource Exchange
PPIN Pollution Prevention Information Network
PPRC Pacific Northwest Pollution Prevention Resource Center
PRA Paperwork Reduction Act
SPCC Spill Prevention, Control, and Countermeasure
TAP Technical assistance program/provider
WRPPN Western Regional Pollution Prevention Network
WRRC Waste Reduction Resource Center
Zero Waste Southwest Network for Zero Waste
-------
Executive Summary
This evaluation focuses on the Pollution Prevention Information Network (PPIN), a grant
program started in 1997 to promote sharing of pollution prevention (P2) information, training,
and technical assistance expertise among states. The eight regional centers receiving these
grants are collectively known as the Pollution Prevention Resource Exchange (P2Rx), although
the centers also receive funding from other sources. P2Rx is managed by EPA's Office of
Pollution Prevention and Toxics, Pollution Prevention Division.
P2Rx is a national network of regional pollution prevention information centers (referred to as
"the centers") dedicated to increasing the adoption of pollution prevention by improving the
dissemination of relevant information. The centers provide pollution prevention information,
networking opportunities, and other services to states, local governments, technical assistance
providers, and businesses. P2Rx receives a total $800,000 in funding through the PPIN grants,
which must be divided among the eight centers.
The purpose of the evaluation is to:
1) Assess the effectiveness of the PPIN Program in promoting the adoption of P2
opportunities.
2) Identify opportunities for creating a more effective interface with customers (both
intermediate customers (i.e., TAPs) and final customers (i.e., businesses and other polluting
entities).
3) Identify current efficiencies in information collection and dissemination as well as future
opportunities for improving efficiency.
4) Determine how P2Rx activities (outputs) can be linked to long-term environmental outcomes
(listed under the goals of the Government Performance and Results Act [GPRA] and the
PPIN's environmental outcomes).
Methods
The Evaluation Team began the evaluation process by developing and refining a set of
evaluation questions to be answered. Abt Associates also developed a logic model to illustrate
the activities, customers, outputs, and outcomes of the entire P2Rx program, based on
individual logic models created by each of the centers. Finally, Abt developed a data analysis
plan to guide the examination of the data collected.
Abt Associates used a variety of qualitative and quantitative data to evaluate the P2Rx centers.
Qualitative information was used to determine the structure and operation of individual P2Rx
centers and the types of P2Rx activities and outputs that are most important and useful to
stakeholders for enabling change. Qualitative data were also used to make observations about
the perceived effectiveness of the P2Rx centers in meeting their stated goals. Quantitative data
were used to examine which P2Rx resources and tools are being used most frequently, and
with what audiences the P2Rx centers are communicating.
Qualitative data sources included:
• Interviews with Center Directors, TAPs, and business representatives from all regions;
• Focus groups with TAPs and business representatives from Regions 9 and 10; and
• Review of national P2Rx and center documents.
IV
-------
Quantitative data sources included:
• National-level surveys, conducted online and in person in 2004 and 2007/2008; and
• Web Measures and Activity data reported twice annually by the centers.
Conclusions and Recommendations
The following is a summary of the conclusions and recommendations of this evaluation. The
results are grouped together by broad themes, as well as by that theme's corresponding
evaluation question(s). The evaluation questions are included in italic text.
Center Operation Models (Evaluation Question 1)
1) What model(s) do the P2Rx centers use to encourage exploration and adoption ofP2
practices?
a) How are centers currently operating and what model(s) are they using to disseminate
information?
b) Which of the centers' model(s) is most efficient in information sharing and/or delivery?
c) When comparing among centers, what opportunities for centers' information delivery to
become more efficient are observed?
d) What opportunities are there for centers to become more efficient in information delivery
by sharing tasks?
The centers use a wide variety of models to fulfill the mission of P2Rx. Variation exists in
organizational structure, physical location (e.g., co-located with a TAP, university, or regulatory
agency), size of staff, availability of resources, sources and use of funding, activity emphasis,
and customer base. With this variation, each center possesses different qualities and meets
their objectives in ways that are both unique and similar, depending on the element. The
evaluation is inconclusive as to which single model may be most efficient but indicates that an
eight-center network is an effective method to reach the network's objectives. P2Rx as an
interactive network does appear to strengthen the abilities of any single provider, but there are
also inherent inequalities. Overall, as an information "wholesaler" with responsibility for
compiling and providing information to more localized entities (e.g., TAPs), a regional model for
the centers does appear to be effective. The existence of a national network makes it possible
for the centers to deliver more and better information to their customers, as the centers can
focus on unique regional needs while tapping into national resources. The centers do have
strong and constructive relationships with organizations and individuals within their region.
Considerable efficiency has been gained over the years in centralizing certain functions in one
or two centers. Successful examples include the case study database supported by Zero
Waste, centralized IT support for certain activities and overall Web site maintenance, the
centralized management of the P2 Programs Database, and of course the library at WRRC.
Web support and programming was cited as an obvious opportunity to centralize so that the
network capitalizes on the strength of certain centers' IT expertise. For the most part, this
seems to be working well and to be enthusiastically supported by center staff. One observation
is that it is not clear if or how a center is compensated for taking on a particularly labor-intensive
task, or if resource limitations could be a disincentive for doing so. Further, there may be an
inherent contradiction in sharing tasks in order to gain efficiencies. For example, if services or
-------
functions are centralized in a particular center (e.g., library services or Web support), that center
will be using a greater portion of their resources supporting that centralized function. The result
may be that they are less able to do other functions, such as providing direct assistance to
businesses, or possibly to measure outcomes.
Recommendations
• P2Rx should continue to work together as a network of distinct programs, and should share
responsibilities where appropriate as well as centralize tasks where appropriate. Continue
to make well-reasoned decisions about what tasks are best suited to be done in a
centralized way. Specifically, there appears to be benefit in centralizing the functions for
collecting case studies, maintaining the P2 Programs Directory, performance measurement,
and some of the coding responsibilities.
• It is an understandable concern that the centers spend considerable time communicating
with one another, an activity that could impinge on the time they have to spend reaching out
to end users. To some extent this is a necessary part of networking. However,
communication should be value-added and done efficiently. Suggestions include the use of
facilitators, limits on the number of conference calls, and setting clear objectives for each
meeting.
• Consider making the PPIN grant process more streamlined and/or award multi-year grants.
This would reduce the time centers spend on writing proposals and allow for longer-range
activity planning.
P2Rx Information Users (Evaluation Question 2)
2) What groups/types of customers access P2Rx information ?
How do the customers vary according to method of outreach by the centers?
• Via the Web site
• Via TAPs
• Via direct contact (e.g., telephone, correspondence, or other communication)
• Via P2Rx workshops, trainings, and regional meetings
The evaluation characterized the groups of customers that access P2Rx information, but was
not able to determine how the groups or types of customers vary according to the method of
outreach by the centers. Some determinations about the customers of P2Rx can be made
based on the data reviewed for the evaluation. Information that would characterize the users of
the Web sites, TAPs, telephone or e-mail correspondence, or face-to-face sessions, either does
not exist or has not been collated.
Through interviews we learned that businesses access P2Rx information primarily via Web
sites, direct contact, or training sessions. Data available from Activity Measures show listserves
and newsletters going out to the largest number of customers, as compared to meetings and
direct contact. Information on the identity of these recipients is kept by some centers, but not
across the network. The same is true for Activity Measures, which show number of meetings
and number of attendees at meetings but does not capture the identity of attendees. Web
VI
-------
measures do not capture the type of user, but do show a significant increase in usage of the
Web site over the years. 2007/2008 Behavior Survey data gave an indication of the types of
users that were using the P2Rx Web sites, and how they were using them. Activity Measures
show that the majority of customers for direct contact are from government, but with a declining
percentage of requests from government entities, and an increasing percentage of business and
industry requests. In contrast, the 2007/2008 Behavior Survey shows more than twice as many
TAPs using the P2Rx Web sites than businesses. Overall, this information is inconclusive in
responding to the evaluation question.
Recommendations
• Use a consistent method of characterizing customers for all of P2Rx. Identification
according to the two groups used in the 2007/8 Behavior Survey is recommended, as it
addresses how the customer uses P2Rx and not just who they are. If an additional level of
detail is desired, affiliation can be used as subcategories.
• Rapid Response (and other telephone or e-mail-based on-demand services) offers a
valuable opportunity to capture, tabulate and analyze information on types of customers and
types of request. Centers should collect and analyze information on the identity of these
customers. Some centers already collect this information, but it does not appear to be
analyzed. Centers that do not currently capture this information should consider doing so.
• Characterize and analyze the identity of conference attendees. While the number of
attendees is captured for the P2Rx Activity Measures, the identity of these participants is not
examined in a comprehensive way.
• Make the Web sites more interactive so that centers can gather information to characterize
their users. Many centers suggested this. Ideally, this would be interactive so that users can
indicate what they are using. The most popular suggestion that came from several centers
was to place a pop-up on the Web site asking a single question, such as what sector or
audience segment the user belongs to. Their usage would then be tracked using a cookie.
This would allow the center to identify their user group, as well as track their repeat usage.
Wthout follow-up the centers still wouldn't know about impact, but they would know about
reach. This recommendation would, however, require the centers to invest more resources
in changing their Web structure, as well as tracking users.
• Require or offer registration for use of the site. This would provide the center with
information they could use to identify their users and follow up with them. Consideration can
be given to have registration for discrete parts of the site rather than the entire site.
Usefulness of P2Rx Products (Evaluation Questions 3 and 4)
3) What products, services, and content do businesses need to make a decision/consider the
adoption of P2 practices?
a) Are the centers providing the right products, services, and content to businesses to
promote the adoption ofP2 practices?
4) What products, services, and content do TAPs need to promote businesses' adoption ofP2
practices?
a) Are the centers providing the right products, services, and content to TAPs to promote
the adoption ofP2 practices by businesses?
VII
-------
Businesses expressed three main priorities in their information needs:
• Up to date technical content that is easy to find on the Web using search engines and is
organized by process, topic or technology in addition to by sector.
• Information on cost of alternatives, viable substitutes, product-specific vendor
information, military prohibitions, and case studies and success stories.
• Services to help them with the cost and technical aspects of implementing P2, ways to
address or preclude regulatory requirements, and networking opportunities.
TAPs expressed three main priorities in their information needs:
• Up-to-date information with working links, maintaining the existing sector-orientation;
• Help with marketing and "selling" P2 to their customers; and
• Assistance with measurement of P2 effectiveness and cost savings.
While many centers do have well-established networks with TAPs and other stakeholders to
determine customer needs and whether the centers are meeting them, it is neither systematic
nor consistent across centers. Further, more feedback mechanisms exist for TAPs and
government entities than for industry. The information obtained from the evaluation, particularly
the interviews and survey results, indicates that the centers appear to be providing some of the
right products, services and content to businesses to promote P2, but the information is
anecdotal. There exist numerous opportunities to address customer needs further and to
provide more useful and appropriate products, services, and content.
Recommendations
• Make better use of steering committees, advisory boards, board of directors, and/or state or
regional roundtables or other stakeholder groups to solicit information about needs and
whether needs are being met by P2Rx. This should have two key components: (1) an
attempt should be made to do this more consistently across centers, possibly having a core
set of questions that are asked and an agreed-upon periodicity; and (2) questions should
include those regarding effectiveness, not just priorities and needs. This will primarily
provide information from intermediate users.
• If P2Rx does, in fact, want to be able to address more directly the needs of end users, a
consistent network-wide mechanism must be established to identify these needs.
Mechanisms for collecting information from end users may include:
o Collecting info from Rapid Response and other on-demand services as a way to
evaluate need. This opportunity should be exploited by all centers to characterize
the needs of the user community, and possibly hone in on repeat requests that can
be addressed system-wide.
o As a way to gauge the usefulness of products, and possibly in addition to collecting
more user information, use a pop-up window on the Web sites to ask a single
question of the user after they access a particular resource to learn if it met their
needs (e.g. "How useful was this fact sheet to you?").
o Conduct a survey or focus groups of a particular, well-defined, user group to
determine their needs.
VIII
-------
o Use the recurring process implemented by Zero Waste, i.e., use assistance
providers to identify needs and develop tools, conduct a workshop, obtain
commitments for action from attendees, follow up six months later, and collect results
information. The information from the workshop is placed on the Web for broader
market penetration. This model can be used for either an end-user an intermediate-
user audience.
If the emphasis is, in fact, on providing services to end-users, the information on the center
Web sites needs to be redirected toward process rather than organized by sector based on
the findings of this evaluation.
If the emphasis is on improving services to TAPs, add information that would help them
persuade businesses to adopt P2, and a measurement tool.
Expend adequate resources to keep information up-to-date on center Web sites.
Adoption ofP2 Practices by Businesses (Evaluation Question 5)
5) To what extent do businesses that use P2Rx centers change their practices or adopt P2
practices?
a) Do businesses use P2Rx information as a resource to solve pollution problems or to
help them adopt P2Rx practices?
b) What causes/influences businesses to seek P2Rx information?
The evaluation provided an indication that businesses use information and services from P2Rx
to change their awareness and practices, leading to environmental outcomes. Survey results
and case studies are currently the best source of information to examine this nationally.
Interviews for this evaluation identified several cases in which businesses claimed to have
changed practices as a result of using P2Rx information; however, this evidence is not well
documented. The evaluation did determine that although some businesses use P2Rx
information as a trusted resource for solving pollution problems, they use many other resources
as well, making it difficult to attribute results to P2Rx. The causes for business to seek P2Rx
information are the same as those that cause them to seek any P2 information, i.e., cost
savings, need for safer substitutes, responding to customer demand, regulatory compliance,
etc. Overall, the design of this evaluation was not able to isolate the affect of P2Rx on the
target populations from other factors that may influence the adoption of pollution prevention
practices.
Recommendations
• Continue to collect case studies. Consider compiling and tabulating them to show
cumulative cause and effect of center activities. However, reliance on information collected
as a result of direct assistance to end users will have limitations with respect to the number
of customers reached and the resulting quantity of outcomes.
• Perform follow up with the target audience to determine if approaches are effective at
making change happen. This can be follow up from Rapid Response-type intervention,
workshop attendance, or any other venue where you know the identity of the customer.
P2Rx should develop some standard protocols to be used by all centers.
IX
-------
• Collect outcome information as part of the activity to best engage the captive audience.
This can be done by collecting information from workshop participants or asking for a
commitment to action. This model is used by Zero Waste.
• Dedicate discrete resources to do follow up for activities. At their discretion, the centers
might hire an individual to support the entire network as another shared activity.
Outreach to Businesses Directly and Through TAPs (Evaluation Questions 6 and
7)
6) Are the centers effective in reaching businesses through TAPs?
a) What do centers do to reach businesses through TAPs?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
7) Are the centers effective in reaching businesses directly?
a) What do centers do to reach businesses directly?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
While EPA has directed the centers to shift their focus to targeting businesses and other end
users, the centers are implementing this to a greater or lesser extent across the network. There
does not seem to be a clear delineation between how centers reach businesses directly as
opposed to through TAPs. Some of the centers are TAPs as well. For many, the TAP/center
interaction worked well, leveraging resources and expertise to effectively serve the end user.
The evaluation concludes that centers more effectively serve intermediate customers than end
users for the following reasons:
• Not all centers have the resources (technical and/or financial) to provide assistance directly
to end users. In many cases it is considered out of scope or not a part of their stated
mission. In these cases, or unless the emphasis is removed from serving the TAP
community, additional resources would seem to be needed for such a retooling.
• Serving end users would appear to duplicate the work of the TAPs. This should be
considered in the context of efficient use of government funds, as well as the specific niche
intended to be filled by each of these two sets of service providers. While there appears to
be some reluctance to work with TAPs when they are connected to a regulatory agency, this
does not seem to be a major concern. One of the P2Rx centers also resides within a
regulatory agency.
• In general, the interplay between the centers and TAPs appears to be highly effective, i.e.,
the centers as researchers and information providers, and the TAPs to provide end users
with useful information for implementation.
• Businesses clearly see benefits in using P2Rx services. This would seem inevitable with a
resource as rich as P2Rx. The question should be asked if that means the centers should
actively direct their efforts towards end users, as opposed to it being a fortunate, but
passive, side benefit.
• The network does not have adequate tools in place to characterize the end user audience,
determine its needs, or determine if P2Rx is providing the right products, services and
-------
• All centers currently focus on intermediate users, but only a portion of them direct their
efforts to any degree to end users.
• Similarly, the recommendation from the 2002 Customer Satisfaction Survey was to focus on
the primary audience of technical assistance providers.
In general, the centers are more effective at reaching businesses through TAPs. For
determining effectiveness on long-term performance measures, however, it is simpler to get
results directly from businesses as opposed to trying to measure the effectiveness of reaching
businesses through TAPs. If the centers would like to better reach end-users, findings
suggested the centers should better promote their services in business-oriented forums. The
fact that the centers were a neutral, non-regulatory entity encourages businesses to seek them
out.
Conclusions regarding Outcomes are addressed under Question 5 above.
Recommendations
• Continue to focus on intermediate users rather than on end users. Work with end users as
a secondary audience as appropriate for each individual center.
• If interested in reaching the end user, promote and market center resources more widely
and beyond the usual venues. For example, advertise in business journals or on business
Web sites, attend business meetings and workshops in addition to or in place of P2Rx-
sponsored events.
Performance Measurement (Evaluation Question 8)
8) What performance measures do P2Rx centers need to collect in order to demonstrate/
document client behavior and other outcomes?
a) How would the centers that currently do not have the capacity to make/gather the
desired measurements build their capacity?
b) How can the information best be collected based on available technology, funds, and
survey approval requirements?
Centers' capabilities to measure performance vary widely, however it is clear that additional
capacity is needed to establish a robust network-wide system of measurement. Individual
centers have different capacity to perform measurement based on their staffing and other
factors. For example, Zero Waste and PPRC have better capacity to obtain outcome
information from Texas and Washington due to the P2 Planning requirements in those states.
The P2 Results system is considered to be useful by some stakeholders, but not by others.
Centers have conducted surveys, both in person and online, of various audiences for various
purposes over the years. Conference pre- and post-surveys can provide an indication of the
effectiveness of that particular event and perhaps the future needs of the audience. However,
questions about outcomes can only be answered prospectively, unless follow up is conducted.
The usefulness of national or regional surveys such as those that have been conducted of users
XI
-------
at various times will vary depending on how they are conducted, how many respond, the ability
to characterize the respondents, and what is done with the results. The recently conducted
2007/2008 Behavior Survey is useful, but it cannot be said to be reliably representative of the
customers of P2Rx. The findings show that short-term and intermediate outcomes are more
clearly influenced by the centers, but final outcomes are extremely difficult to assess.
Recommendations
The two tables below present recommendations for improving the usefulness of existing
measures and gathering new measures to better assess the centers' performance.
Table A: Recommendations for Existing Measures
Existing
Measure
Activity
Measures
Description
Currently collect:
Technical Assistance Requests—
number, type of organization, and
location
Newsletters—number
Listserves—number and number of
subscribers
Meetings/trainings organized—
number and number of attendees
Meetings attended and talked about
P2Rx—number and number of
attendees
Regional conference calls—number
Recommendations and Affected
Outcomes
• Continue to collect Activity Measures
=> Defines Outputs and some
Customers
• For Newsletters, Listserves and
Meetings, identify recipients/ attendees
using commonly agreed upon identifiers
=> Defines Customers
• For Technical Assistance Requests, use
contact information to conduct follow up
=> Defines Outcomes
Web
Measures
Currently collect:
• User sessions
• Page views
• Visitors that visit more than 10 times
• User sessions for specific products
• Continue to collect Web Measures
=> Defines Outputs
• Identify user with a single question upon
entry - or - via registration
=> Defines Customers
• Use cookies to track which pages these
users view
=> Links Outputs to Customers
• Attach cookies to a pre- and post-test for
users. If someone accesses a document,
for example, they can be asked "Was this
information useful to you? Yes/No."
=> Links Customers to Short-Term
Outcomes
XII
-------
Case Studies
P2 Results
National
Surveys
User Needs
Assessments
Pre- and Post-
Conference
Surveys
Currently collected to varying degrees
among centers. P2RIC and Peaks to
Prairies currently contribute to Zero
Waste's case study database
Centers currently conduct or facilitate
data entry from others
Currently have results of two National
Surveys
Currently conducted to varying degrees
among centers
Currently conducted to varying degrees
among centers
• Continue to collect case studies
=> Defines Activity, Output
• Consider summarizing the activities and
customers and tabulating the quantitative
data to show cumulative cause and effect
of center activities
=> Links Customers to Outcomes (with
causality)
• Enhance P2 Results so that the results
reporting by TAPs and others can be
attributed to specific center activities.
Recommend a more in-depth analysis of
P2 Results to generate detailed
recommendations
=> Intermediate and Long-Term
Outcomes (with causality)
• Conduct center-specific analysis on
2007/2008 Behavior Survey
=> Customers and Outcomes
• Survey established stakeholder groups
periodically and consistently (NEWMOA
model), or assess needs during
workshops (Zero Waste model). In either
case, standardize questions to obtain
needed information
• See below
Table B: Recommended New Measures
Recommended
Measures
Pre- and Post-
Conference
Surveys
Recommendation and Affected Outcomes
• Standardize use of pre- and post-test surveys at conferences and workshops for all
centers
• Ask respondents to identify themselves by two groups, similar to what was used in
the 2007/2008 Behavior Survey
=> Defines Customers
• In addition to asking questions about the effectiveness of the specific event (for
center use), ask general and standard questions about needs and
recommendations for P2Rx as a whole
=> Short-Term Outcomes, Needs Assessment
• Have questionnaires be collected by EPA and analyzed.
=> Defines Customers, Short-Term Outcomes, Needs Assessment Network-wide
• Conduct follow up with attendees to record changes
=> Intermediate and Long-Term Outcomes
XIII
-------
National and
Regional
Surveys
Conduct a random assignment survey of a specified target population
Note: May be used to provide a probability sample and would allow more
statistically valid inferences to be drawn from the survey. Would likely require an
ICR.
=> Short-Term, Intermediate, Long-Term Outcomes
Pre- and Post-
Outreach
Surveys
Provide a resource to an established group of people (e.g., partners) that has
agreed to provide results information. For example, a particular information packet
can be distributed to a partner audience (e.g., schools in a state or district,
businesses in a consortium) whose identities are known. Develop boilerplate
language for agreements.
Care should be taken in this case to address potential bias, and ideally a control
group would be used of similar individuals who did not receive the resource.
Survey the audience pre- and post-assistance
=> Short-Term, Intermediate, Long-Term Outcomes
Measurement
Prerequisite
Require a commitment to measure as a precondition of any provision of services, as
some centers are already doing or considering
=> Long-Term Outcomes
Direct Contact
Follow up
Perform follow up with all customers for whom contact information is known to
gather outcome information (e.g., Rapid Response and other telephone ore-mail
based on-demand services). Attempt to make this an intrinsic part of the activity for
maximum resource efficiency.
=> Short-Term, Intermediate, Long-Term Outcomes
Other recommendations:
• Conduct further study to determine how similar information providers measure their
performance. This was initially proposed to be covered in this evaluation, but was ultimately
excluded from the scope of the study. Several potential sources of information are cited in
the literature review developed by Fred MacVaugh for P2RIC, including work done by the
Centers for Disease Control and United Way. These organizations and others, such as
university extension programs, have examined ways to link information dissemination with
behavior change and other related outcomes.
• Do not hesitate to capture behavior change or other intermediate outcomes if ultimate
outcomes are not available. Behavior changes can be assumed to be correlated in a
general way with long-term outcomes, even if these outcomes are not quantified.
XIV
-------
1 Introduction
This evaluation focuses on the Pollution Prevention Information Network (PPIN), a grant
program started in 1997 to promote sharing of pollution prevention (P2) information, training,
and technical assistance expertise among states. The eight regional centers receiving these
grants are collectively known as the Pollution Prevention Resource Exchange (P2Rx), although
the centers also receive funding from other sources. Total annual funding for PPIN is $800,000.
1.1 Program Description
P2Rx is a national network of regional pollution prevention information centers (referred to as
"the centers") dedicated to increasing the adoption of pollution prevention by improving the
dissemination of relevant information. The centers provide pollution prevention information,
networking opportunities, and other services to states, local governments, technical assistance
providers (TAPs), and businesses. The long-term goals of P2Rx are to:
• Serve as the first stop for P2 information
• Increase the awareness, accessibility, and usability of P2 information
• Facilitate dynamic regional P2 networks.1
P2Rx centers work closely with assistance programs in their geographic region and support the
efforts of these programs in advancing the adoption of source reduction practices by
businesses. Centers collect, organize, and develop P2 information, as well as facilitate
networking among practitioners and others in their region. P2Rx centers also collaborate
together on various projects and a National Program Manager facilitates these interactions.
Each center is funded, at least partially, through grants from EPA. The regional centers are:
• Northeast Waste Management Officials' Association (NEWMOA) (EPA Regions 1 and 2)
• Waste Reduction Resource Center (WRRC) (EPA Regions 3 and 4)
• Great Lakes Regional Pollution Prevention Roundtable (GLRPPR) (EPA Region 5)
• Southwest Network for Zero Waste (EPA Region 6)
• Pollution Prevention Regional Information Center (P2RIC) (EPA Region 7)
• Peaks to Prairies Pollution Prevention Information Center (EPA Region 8)
• Western Regional Pollution Prevention Network (WRPPN) (EPA Region 9)
• Pacific Northwest Pollution Prevention Resource Center (PPRC) (EPA Region 10)
Each of the eight regional centers is a unique entity, providing a different mix of services
customized to address the needs and interests of its customers. The centers each have a Web
site which serves as a "storefront." Most customers are familiar with P2Rx through their
regional center and go to the regional Web site first for information. The Regional Center Web
sites act as portals to the resources of the overall network. The national P2Rx.org Web site
provides an additional point of entry into the network and is easily accessible to users that
recognize the P2Rx brand.
1 From 2007 - 2008 P2Rx Strategic Plan EXTERNAL DRAFT Revised 1/18/07.
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
1.2 Purpose and Evaluation Questions
The purpose of this evaluation is to:
1) Assess the effectiveness of the PPIN Program in promoting the adoption of P2
opportunities.
2) Identify opportunities for creating a more effective interface with customers (both
intermediate customers (i.e., TAPs) and final customers (i.e., businesses and other
polluting entities).
3) Identify current efficiencies in information collection and dissemination as well as future
opportunities for improving efficiency.
4) Determine how P2Rx activities (outputs) can be linked to long-term environmental
outcomes (listed under the goals of the Government Performance and Results Act
[GPRA] and the PPIN's environmental outcomes).
Based on the evaluation purpose, the Evaluation Team worked with Abt Associates to refine a
list of evaluation questions. The team is comprised of EPA staff from both program evaluation
and PPIN programs, and P2Rx center staff. These eight questions served as the basis for our
data collection and analysis strategies.
1) What model(s) do the P2Rx centers use to encourage exploration and adoption of P2
practices?
a) How are centers currently operating and what model(s) are they using to disseminate
information?
b) Which of the centers' model(s) is most efficient in information sharing and/or delivery?
c) When comparing among centers, what opportunities for centers' information delivery to
become more efficient are observed?
d) What opportunities are there for centers to become more efficient in information delivery
by sharing tasks?
2) What groups/types of customers access P2Rx information?
a) How do the customers vary according to method of outreach by the centers?
• Via the Web site
• Via TAPs
• Via direct contact (e.g., telephone, correspondence, or other communication)
• Via P2Rx workshops, trainings, and regional meetings
3) What products, services, and content do businesses need to make a decision/consider the
adoption of P2 practices?
a) Are the centers providing the right products, services, and content to businesses to
promote the adoption of P2 practices?
4) What products, services, and content do TAPs need to promote businesses' adoption of P2
practices?
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
a) Are the centers providing the right products, services, and content to TAPs to promote
the adoption of P2 practices by businesses?
5) To what extent do businesses that use P2Rx centers change their practices or adopt P2
practices?
a) Do businesses use P2Rx information as a resource to solve pollution problems or to
help them adopt P2Rx practices?
b) What causes/influences businesses to seek P2Rx information?
6) Are the centers effective in reaching businesses through TAPs?
a) What do centers do to reach businesses through TAPs?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
7) Are the centers effective in reaching businesses directly?
a) What do centers do to reach businesses directly?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
8) What performance measures do P2Rx centers need to collect in order to demonstrate/
document client behavior and other outcomes?
a) How would the centers that currently do not have the capacity to make/gather the
desired measurements build their capacity?
b) How can the information best be collected based on available technology, funds, and
survey approval requirements?
1.3 Evaluation Audience
The primary audience for this evaluation will be those responsible for managing and operating
P2Rx center activities. These include the PPIN/P2Rx program manager at EPA headquarters,
EPA regional P2 coordinators, and the staff and boards of directors of each of the P2Rx
centers. We expect that the P2Rx centers and the EPA program manager will use the results of
the evaluation to improve the efficiency of P2 information delivery and develop better
approaches for documenting the P2 outcomes that result from their activities.
Secondary audiences for the evaluation results will include other external stakeholders. For
example, the evaluation results will be of interest to the Office of Management and Budget
(OMB) to the extent that this evaluation establishes a basis for relating national P2 results to
P2Rx grants (outputs and outcomes). The EPA Office of Pollution Prevention and Toxics
(OPPT) may also use the results to meet the environmental results requirements under GPRA.
1.4 Program Logic Model
The evaluation questions developed by the Evaluation Team were guided by a logic model of
the P2Rx centers (see Figure 1-1 and the evaluation question numbers indicated in the model).
The logic model describes the work the centers do related to information dissemination. A logic
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
model is a representation of how a program is hypothesized to work by illustrating the causal
relationships among its resources, activities, outputs, and short-term, intermediate, and long-
term outcomes. It also helps create an evaluation framework by identifying questions for each
logic model component, which can enhance the evaluation by focusing it on questions that
produce valuable and relevant answers for the program.2 The logic model highlights the key
connections between program components and outcomes and provides a context with respect
to how the program operates.
2 W.K. Kellogg Foundation, "Guiding Program Direction with Logic Models," Item #1213, undated.
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
Figure 1-1: Generic P2Rx Information Dissemination Logic Model
Pollution Prevention Resource Exchange (P2Rx) Centers Program Logic Model—Information Dissemination
Mission: P2Rx is a national network of regional centers dedicated to improving the dissemination of
pollution prevention information in the service provider community.
Resources ^1
Funding
•EPA PPIN
•State PPIN
Match
•Other State /
Local Funding
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
Partners
•National P2Rx
•States
•Boards /
Committees /
Roundtables
1 Activities ^H Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
Provide technical
assistance
Design and
maintain websites
Collect and produce
information
resources and
content
Market resources
•P2 Results
•Case studies of
successes
Trainings,
seminars,
conferences
|
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls /
meetings
Technical Response
(e.g. Rapid Response)
f
•Center websites
•Ancillary websites
f
•Topic Hubs
•Newsletters
•Case Studies
•Databases
Marketing materials/1
V
^^^^^^1 Customers ^H
lace different ^-jf
Dmer categories
£ ^
0 O
C P
0 §
O -£
§1
.> °
c •;::
_0. JS
0
D
Other
Cente
i
Interm
•TAPs
•Local /
Governr
•Non-Pr
Organ iz
•Univers
Technic
Centers
•Busine;
Associa
i
rs
ediate
State
nents
Dfit
ations
ity
al
>s
ions /""
Final
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments
•Other £
)
)
Outcomes
|J
Inter-
mediate
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
i
it
•= ,
-------
The logic model presented here represents a synthesis of all P2Rx centers' activities, and so is
highly generalized. Although it notes some variations we identified among centers in particular
areas, the model represents the main inputs, activities, customers, and outcomes common to
most of the centers. During interviews with center contacts we learned more about the
similarities and differences between centers with respect to elements of the logic model. This
information was used to create a set of refined logic models that reflect more specifically the
inputs, activities, outputs and customers of each individual center (see Appendix A). The
intention was to be able to identify distinct groupings among them that might reflect models of
information dissemination.
In examining these logic models it may be noted that while the centers are largely similar in their
information dissemination activities and outputs, there is considerable variability among them.
Additionally, there has historically been a shared agreement among the P2Rx centers to focus
on TAPs as their primary customers, with industry and other polluting entities as secondary
customers (termed "intermediate" and "final" customers, respectively, on the logic model). EPA
recently shifted its focus to encourage centers to directly target the end user. This has been
and is being implemented to differing extents in each center, a fact that is addressed in this
evaluation.
With these points in mind, we identified the key linkages between the evaluation questions and
the logic model. The following table presents the logic model components that are relevant to
each of the evaluation questions.
Table 1-1: Crosswalk of Evaluation Questions with Logic Model Components
Evaluation
Question Relevant Logic Model Component
1
2
3,4
5
6
7
8
Generic logic model as a whole, as well as those that reflect center groupings
based on model type
Intermediate and Final Customers
All Activities and Outputs
Outcome: Implementation of P2 plans and practices
All Activities and Outputs that Are Directed at TAPs (6a)
All Outcomes (6b)
All Activities and Outputs that Are Directed at Business
Outcome: Increased Awareness of P2 (7a)
Outcome: Implementation of P2 Plans and Practices (7b)
All Outcomes
1.5 Organization of Report
This report is composed of four main chapters:
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
Chapter 1, Introduction, describes the P2Rx program and identifies the purpose of the
evaluation, its audience, and key questions that will be answered.
Chapter 2, Methods, summarizes the approach for conducting the evaluation, based on the
EPA-approved methodology and data analysis plan.
Chapter 3, Findings, discusses the collected, analyzed, and reviewed data on the centers,
technical assistance providers, and businesses according to the following general themes:
center operation models, P2Rx information users, usefulness of P2Rx products, outreach to
customers, attribution of outcomes, and performance measurement.
Chapter 4, Conclusions and Recommendations, provides concluding statements about the
evaluation findings and discusses the Evaluation Team's recommendations for P2Rx to improve
its efficiency and better measure the outcomes it influences.
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
2 Methods
The evaluation of P2Rx used a mixed method approach, and involved collection and analysis of
both quantitative and qualitative data. This chapter summarizes the data collection and analysis
methods. Prior to collection of data, the Evaluation Team began the evaluation process by
developing and refining the set of evaluation questions to be answered (see Section 1.2). Abt
Associates also developed a logic model to illustrate the activities, customers, outputs, and
outcomes of the entire P2Rx program, based on individual logic models created by each of the
centers (see Section 1.4). Finally, Abt developed a data analysis plan to guide the examination
of the data collected (see Section 2.2).
Abt Associates followed practices and procedures as per the project's Quality Assurance
Project Plan (March 4, 2008). For more information on the approach taken for this analysis,
please see "Evaluating the Effectiveness of EPA's Pollution Prevention Information Network
(PPIN) Grant Program: Program Evaluation Methodology," February 1, 2008.
2.1 Data Collection Methods
Abt Associates used a variety of qualitative and quantitative data to evaluate the P2Rx centers.
Qualitative information was used to determine the structure and operation of individual P2Rx
centers and the types of P2Rx activities and outputs that are most important and useful to
stakeholders for enabling change. Qualitative data were also used to make observations about
the perceived effectiveness of the P2Rx centers in meeting their stated goals. Quantitative data
were used to examine which P2Rx resources and tools are being used most frequently, and
with what audiences the P2Rx centers are communicating.
The sources and methods were chosen because they were feasible given the available time,
resources, and information. The data, when taken together, provide a comprehensive response
to the evaluation questions. Existing data for this evaluation are rich and multi-faceted, yet often
anecdotal or center-specific. To address these limitations, new information was collected
through interviews, focus groups, and a national survey. Interview and focus group questions
had the advantage of specifically addressing the evaluation questions for this project.
Each data collection method used is described below along with the evaluation question(s) it
helped answer. Limitations to each approach are identified. Table 2-1 summarizes the collection
methods and the evaluation questions to which they will be applied. Appendix B provides a
more detailed table relating the data collection methods to each evaluation question and sub-
question.
Evaluating the Effectiveness of the EPA PPIN Grant Program 8
-------
Table 2-1: Summary of Data Collection Methods
Evaluation Question
Collection Method , , , , , , ,
12345678
National-level Surveys*
Web Measures and
Activity Data
Interviews**
Focus Groups
Document Review***
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
V
* Includes the 2004 Customer Satisfaction Survey and the 2007/2008 Behavior Survey.
** Includes the interviews conducted in 2004 by Industrial Economics Inc. and the interviews conducted by Abt
Associates in 2008.
*** Includes center-specific documents and survey results, select case studies, and P2Rx documents.
The information collection activities above are governed by requirements under the Paperwork
Reduction Act (PRA). Under PRA, EPA's information collection is limited to nine or fewer non-
federal individuals or entities. This evaluation was conducted in compliance with the PRA and
other OMB rules on information collection requests.
2.1.1 National-level Surveys
Abt Associates used data from two individual surveys:
• Behavior Survey conducted by P2Rx in 2007/2008
• Customer Satisfaction Survey conducted by Weinreich Communications in 2004
Both of these surveys were conducted online, accessed by a link on each center's Web site.
The 2007/2008 Behavior Survey (OMB Control #2020-0015) was conducted from October 15,
2007, until February 29, 2008. It was posted as a link on each center's Web site as well as
administered to various live audiences in a hard-copy format. Some promotion was done
through the centers' electronic mailing lists (listserves) to encourage people to participate in the
survey. Surveys were logged to track how they were distributed, the number of people to whom
the survey was distributed, and the characteristics of that group. These logs were consulted in
the evaluation to provide information about the characteristics of certain subsets of respondents
(See Section 2.2.1 for more information about survey distribution logs). Web respondents
consisted of users of the center Web sites who chose to participate in the survey (i.e., by
clicking on the survey link). Additional respondents included conference participants and center
list serve subscribers, among others. Table 2-2 lists the respondents to the 2007/2008 Behavior
Survey by center.
Evaluating the Effectiveness of the EPA PPIN Grant Program
-------
Table 2-2: 2007/2008 Behavior Survey Respondents by Center
Center
NEWMOA
WRRC
GLRPPR
Zero Waste
P2RIC
Peaks to Prairies
WRPPN
PPRC
P2Rx (p2rx.org)
Total
Respondents
67
34
50
234*
80
48
77
69
10
669
* Includes 207 surveys administered in hard copy
The 2004 Customer Satisfaction Survey had nearly 500 responses, mostly from state and local
government and TAPs, with only 7 percent of respondents from industry. Similar to the
2007/2008 Behavior Survey, respondents consisted of users of the center Web sites who chose
to participate in the survey (i.e., by clicking on the survey link). To address specific evaluation
questions, the data were broken out by individual center and also by customer group (i.e.,
business versus government/TAP respondents, where possible).
The distinction between businesses/end users and TAPs/intermediate users is made in slightly
different ways in the data sources used for this evaluation. Consistent with this distinction, the
2007/2008 Behavior Survey distinguishes between the two types of customers by how they use
information, as determined in the first survey question:
Survey Question 1: Please select one option from the following list:
• I primarily use [P2Rx/Center Name] to find environmental assistance information to
apply to my own organization or facility. [Type: Business]
• / primarily use [P2Rx/Center Name] in my capacity as a provider of environmental
assistance to others outside my organization. [Type: TAPs]
This survey question and the distinction made therein represent one of the greatest strengths of
the 2007/2008 Behavior Survey. Although the precise survey audience and respondent sample
is uncertain, the customer types are distinguished in the way they have been defined by P2Rx
and for this evaluation (i.e., by end users and intermediate users).
The 2004 Customer Satisfaction Survey conducted by Weinreich Communications divides the
customer types differently. That survey asked respondents:
Please choose the category that best describes you: (select one)
• State Government (not TAP)
• Technical Assistance Provider (TAP)
• Local Government (not TAP)
• Consultant, Vendor, Trade Group, etc.
• Federal Government (not TAP)
Evaluating the Effectiveness of the EPA PPIN Grant Program
10
-------
• Business/Industry
• Non-Governmental Organization (NGO)
• Academia (not TAP)
• Other
This earlier Weinreich Communications survey distinguished between respondents identifying
themselves as "Business/Industry," and all others. This distinction was made differently in the
more recent 2007/2008 survey; therefore, results from the 2004 survey are presented in this
report in terms of "business" respondents and "all others." Note that while the 2007/2008 survey
allows respondents to identify themselves by the way they use P2RX information, the 2004
survey does not. Thus, we cannot make a direct comparison of responses across the two
surveys divided by user type.
Both surveys used for this evaluation used a self-selected sample of respondents from a target
population that is not defined, which precluded us from estimating the probability that the group
of respondents represents the whole. This impacts the ways in which survey data can be used
for purposes of the evaluation. Section 2.2, Data Analysis Methods, goes into more detail in this
regard. For a full summary of the 2007/2008 Behavior Survey results, see Appendix C, and for
center-specific Behavior Survey results, see Appendix D. For further analysis of the 2004
Customer Satisfaction Survey, see Appendix E.
2.1.2 Web Measures and Activity Data
P2Rx centers collect standard information to measure activities and Web site usage, which
helps determine the extent to which different P2Rx services are used. The centers report
general Web site use statistics as well as usage data on a number of specific P2Rx online
resources. Twice a year, aggregate data for the entire P2Rx network are compiled and reported
to EPA by the P2Rx Coordinator. Abt Associates examined data from 2005 to 2007 on client
type; Activity Measures, i.e., types of assistance provided by centers, broken down by direct
contact, newsletters, listserves, and meetings; and Web Measures, i.e., types of Web use by
product (e.g., P2 libraries, topic hubs,3 P2 programs directory, Rapid Response,4 sector
resources, P2 news, and other resources). Using this information, we examined use by
different categories of users. We also used the information to help understand the level of
demand for each type of assistance.
There are two primary limitations associated with using this information source for the
evaluation. First, it is not possible to draw conclusions regarding the use of center resources as
compared to the total population they serve, because the universe of Web users is unknown.
Second, types of customers and their usage patterns are not differentiated enough to
adequately answer evaluation question 2.
3 Topic Hubs™ are compilations of information on a particular topic. They were the first standardized
product adopted by the centers.
4 Rapic
Rudy.
4 Rapid Response is on-line research assistance. Alternatively, some centers call this Help Desk or Ask
Evaluating the Effectiveness of the EPA PPIN Grant Program 11
-------
2.1.3 Interviews
Two sets of interviews were used by Abt Associates for this analysis. First, Abt Associates
conducted interviews with all eight P2Rx center directors, seven TAP representatives, and eight
business representatives who use the P2Rx centers. The Evaluation Team invited the centers
to provide Abt Associates will a list of contacts for this purpose. We selected nine business
representatives and nine TAPs from the names provided based on geographic distribution, to
attempt an even representation among the centers. WRPPN did not provide names of TAPs
and business representatives because that center had already provided similar contacts for the
focus groups (see Section 2.1.4). Due to time and scheduling constraints, we were able to
interview eight business representatives and seven TAP representatives, as well as all eight of
the P2Rx center directors.
The interview guide (see Appendix F) provides a basic script for the interviews, and was given
to the participants ahead of time. Abt Associates scheduled interviews in advance and
conducted them over the phone. We tape-recorded the interviews for quality assurance
purposes; however, in order to encourage open and candid responses, individual interview
responses and comments are not referenced or attributed to specific individuals in this
evaluation report, nor are the names of the interview participants listed.
Abt Associates interviewed the following end-user/business representatives, including three
from PPRC, two from GLRPPR, one from P2RIC, one from NEWMOA, and one from Peaks to
Prairies:
• University environment, safety, and health (ES&H) director, who is responsible for
compliance with EPA regulations and campus sustainability
• Large retail dry cleaner representative
• Pulp and paper mill representative
• Representative of a naval air station who serves as a training command, and provides
hazardous and solid waste support, P2 support, etc.
• Semi-custom cabinet maker who produces 1,000 cabinets a day
• Maker of aerospace parts for commercial passenger or transport aircraft, with some military
work
• Ski area and real estate developer
• Environmentally-sensitive jewelry maker
Abt Associates interviewed the following TAP representatives, including 2 from GLRPPR, 2 from
P2RIC, 1 from Zero Waste, 1 from WRRC, 1 from NEWMOA:
• Four based in state TAPs
• Two based in university TAPs
• One based in a consulting organization that provides technical assistance
The second set of interviews was from an evaluation of the PPIN grants that Industrial
Economics Inc. (lEc) conducted for EPA in 2004. While the 2004 evaluation did not directly
address the questions in the current evaluation, Abt Associates gleaned additional information
by reviewing the transcripts of the interviews, which were with EPA managers and project
officers, P2Rx coordinators, P2Rx center contacts, and TAPs.
Evaluating the Effectiveness of the EPA PPIN Grant Program 12
-------
We summarized interview responses to present common themes and key observations. In
addition, the generic logic model developed by Abt Associates was used during the interviews
with each of the eight P2Rx center directors. It was discussed in detail in conjunction with the
logic models that had been developed earlier by each center, to help determine which activities
the center undertakes and what outputs it produces. We then developed a unique logic model
for each center based on this discussion (see Section 3.1.1). Unique outcomes were not
addressed as a part of this exercise.
2.1.4 Focus Groups
Abt Associates conducted focus groups at the Western Regional Pollution Prevention
Conference in San Diego, California, in October 2007. Three focus groups were conducted, two
with TAP representatives and one with business representatives. Abt Associates solicited the
perspectives of each of these groups about P2Rx and their local centers. The goal of these
focus groups was to solicit perspectives and opinions from each of these audiences on center
offerings and effectiveness, and allow group interaction to generate information and identify
considerations that would have been unlikely to emerge with individual interviews.
Each focus group had between five and six participants. Focus group participants were
identified with the assistance of the conference organizers: Ed Gonzales (WRPPN) and Chris
Wley (PPRC). Although a list of the individual participants is not included in this report, a
general description of the participants is as follows:
Business Representatives:
• New business-owner (water distribution company)
• Consultant whose business assists companies in implementing sustainable practices
• Food company employee who is involved in non-food product development
• Representative of a hazardous products and chemical database company who helps
organizations comply with regulations and provides information
• Strategy and management consultant focused on sustainability
TAP Representatives:
• Five individuals based in state technical assistance programs
• Three individuals based in county technical assistance programs
• Two individuals based in local (city) technical assistance programs
Prior to the sessions, Abt Associates prepared a moderator's guide with a summary of the focus
group's goals and objectives and a list of questions to be covered (see Appendix G). Sessions
were facilitated by Abt Associates staff and were tape-recorded and transcribed.
2.1.5 Document Review
Based on the recommendation of three center staff on the Evaluation Team, Abt Associates
conducted a review of grant reports to help identify center-specific documents that could be
relevant to the evaluation. Each center publishes different documents and occasionally
Evaluating the Effectiveness of the EPA PPIN Grant Program 13
-------
conducts surveys that are not standardized across all centers. Abt Associates reviewed the
grant reports submitted to EPA by each center for the calendar years 2005, 2006, and 2007.
Based on what each center described in its report, Abt Associates identified additional
documents that could support the evaluation, such as survey results (needs assessments and
others), case studies, and possible contact names for interview subjects.
Abt Associates drafted a request to the centers describing the purpose of the information
collection and listing the suggested documents to be provided by each center, which was
reviewed by the evaluation team and then sent to each center contact via e-mail. Abt Associates
received survey data from five centers, case studies from four centers, and interview contact
information from all centers except WRPPN, which had already provided contact information for
the focus groups.
The primary limitation of this data source is that information obtained from these documents
only allows us to draw conclusions about the source center (i.e., results will not be
generalizable).
In addition to center-specific documents, several national-level documents provided to Abt
Associates were used to supply background information to the reviewers and add detail to the
analysis. These documents include the 2007-2008 P2Rx Strategic Plan and the "Why P2Rx?"
document. For a list of all documents consulted, please see Appendix H.
2.2 Data Analysis Methods
Abt Associates compiled and assessed a variety of information, both qualitative and quantitative
in nature. For qualitative information obtained from interviews and focus groups, Abt Associates
referred to its notes and interview tapes to summarize the information provided. For quantitative
information obtained from on-line surveys, Abt Associates used a standard software package
and data analysis and presentation techniques to summarize the results. The data was taken
from the Survey Monkey Web site and downloaded for each site in CSV format. It was opened
in Excel, and arranged onto one spreadsheet, removing answers to question 6 that were not
consistent across all surveys. The data was then condensed so that responses to questions
aimed at business and TAPs were taken together where possible (all except question 7). Also,
the hard copy TAPs and business surveys were added to the dataset. Pivot tables were used to
sum the scores for each question for TAPs and business, and then this sum was divided by the
number of respondents in that category for that question, to obtain the final frequency number.
For a detailed explanation of Abt Associates' data analysis methods, please refer to "Evaluating
the Effectiveness of EPA's Pollution Prevention Information Network (PPIN) Grant Program:
Data Analysis Plan," April 1, 2008 (Appendix I). Abt Associates documented its quality
assurance procedures in its Quality Assurance Project Plan, which was approved by Abt
Associates, Industrial Economics, and EPA on April 11, 2008.
Abt Associates analyzed the data collected and reviewed in the context of two important
limitations on the study. First, there is no representative sample of TAPs or end users present in
Evaluating the Effectiveness of the EPA PPIN Grant Program 14
-------
any of the primary or secondary data available for the study. The available survey results are
based on a self-selected sample of those deciding to respond. The focus groups are small
groups of informants, and the interviews are based on individuals selected by the centers, or
center staff themselves. Second, and particularly pertaining to evaluation questions 5, 6, and 7,
there is no strong design to isolate the true effects of P2Rx activities from other sources of
information or influences on behavioral change. Given these factors, however, the information
sources used for the evaluation taken as a whole did allow for a comprehensive and meaningful
response to the evaluation questions.
In general, surveys are conducted using either probability samples or non-probability samples.
In probability sampling, a sampling frame of all members or close to all members of the target
population is created, and a random sample is drawn from the population. Part of this process
involves defining the target population. Random sampling gives each member of the target
population a known probability of selection, and this allows for the calculation of the standard
errors.
Non-probability samples, on the other hand, often do not have a sampling frame of all members
of the target population. Non-probability samples include quota samples, judgmental samples,
purposive samples, and self-selected samples. Widely used opt-in Web surveys, such as the
P2Rx 2007/2008 Behavior Survey, fall into the category of self-selected samples. There is not a
random selection mechanism from a known target population in this case. Response rates are
almost always unknown and are thought to generally be much lower than mail, telephone, or in-
person survey response rates. Standard errors cannot be calculated for these types of samples,
and so the information from these surveys may not be reliably representative of the universe.
This is because respondents self-select and are only a very small proportion of people who use
the Web site. Such surveys can be useful in some cases, but the survey results need to be
qualified as only potentially applying to the individuals or business establishments that
responded.
2.2.1 2007/20008 Behavior Survey Distribution Logs
Seven of the eight centers kept logs recording to whom they sent notification of the survey.
Note that while the following reflects the population who were sent the survey, it is not known
who responded. Responses to the survey were anonymous.
• NEWMOA distributed the survey link electronically to their Northeast Assistance and P2
Roundtable listserve two times, for a total of 176 contacts. This listserve is comprised of
federal, state, and local assistance programs.
• WRRC distributed the survey link electronically to EPA Region 4 member states and TAPs.
The survey was also attached to "Ask Rudy" responses.
• GLRPPR distributed the survey via e-mail to: users of GLRPPR Help Desk Librarian service
(4); GLRPPR members, two times (548 each); DOD environmental coordinators in the Great
Lakes region (185); and GLRPPR steering committee members (20). The predominant
Evaluating the Effectiveness of the EPA PPIN Grant Program 15
-------
recipients were GLRPPR members, with a total of 1,096 contacts made. The composition of
this member group is not known.
• Zero Waste Network sent the survey electronically to 1,919 TAPs. In addition, they
distributed 207 surveys in hard copy at workshop events, of which 77 were TAPs and 130
were businesses.
• P2RIC sent out 1,084 e-mails to various listserves at eleven times over the course of the
survey period. Of these 1,084 e-mail recipients, they are able to determine that 87 clicked
through to the survey itself.
• Peaks to Prairies sent the survey to 624 contacts, many of whom were contacted more than
once. The recipients included Region 8 state P2 coordinators, Region 8 listserve, Homes
Across America contacts, Montana Eco-Star award recipients, users of their help desk,
recipients of their Green Parks Environmental Toolkit, participants on the Sustainability
Workgroup listserve, participants on the Tribal P2 Workgroup listserve and to participants in
the Parks Stewardship Network. Of these groups, the following is known 6 are TAPs, 38 are
from state or local government, 10 are from the federal government, 51 are from
universities, 9 are from the military, and 74 are from the private or non-profit sector. The
remainder of recipients is of unknown type. It can be assumed that there was relatively high
distribution to the private sector/end-user community based on the fact that the survey was
sent to award recipients, users of the help desk, and other groups that likely have some
representation from this group.
• WRPPN sent an e-mail notification of the survey to its regional listserve two times. This
listserve has approximately 300 members of unknown type.
• PPRC sent160 survey notifications to users, out of which about 120 were from city, county
or state government agencies while 16 recipients were from utilities, 14 were from
companies, 4 were from colleges or universities, 2 were from the Army, and 4 were from
local school districts (40 total).
This log information provides information about who was notified of the survey, and can provide
a general sense of which audiences may have a higher probability to take the survey. However,
without knowing which of these entities actually did respond, the information cannot be used to
determine bias. For example, while it would appear that the vast majority of recipients of
notification of the survey were government entities or TAPs, it is unknown how many of these
individuals took the survey as a result of being notified as opposed to those who accessed the
survey on the Web site independently. The log information, however, could be used to examine
individual response rates for the centers, as well as to learn about different distribution methods
and target audiences for the various centers. As a method of characterizing respondents, the
information on the survey itself that asks the category of respondents is very useful, but is
independent of the log information.
Evaluating the Effectiveness of the EPA PPIN Grant Program 16
-------
3 Findings
This chapter presents the findings of this evaluation, discussing the primary and secondary data
collected on the centers, technical assistance providers, and businesses according to the
following general themes: center operation models, P2Rx information users, usefulness of P2Rx
products, outreach to customers, attribution of outcomes, and performance measurement. The
themes serve to consolidate and summarize the evaluation questions to enable a more
cohesive organization for the analysis. Table 3-1 illustrates how the themes relate to the
evaluation questions.
Table 3-1: Crosswalk of Report Sections and Themes with Evaluation Questions
Report Section/Theme Relevant Evaluation Questions
3.1 Center Operation Models
3.2 P2Rx Information Users
3.3 Usefulness of P2Rx Products
3.4 Outreach to Customers
3.5 Attribution of Outcomes
3.6 Performance Measurement
Questions 1, 1a, 1b, 1c, 1d
Questions 2, 2a
Questions 3, 3a, 4, 4a
Questions 6, 6a, 7, 7a
Questions 5, 5a, 5b, 6b, 7b
Questions 8, 8a, 8b
An important distinction made throughout this evaluation and relevant to the findings is made
between businesses, or end users, versus TAPs, or intermediate users. In this chapter, we use
the term "business" to refer to any end user of P2Rx information. The term "TAP" is used to
refer to any intermediate user who assists businesses in implementing P2.
In general, when the term "business" is used in this report, it is referring broadly to end users
who may include academia, military, or other organizations that are responsible for their own
environmental concerns. Ideally, audiences for P2Rx are best defined by how they use the
information, regardless of what type of organization they are in (i.e., business, government,
etc.).
Evaluating the Effectiveness of the EPA PPIN Grant Program
17
-------
3.1 Center Operation Models
Evaluation Question 1: What model(s) do the P2Rx centers use to encourage exploration and
adoption of P2 practices?
a) How are centers currently operating and what model(s) are they using to disseminate
information?
b) Which of the centers' model(s) is most efficient in information sharing and/or delivery?
c) When comparing among centers, what opportunities for centers' information delivery to
become more efficient are observed?
d) What opportunities are there for centers to become more efficient in information delivery by
sharing tasks?
The centers vary extensively in their structure, location, staffing, resources, activity emphasis,
and customer base as described in the following analysis, and their location and organization
affect the resources each center is able to leverage (see Appendix A for individual center logic
models with variations among the centers highlighted). An additional distinction is in the use of
PPIN funds as a sole versus a contributing source of funding for centers. All centers have some
sort of external stakeholder or advisory group with whom they communicate. With
understandable variability among them, P2Rx clearly enables P2 programs to enhance their
individual capacity by sharing and leveraging the resources and knowledge of the entire P2Rx
network. The national scope of the network, and the close working relationship among the
centers, facilitates this. Efficiencies can be realized by having the best qualified center perform
particular tasks. For example, case study collections, the Programs Directory, and some of the
Web programming functions are well-suited to being centralized. However, there is overhead
associated with this as well, in terms of time spent on conference calls or performing
administrative activities. For all of these reasons, an interactive and dynamic network does
appear to strengthen the abilities of any single organization.
The general mode of information dissemination for the centers is to cast a wide net to a broad
audience, for example by conducting workshops, using the Web, or networking. The research
or information they provide to TAPs allows the TAPs to be more efficient when they go out in the
field. It was felt that the centers are better able to do in-depth research than are TAPs because
center staff do not spend their time in the field. Efficiencies are already being gained through
various shared or centralized activities. The case study database run by Zero Waste is
considered effective in sharing a well-needed resource. Centralized IT support in certain areas
and overall Web site maintenance is also effective, as well as the centralized management of
the Programs Database. Developing IT platforms was cited as a good activity to centralize, as
well as overall Web site maintenance, since some centers have stronger IT support than others
and can help with this. Inefficiencies were cited, including the requirements for measuring the
effectiveness of the centers and the committee-based management of P2Rx.
Evaluating the Effectiveness of the EPA PPIN Grant Program 18
-------
3.1.1 Center Structure and Staffing
Centers are situated in a variety of organizations. For example, NEWMOA is a non-profit
interstate organization; WRRC is in a state regulatory agency; GLRPPR is co-located with a
TAP on a university campus; Zero Waste has the resources of a university at its disposal and
works closely with the Manufacturing Extension Partnership (MEP) due to proximity of their
office space; P2RIC is part of a business development center at a university; Peaks to Prairies
is located at a university extension service; WRPPN is housed with a university Business
Environmental Program, which serves as the state P2 program/TAP; and PPRC is a stand-
alone non-profit organization.
The location and organizational structure of each center affects the resources it is able to
leverage. For example, WRRC has access to the resources of a 30-person staffed Pollution
Prevention program located within a regulatory agency. Other centers can leverage a co-located
TAP or a university. However, for a center like Peaks to Prairies there is not a lot of overlap
between what the center does and what their larger organization does, and their activities are
isolated from the rest of the university, so it is difficult to leverage resources.
Centers use and rely on PPIN funding in a variety of ways. Some leverage state contracts with
PPIN funds, some rely on other sources for the majority of their funding (federal and non-
federal), others are primarily funded by PPIN. Additional funding may come from state or local
sources (rare), in-kind match from various sources, fees from training or other events, or an
array of other sources, (see individual logic models in Appendix A).
Some centers have many staff; some have only one or two. All of the centers have some
combination of a steering committee, advisory board, board of directors, and/or roundtable
comprised of states and/or a variety of stakeholders with whom they communicate periodically.
Centers can also be characterized by the extent to which they work with end users. For
example, NEWMOA doesn't target end users at all, while Peaks to Prairies, PPRC and Zero
Waste actively do. Other centers fall somewhere in between.
P2RIC uses a graduate student staff model. These students are trained to do content
development, which was previously done by a professional librarian. This staff model brings in a
lot of energetic and talented people; however, they mostly have academic knowledge and lack
field experience. This limits their ability to engage with businesses to achieve and measure
outcomes. Higher-level professional skills are needed to reach businesses, but P2RIC's funding
will not support the hiring of professionals. However, IT students have made valuable
contributions to P2RIC's online resources, and use of student IT workers allows P2RIC to
forego the sudden loss of capacity and institutional knowledge that can occur with the loss of a
full time employee. Overall, students provide very cost-effective work for P2RIC. Other centers
may use similar student staff models as well, but this was not mentioned in other interviews.
Each center has its own "personality." A single commenter said that the quality of the centers
varies quite a bit, as does their utilization, and that it is difficult to support and defend the
existence of eight centers. However, this does not seem to be a widely held sentiment. In fact,
others believe the work of each individual center is important. As Table 3-2 below indicates, the
Evaluating the Effectiveness of the EPA PPIN Grant Program 19
-------
centers do indeed vary in terms of their utilization and areas of emphasis. It was also discussed
in the center interviews that certain centers have specific technical capabilities in P2 technology
or other areas. Others focus on information or program management. Three centers have
particularly strong Web expertise.
Table 3-2: Selected Web and Activity Measures for Each Center: First Half 2007
Activity/Web Measure
Technical Assistance Contacts
Newsletters Distributed
Listserve Subscribers
Number Meetings Organized
Web User Sessions
NEWMOA
20
1,400
1,734
2
96,192
WRRC
38
0
350
0
87,798*
GLRPPR
325
1,025
1,358
6
239,841
Zero
Waste
42
0
48
2
40,165**
P2RIC
76
452
1,381
7
54,129
Peaks
1,173
0
232
6
85,835
WRPPN
397
0
275
1
149,530
PPRC
163
2,827
0
38
0**
1 WRRC also received 2,076,727 hits to its library during this period
** Incomplete Web Measures information due to technical problems
Source: Semi-annual Web and Activity Measures report from the first half of 2007.
It was noted in the 2004 interviews conducted by I EC as well as in "Why P2Rx?," that four of the
centers: NEWMOA, WRRC, GLRPPR, and PPRC were already established entities before the
P2Rx network was developed and were created by state agencies explicitly to support state
agency P2 programs. At the time of the 2004 interviews it seemed that having been established
prior to the network provided both benefits and difficulties for those centers; their established
infrastructure was a good launching point for the network, however their existing infrastructure
allowed less flexibility in changing the direction of their focus.
One center felt strongly that the way they were established prior to P2Rx, i.e., using TAPs to
provide information and collect data from businesses, is most efficient for them. A change in that
structure would seriously jeopardize the way pollution prevention is addressed in their region.
Another center similarly said that their position within their larger organization makes it very
difficult to change their focus to start assisting businesses directly.
A business representative suggested that some centers may work more closely with the state in
which they are located than they do with other states in their region. In general, larger, better
resourced, centers may have better capacity to cover more of their geographic region. The
evaluation did not examine the issue of geographic reach of the centers. It is a policy decision
whether the centers are (1) expected to service each state in their region equally, (2) focus on
serving the most pressing environmental need, or (3) go beyond the region to become centers
of excellence for the nation.
3.1.2 Center Information Dissemination Activities
Activities vary from one center to another. Most centers seem to aim to cast a wide net and
reach as many entities as possible, whether by holding conferences or placing content on a
Web site, as opposed to providing one-on-one assistance. Core center activities include:
creating and conducting trainings, participating in networking, providing technical assistance,
design and maintaining Web sites, and collecting and producing of information resources. The
Evaluating the Effectiveness of the EPA PPIN Grant Program
20
-------
centers accomplish these activities in different ways, and emphasize them to different degrees.
The logic models in Appendix A illustrate the similarities and differences among centers with
respect to information dissemination activities, specifically inputs, activities, outputs and
customers served. The following findings are arranged consistent with the activities in the logic
models.
Evaluation and Measurement. Most centers spend a lot of time on performance
measurement. It was expressed by every center that measuring results, and attributing results
to their activities, is difficult or impossible. Centers currently accomplish measurement in
various ways, which are discussed in greater detail in Section 3.6.
Trainings and Seminars. All centers have a very active role in organizing, conducting, and
participating in trainings and seminars. This is a key activity to enable them to reach large
audiences to disseminate information. Centers such as PPRC and WRPPN have been
increasingly working together and now hold an annual joint conference. This provides greater
efficiency in being able to share responsibility for logistics, registration, and other overhead
functions associated with putting on a conference.
Networking is an important activity for all the centers. An example was offered of a document
that was developed in Alaska, re-published by a center, and then given to someone in Arizona.
Such a transfer of information would not be likely to happen without the network. In other
cases, networking with other information providers can broaden the reach of the centers. For
example, a staffer from the Great Lakes National Program Office (GLNPO) reported how
GLRPPR has worked with GLNPO on multiple projects to promote their activities. Networking is
also used as a form of needs assessment, gathering information about needs from
stakeholders. The ability to use the P2Rx infrastructure and network to disseminate results
nationally was widely cited as very useful and effective.
Technical Assistance. Rapid Response is used to varying degrees across P2Rx, and is also
referred to as "Ask Rudy" and "Help Desk" by certain centers. It is defined broadly by some as
any request for assistance, be it from a business or a TAP, and can vary from a quick phone
number lookup to a technical question that requires research. Technical Assistance Requests to
the centers vary widely; as shown in Table 3-2, NEWMOA, WRRC, and Zero Waste received
fewer than 50 Technical Assistance Requests, while GLRPPR and WRPPN received over 300,
and Peaks to Prairies received 1,173 requests during the first half of 2007. An observation was
made that often helping businesses via Rapid Response takes less time, while helping a TAP
may take longer and be more complex. Assisting TAPs is more complicated, further removed
from the final result, and, ultimately, more difficult to measure. However, assisting TAPs in
helping multiple businesses can potentially have a greater impact than helping a single
business.
Web site Maintenance. Some centers are trying newer methods of communicating and
disseminating information using the latest Web technology. For example, GLRPPR has a Web
Blog open to everyone that facilitates networking and information sharing in an interactive
electronic format, and WRRC is interested in creating video content for posting on YouTube.
Other centers indicated interest in novel Web networking concepts, such as YouTube videos or
Evaluating the Effectiveness of the EPA PPIN Grant Program 21
-------
Face Book-type social networking, which would allow people to connect to the type of
information they are most interested in. Admittedly, some of these pursuits would involve a
learning curve for less Web-savvy users, but these approaches will allow P2Rx to expand their
audience and utilize more adaptable, interactive technology. Developing IT platforms was also
cited as a good activity to centralize, as well as overall Web site maintenance, since some
centers have stronger IT support than others and can help with this.
Information Resources and Content. Five of the centers develop and write up case studies of
successful P2 implementation as part of their activities. Case studies were mentioned by six
centers as being critical pieces of information for training and other assistance. Specifically,
P2RIC and WRRC emphasized the usefulness of case studies and mentioned the efficiency of
partnering with Zero Waste to build their database and using the database as a repository for
this information. Zero Waste's P2Planner5 is being requested by other states that have P2
planning programs, for which they can easily adapt it.
Individual topic hubs6 are led by certain centers, with others contributing to those efforts and
linking to the final product. This seems to centralize the work in a way that allows all to benefit.
P2RIC does not do much with topic hubs besides updating them with new information, and
WRRC only does link checking to keep the topic hubs alive, due to lack of funds. Three centers
noted that while there was benefit to developing shared topic hubs, there was also a huge
resource investment involved. One center commented that the topic hubs are geared to
informing the state TAPs, rather than helping industry. Indeed, the topic hubs were one of the
first products to be adopted by the centers, and were intended as a primer for TAPs to
familiarize themselves with a particular sector. One commenter said that there is too much
background information in the topic hubs to be useful to industrial users, who just want "the
answer." Development of a derivative product from the topic hubs has been considered,
eliminating the background information and repackaging the core links to make it more
accessible to businesses. At this time there may also be a liability associated with the current
structure of the topic hubs in terms of customers being able to find information, as the hubs are
not designed well for Google-type architecture. Based on the observations of the Abt
Evaluation Team, P2Rx resources such as case studies, fact sheets, and best references
pages, are frequently returned on the first page of a Google search; while the topic hubs often
do not appear on the first page unless the key words in the title are included in the search.
NEWMOA described their service as allowing TAPs to spend more time in the field, and less
time looking for information. The center "arms" TAPs with selected and prepackaged information
to improve their efficiency. According to the center representative, it is fair to say that a similar
sentiment would be shared by many of the other centers.
Marketing is typically not done as a discrete activity, but rather is done by distributing materials,
getting the word out at workshops or leveraging partners to do this. Participants in the 2004 I EC
interviews expressed the need to do more marketing of P2Rx's services and resources.
Although marketing was mentioned in 2004 as an activity that should become a priority, it
5 P2 Planner is a tool for preparing a pollution prevention plan.
6 Topic Hubs are compilations of information on a particular topic. They were the first product adopted by
the centers.
Evaluating the Effectiveness of the EPA PPIN Grant Program 22
-------
seems from the current evaluation that marketing has not been prioritized, either by the centers
or by EPA.
3.1.3 Opportunities for Efficiency
Efficiencies7 that were cited ranged from information sharing among centers to administrative
matters. For example, because PPRC is an independent non-profit organization, they are not
restricted by government bureaucracy and are able to respond more quickly to changing needs
and to pursue what they feel is important. NEWMOA also felt that their internal structure,
specifically their well-defined decision-making structure, works well for them. They have a clear
understanding of who their members are, whom they are serving, how to get input from their
members (who are also their customers). The state regulatory programs that comprise their
members are best positioned to identify local priorities. NEWMOA's longstanding and regular
communication with their states about priorities, sector foci, etc. enhances their capability as a
center.
Content sharing among centers takes place on certain specific fronts, and was suggested in
areas. GLRPPR provides a news service for multiple centers, and will be adding more. It is a
relatively simple programming task to enable all centers to access this resource. GLRPPR felt
that it is not much more difficult to do this for multiple centers once they were doing it for some.
However, another participant commented that because EPA is counting Web user sessions at
each center, there may be resistance to sharing documents with other sites. Several centers
seemed to see value in having a centralized library repository.
One center commented that P2Rx has not yet figured out a good way to utilize each center's
individual strengths and play to those strengths, such as Web development or content
development. Each center seems to want to maintain their own expertise and autonomy, but
services may be more economically provided if centralized. Two participants observed
reluctance among the centers to give a particular center a lot of responsibility for an activity
because it might compromise another center's autonomy and control.
Four centers (WRRC, P2RIC, Peaks to Prairies, and Zero Waste) commented that the Case
Study database run by Zero Waste is an example of effective sharing of information and
development of a well-needed resource. Commenters on the P2 Programs database
appreciated that separate databases no longer needed to be maintained and that programming
and maintenance was all being coordinated nationally. Work is currently being done by
GLRPPR and others to centralize the Web programming for topic hubs, which are largely
considered to be P2Rx's primary informational product.
Two P2Rx activities were highlighted as being particularly inefficient: administration of the PPIN
grant, and fulfilling measurement requirements. One center noted that the administration of the
PPIN grant was very burdensome, and two centers identified inefficiencies with the timing of the
PPIN grant process. These two centers noted that the one-year PPIN funding cycle makes it
7 Efficiency in the context of this report is based on informants' or the evaluators' judgment that a
particular strategy could reach more people for the same money. Quantification of output per unit of cost
were not conducted.
Evaluating the Effectiveness of the EPA PPIN Grant Program 23
-------
difficult to do longer-term projects. Additionally, getting funding under PPIN usually takes 6-8
months and could take as long as a year. If people identify a deeper need that requires some
long-term work, it is frustrating because then centers feel they cannot respond quickly. There is
a purely administrative delay between identifying need and being able to produce something,
and that impacts efficiency. While this is not a function of intrinsic center operations, it is an
external factor that impacts the centers' ability to operate at full effectiveness.
Additionally, every center felt that the requirements for measuring the effectiveness of the
centers were particularly inefficient. Centers spend a lot of time and resources fulfilling
measurement requirements, and see very little benefit from them. The centers understand the
importance of measurement, but the current measures they are required to collect are
burdensome (see Section 3.6 for more about performance measurement).
PPRC sees inefficiencies in not being a part of a larger organization through which they can
leverage resources and absorb costs, e.g., having an IT department when the server goes
down. It also takes a great deal of resources and time for PPRC to procure funding because of
their non-profit status. Peaks to Prairies also identified inefficiencies in their organization, since
they are very reliant on PPIN funding and lack the ability to leverage other funds that many
other centers have. Although Peaks to Prairies is located within a university, they are isolated in
terms of their activities and have difficulty capitalizing on university resources.
One center commented that there are things for which a decentralized approach is best, and
others for which it is not. For example, for something like the Programs Directory, it is more
efficient to divide up the task of gathering information about programs within each region. At the
same time, centralizing the development of the programming code for this project was most
efficient. Similarly, something like updating code for topic hubs can efficiently be done by a
single center with IT expertise. Another point of view was expressed on this subject regarding
the inefficiency of committee-based or "majority rules" type management. Two participants
expressed concern that the network spends a lot of managerial time coordinating with other
centers, and this is not always value added. On the other side of that, there was a comment that
it is worthwhile to have eight sets of eyes looking for new technical content, or to look at a sector
as a whole to see needs and trends and work together to address them.
Business View of Centers
Business representatives initiated their relationship with their local P2Rx center in a variety of
ways, including finding the center on the Web either from a direct link (e.g., from the Montana
State University Web site) or when doing Internet research; receiving a business card for the
center during an inspection; or meeting a staff member at an industry event. One business
representative noted that he accesses center information through the Web site and listserve.
Business representatives reported receiving numerous and varying services from their P2Rx
center, including communication advice, promotion for events, and technical assistance; training
sessions (waste reduction, lean manufacturing, general topics); and responses to phone calls
requesting P2 information. Respondents noted that the center was helpful in compiling
requested information.
Evaluating the Effectiveness of the EPA PPIN Grant Program 24
-------
Business representatives have used P2Rx center services for a variety of purposes and on a
range of topics. One respondent has mostly worked with the center on byproduct synergy
(industrial waste exchange). Another participant noted that the center has responded to multiple
phone calls to provide information on chemical substitutes. Another business representative has
been working with the center extensively, receiving help with its Lean Environmental Pilot Grant,
four week-long events involving value stream mapping, lean tools, kaizen (improvement)
events, and spray painting training (the center has particular technical expertise with spray
coating). This representative noted that the center seems to serve as a consultant to the
business.
Focus group participants were attending a center-sponsored conference, but were generally
unfamiliar with the center Web sites. They indicated that they had used the Web sites
peripherally or had found the sites in relation to the conference.
TAP View of Centers
In general, TAP comments suggested that they understand the purpose of the centers to be
collecting, developing, and distributing information, as well as providing opportunities for
networking. Most TAPs agreed that the centers serve TAPs better than they serve businesses.
One TAP commented that because of their structure, the centers are better equipped and more
able to do in-depth research and spend more time on case studies because they are not out in
the field with customers.
TAPs indicated particularly close and effective relationships with Zero Waste and PPRC. One
mentioned that because Thomas Vinson of Zero Waste had previously worked for the Texas
state TAP, he had a particularly good understanding of the role of TAPs and ran his center in
order to complement their work. One TAP said that although she was not in NEWMOA's region,
she had worked closely with them because of their expertise in mercury issues.
Evaluating the Effectiveness of the EPA PPIN Grant Program 25
-------
3.2 P2Rx Information Users
Evaluation Question 2: What groups/types of customers access P2Rx information?
a) How do the customers vary according to method of outreach by the centers?
* Via the Web site
Via TAPs
* Via direct contact (e.g., telephone, correspondence, or other communication)
* Via P2Rx workshops, trainings, and regional meetings
The P2Rx centers fill three main roles for businesses: (1) networkers who connect them with
people or services that can help them; (2) information providers who answer questions or
offering written materials in person or from the Web; and (3) event sponsors who provide
training, conferences, and other forums. In general, the preference among business
representatives was to receive information by e-mail and the Web, and also through in-person
events.
the networking
opportunities provided
by the P2Rx centers.
In select cases, centers provide on-site technical assistance because
they possessed specialized expertise. In one instance, a company ° . "p/nesses
was seeking regulatory compliance information rather than P2
information, but because they were highly satisfied with what they
received they plan to use the center services again for other things.
Some states, including Texas and Washington, have requirements to
prepare P2 Plans so companies in those states are more highly motivated to seek out the
assistance of the P2Rx center.
TAPs also valued the networking opportunities provided by the P2Rx centers, such as
roundtables and conferences. The information resources they use and prefer depends primarily
on the type of assistance program for which they work, which determines the type of audience
they assist. State TAPs, who tend to work with larger businesses and industry, tended to prefer
the topic hubs, while both state and local TAPs found case studies useful, particularly when they
were of local or regional businesses. TAPs varied greatly in the ways in which they prefer to
receive information; some prefer face-to-face contact through conferences or seminars, some
would like information via e-mail or listserve, and some prefer to search for information
themselves by browsing trusted sources or simply using a general search engine.
Overall, there are several mechanisms P2Rx uses to try to understand what types of customers
access P2Rx information, and how they vary according to the method of outreach. In summary,
through interviews we learned that businesses access P2Rx information primarily by the Web
site, direct contact, or training sessions. Data available from Activity Measures show listserves
and newsletters going out to the largest number of customers, as compared to meetings and
direct contact. Information on the identity of these customers is kept by some centers, but is not
characterized across the network. The same is true for Activity Measures which shows number
of meetings and number of attendees at meetings. Web measures do not capture the type of
user, but do show a significant increase in usage over the years. 2007/2008 Behavior Survey
data showed what types of users were using the P2Rx Web sites, and how they were using
Evaluating the Effectiveness of the EPA PPIN Grant Program 26
-------
them. Activity Measures show that the majority of customers for direct contact are from
government, but the data shows a declining percentage of requests from government entities,
and an increasing percentage of business and industry requests. However, the 2007/2008
Behavior Survey shows more than twice as many TAPs using the P2Rx Web sites than
businesses. Overall, this information is inconclusive in responding to the evaluation question.
3.2.1 Types/Groups of Customers
Users of P2Rx seem to vary widely, and the universe is not well defined. For example, WRRC
commented that they have millions of user sessions each year and do not know who these
users are. Most of the information available is anecdotal, based on conference attendance and
requests for information.
The 2007/2008 Behavior Survey results give an indication of who might use the Web sites,
based on who chose to respond to the survey. Table 3-3 shows the types of organizations from
which survey respondents came, divided into business and TAP user types. Note that
respondents were allowed to choose more than one response, so the numbers in the table will
not add up to 100%. The total number of responses to this question is 275 for business, 281 for
TAPs, and 556 total. Among business respondents, manufacturing was the leading type of
organization at 41% of respondents, while among TAPs, state and local government were the
leaders at 35% and 28%, respectively.
Table 3-3: Type of Organization (% of responses)
Type of Organization Business TAPs Total
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
Other
11%
12%
14%
41%
12%
6%
5%
4%
13%
9%
35%
28%
5%
6%
11%
11%
7%
33%
10%
24%
21%
22%
9%
8%
8%
6%
23%
Source: 2007/2008 Behavior Survey
Other types of customers identified by centers during interviews included:
• International organizations, such as Baja California Roundtable and Canadian groups
• Local governments implementing P2 and doing technical assistance
• University representatives, government site personnel, utilities, or other industry as end
users
• EPA regional staff
Evaluating the Effectiveness of the EPA PPIN Grant Program
27
-------
P2RIC commented that they are increasingly seeing customers who are not TAPs (e.g.,
institutional end users such as universities, businesses referred through the Green Building
Council or business development centers, and manufacturers referred through the MEP).
Businesses
Business respondents indicated a variety of preferences for the types of products they find
useful. Some businesses prefer to access information via the Web sites and listserves, while
others have relied on direct interaction or on site assistance and training. Other representatives
mentioned webinars, emails, and print materials as useful P2Rx products, although preferences
varied. Several pointed to networking opportunities, such as conferences, meetings, or
byproduct synergy groups as the most useful functions of the centers.
Table 3-4 summarizes specific ways that business center representatives interviewed use P2Rx
centers. These businesses were selected by the centers and as such, tended to have been the
recipients of direct technical assistance. They may not be representative of most businesses.
Table 3-4: Uses of P2Rx Centers by Interviewed Business Representatives
Business Representative Uses of P2Rx Center
University ES&H Director
Naval Air Station
Cabinet Maker
Aerospace Parts Supplier
Ski Area Developer
Jewelry Maker
Networking, technical assistance, promotion of local and national events,
best practices, audit information
Information on chemicals and MilSpec, networking opportunities, analytical
tools, case studies, success stories
Participant in byproduct synergy project, direct contact
Event promotion, direct technical assistance
Regulatory compliance assistance for Spill Prevention, Control, and
Countermeasure Rule.
Direct technical assistance, topic hubs, vendor selection information
Focus group participants from the business community used the P2Rx center in various ways,
including as a portal to start a search, to find case studies and success stories, and to find out
about conferences and networking opportunities. They felt that the centers are not looked at as
part of EPA, and so are more trusted because a business can ask a question of the centers
without fear of inspection or fines.
Technical Assistance Providers
TAP participants in our interviews were mostly from state assistance programs, and all had at
least some familiarity with their local center or with P2Rx as a whole. These participants also
primarily worked with larger businesses or with entire industries, rather than with small
businesses. Differing from the interviewees, the TAPs who participated in our focus groups were
split—about half of the participants were from state programs, and half were from local or
county programs. The state TAPs who participated in the focus groups tended to work more
with larger, industrial businesses, while the local TAPs worked primarily with small businesses.
Evaluating the Effectiveness of the EPA PPIN Grant Program
28
-------
3.2.2 Outreach Methods
Of the four main types of assistance activities for which data were available, listserves and
newsletters consistently reached the greatest number of clients, followed by meetings, with
direct contact reaching the fewest clients. Figure 3-1 below illustrates the data available from
Web and Activity Measures for January 2006 to June 2007. The data over time indicates that
distribution of newsletters has grown steadily since January 2006, while meeting attendance
declined slightly. Direct contact with clients was steady over 2006, but jumped by almost 1,000
clients in the first half of 2007. This increase is partially attributable Peaks to Prairies, who
contributed 1,173 of the total 2,234 direct contacts in the first half of 2007 due to a change in
their reporting practices.
Figure 3-1: P2Rx Centers' Clients Served by Type of Assistance
Types of Assistance Provided, January 2007 - June 2006
Meetings
Listserves
Newsletters
Direct Contact
l
I
l
I
l
i
l
D Jan - Jun 2007
• Jun - Dec 2006
D Jan - Jun 2006
1000 2000 3000 4000 5000
Number of Clients Served
6000
The remainder of this section discusses customer access to various outreach methods,
including Web sites, TAPs, direct contact, and workshops, trainings, and regional meetings.
Via Web Site
In the 2007/2008 Behavior Survey, businesses were asked, "How did you utilize the [P2Rx]
website(s)? (Check all that apply.)" Responses are summarized in Table 3-5. Over half of
business respondents to the survey indicated that they had attended a training, workshop, or
conference, or had reviewed Topic Hub information.
Evaluating the Effectiveness of the EPA PPIN Grant Program
29
-------
Table 3-5: Business Utilization of P2Rx Web Sites
Use
Attended a training, workshop, or conference promoted by a P2Rx Center
Reviewed Topic Hub information (or other P2Rx website information)
Asked for pollution prevention technical assistance
Contacted a technical assistance provider
Requested additional resources from a P2Rx Center
Other (please specify)
Number of responses
Business
55%
51%
12%
10%
9%
14%
203
*Note that since more than one answer could be chosen, the numbers will not add up to 100%.
Source: 2007/2008 Behavior Survey
Note that this survey asked only how users were using the Web sites to reach a TAP, request
assistance, or attend a training, so was not able to fully answer the evaluation question.
Web Measures data shows a quadrupling of total combined usage of the P2Rx Web sites over
the past 4.5 years - from 637,000 in 2002 to 2.8 million by the end of 2006. Examining the
specific resources that are used on the Web, libraries consistently had the vast majority of user
sessions in all three six-month periods of Web Measures data analyzed. In fact, the number of
user sessions for the library is approximately an order of magnitude greater than any other
resource during each of the six-month periods from January 2006 to June 2007. A large
number of these sessions are the result of hits via a search engine and not through the Center
Web site. Sector Resources were typically the second-most accessed Web resource, followed
by Topic Hubs, News and other resources not necessarily in that order in all years or for all
centers. Although they are most frequently used, the libraries do not receive the highest
usefulness ratings in either the 2004 or 2007/2008 surveys (see Table 3-8). This is possibly due
to the fact that many of the library user sessions are the result of search engine links, and users
may not be aware that the resource they use comes from the library.
As noted in the P2Rx Administrative Documents, "It is recognized that each regional information
center is unique in terms of audience served, maturity of program, information model, and a
number of other factors. These differences make direct comparison of centers based solely or
primarily on their Web site statistics of questionable value." At this time it is not possible to
definitively answer "Who uses the P2Rx centers?" relative to the center Web sites. Wthout the
use of more advanced tracking tools such as identifying questions upon entry to the Web site, or
registration, the Web site user community is unknown. WRRC's Web use logs indicate that 20-
30 percent of users have their center Web site bookmarked, rather than coming from a link or a
search using Google or another search engine. Also, there are very few Google referrals to the
center home page; most customers are linked directly to content pages from Google.
Registration for Web site access can be a useful source of information about a center's Web
site users, but may be an obstacle to attracting users as well. One center Director commented
that if a Web site can fulfill a compelling need for services that businesses want, businesses
may use the site even if they are being asked to register. If such a requirement does not exist,
Evaluating the Effectiveness of the EPA PPIN Grant Program
30
-------
the center can offer "some identified business need" to encourage registration, such as
information on or assistance with general regulatory compliance, pretreatment requirements for
P2, or area source rules for small businesses. Non-regulatory motives may include a reduction
in insurance premiums or the desire to reduce one's carbon "footprint." For certain, if the center
sees a new regulation coming, they can safely assume that companies will need help.
Via TAPs
TAP respondents to the 2007/2008 Behavior Survey used the P2Rx Web site to provide their
clients with training, workshop, or conference information, or to provide other technical
assistance or Topic Hub information. TAPs were asked, "What information did you use to
improve or influence your clients' understanding of environmental practices to reduce pollution?
(Check all that apply.)" The responses are summarized in Table 3-6.
Table 3-6: TAP Utilization of P2Rx Web site
Information Used
Provided training, workshop, and/or conference information
Provided information on technical assistance providers
Provided Topic Hub information
Provided vendor information
Other pollution prevention technical assistance. If so, please
specify.
Number of responses
% of TAPs
59%
48%
38%
24%
57%
208
*Note that since more than one answer could be chosen, the numbers will not add up to 100%.
Source: 2007/2008 Behavior Survey
Again, note that the survey asked only how users were using the Web sites to reach a TAP,
request assistance, or attend a training, so was not able to fully answer the evaluation question.
Two TAPs reported having worked with a center outside of their own regions because of that
center's expertise in a particular area. These TAPs both indicated that they worked with their
own centers almost exclusively through their regional roundtables; however, they had used
information resources from other centers because of those centers' specific expertise or focus.
One TAP indicated that she does not go straight to her own center for information; rather, she
begins her search at the main P2Rx page, navigates to the topic hub in which she is interested,
and then contacts the center responsible for creating that topic hub for more information and
help. Many TAPs in both the interviews and focus groups indicated that they tend not to
distinguish between the resources of their own regional center and those of the entire P2Rx
network, although a few work only with their own regional center and never utilize the rest of
P2Rx.
Evaluating the Effectiveness of the EPA PPIN Grant Program
31
-------
Via Direct Contact
Information is available from the semi-annual Activity Measures that quantifies or characterizes
the recipients of direct technical assistance from the centers. Figure 3-2 below shows the
distribution of direct contacts with the centers by type of institution during 2006 and the first half
of 2007. The majority of customers are from government, but the data shows a declining
percentage of requests from government entities, and an increasing percentage of business and
industry requests. It is interesting to note that the jump in business requests in the first half of
2007 was largely attributable to Peaks to Prairies, accounting for 429 of the total 562 "Business,
Industry" clients, or 76 percent of requests. This was due to the fact that prior to 2007, the
Peaks to Prairies reports had not included technical assistance inquiries fielded by the center
Director, but only by the staff.
Evaluating the Effectiveness of the EPA PPIN Grant Program 32
-------
Figure 3-2: P2Rx Centers' Direct Contact with Clients, January 2006 - June 2007
Direct Contact with Clients, January - June 2006
(Universe -1,345 Clients Served)
Other
Individual 2%
12%
Consultant
4%
Business
17%
Direct Contact with Clients, June - December
2006 (Universe -1,335 Clients Served)
Individuals
10%
Governments
45%
Direct Contact with Clients, January - June 2007
(Universe - 2,234 Clients Served)
Individual
12%
Other
Educational
Institution,
University
8% ConsultanT
Business,
Industry
25%
*From Activity and Web Measures, 1st Half 2006, 2nd Half 2006, 1st Half 2007.
Evaluating the Effectiveness of the EPA PPIN Grant Program
33
-------
Via Workshops, Trainings, and Regional Meetings
The number of people reached through meetings and workshops is tracked in the semi-annual
activity measures, but their identities are not tracked. The following table and figure display the
known information about the number of customers reached through workshops, trainings, and
meetings.
Table 3-7: Meetings and Workshops Organized by P2Rx Centers
1st Half
2006
Number Meetings Organized
Attendees at Meetings
34
716
2nd Half
2006
41
1084
1st Half
2007
61
811
Source: Activity Measures
Figure 3-3: Meetings and Workshops Organized by P2Rx Centers
1200
1000
800
D Number Meetings
Organized
• Attendees at
Meetings
200
1stHalf06 2ndHalf06 1stHalf07
Source: Activity Measures
Evaluating the Effectiveness of the EPA PPIN Grant Program
34
-------
3.3 Usefulness of P2Rx Products
Evaluation Question 3: What products, services, and content do businesses need to make a
decision/consider the adoption ofP2 practices?
a) Are the centers providing the right products, services, and content to businesses to promote the
adoption of P2 practices?
Evaluation Question 4: What products, services, and content do TAPs need to promote businesses'
adoption of P2 practices?
a) Are the centers providing the right products, services, and content to TAPs to promote the adoption of
P2 practices by businesses?
Much of the information provided by interviewees and focus groups addressed content more
than services from P2Rx. On the question of products and services, the response from
business and TAPs during interviews was overwhelmingly positive, but not so from focus group
participants.
Businesses expressed three main priorities in their information needs:
• Up to date technical content that is easy to find on the Web
using search engines and is organized by process or topic in
addition to by sector
Information on cost of alternatives, viable substitutes, product- Savm9 in orma ion.
Businesses and
TAPs emphasize a
need for up-to-date
content and cost
specific vendor information, military prohibitions, and case
studies and success stories.
• Services to help them with the cost and technical aspects of implementing P2, ways to
address or preclude regulatory requirements, and networking opportunities.
TAPs expressed three main priorities in their information needs:
• Up-to-date information with working links;
• Help with marketing and "selling" P2 to their customers; and
• Assistance with measurement of P2 effectiveness and cost savings.
Ultimately, and outside of the purview of P2Rx, most TAPs indicated that their greatest needs
were for more financial resources and more staff to be able to respond to increasing customer
demands.
According to our interviews, focus groups, and the 2007/2008 Behavior Survey, both TAPs and
businesses find different P2Rx resources and products useful. Table 3-8 below presents the
percentages of active users of P2Rx who rated major P2Rx Web-based resources as "useful" or
"very useful" In the 2007/2008 Behavior Survey. Active users were defined for this analysis as
all respondents who did not respond "unaware of - do not use." The percentage of respondents
who are considered Active users is included to provide context for the proportion of total users
that are active vs. inactive.
Evaluating the Effectiveness of the EPA PPIN Grant Program 35
-------
Table 3-8: Percent of Active Users Rating Resource as Useful or Very Useful
Resource
Businesses TAPs
% Active = %of % Active = %of
Useful/Very Respondents Useful/Very Respondents
Useful who are Active Useful who are Active
Total %
Active =
Useful/Very
Useful
Topic Hubs
P2 Programs Directory
News
Industry Sector Information
Case Studies
Listserves
Library
Rapid Response
P2 Results system
Vendor Database
87%
84%
83%
77%
67%
81%
70%
72%
79%
57%
69%
65%
75%
65%
58%
66%
56%
41%
40%
50%
81%
73%
71%
76%
71%
70%
65%
61%
52%
40%
80%
73%
86%
74%
73%
72%
70%
49%
63%
57%
84%
78%
77%
77%
76%
76%
67%
66%
64%
46%
Source: 2007/2008Behavior Survey
On average, a greater percentage of TAPs considered themselves "active users" (i.e. did not
select "unaware of- do not use") than were businesses. Business respondents who were active
users rated almost all resources more favorably than did TAPs. For both respondent groups,
News had the highest number of active users, while Rapid Response had high numbers of
inactive users in both groups. One interesting note is that the Library did not receive particularly
high ratings in the 2007/2008 Behavior Survey, but the library pages receive the highest Web
traffic by far according to P2Rx Web Measures. This is likely because much of the library traffic
comes from a search engine directly to a PDF document. The user does not always know they
are accessing the P2Rx library. Further, users who access library resources via search engines
may bypass the page with the survey link on it and are therefore may not have taken the survey.
Results from the 2007/2008 Survey provide an illustration of how useful survey respondents
found different aspects of the P2Rx Web sites:
• Topic Hubs, P2 Programs Directory, and News fall within the top four resources with
active users in both categories.
• These three resources were also the most highly rated in the 2002 Customer
Satisfaction Survey, along with the Library.
• TAPs considered Industry Sector Information and Case Studies more useful than did
businesses.
• Listserves and Rapid Response ranked similarly for both audiences.
More detailed information on Resources can be found in Appendix C.
3.3.1 Center Perspective on Products, Services, and Content
The centers determine the needs of their users in a variety of
ways. They most often consult assistance providers for
guidance regarding the needs of industry. More than one
center said they get feedback from state P2 coordinators about
the needs of TAPs, their encounters with businesses, and what
they wish they had known about in the field, which all indicates
Mosf centers consult
assistance providers and
state P2 coordinators to
learn about their
customers' needs.
Evaluating the Effectiveness of the EPA PPIN Grant Program
36
-------
information needs and priorities. For online resources, the centers look at the Web measures
and Web trends to determine demand for specific products. They occasionally conduct surveys
(often with funding other than PPIN), or look at the results of others' surveys including the two
recent national surveys. One center mentioned sitting in on MEP meetings with the lean
manufacturing group to learn their needs; likewise, NEWMOA is planning focus groups with
businesses in the future. Conference evaluations are also widely used to determine user needs,
as is communication with advisory boards, roundtables, etc. Listserves are used to solicit
feedback about what customers want to see at conferences, and high attendance at certain
repeat conferences (e.g., "P2 101") indicates an ongoing need for introductory training. Other
means of determining information needs include tracking environmental blogs and news to stay
informed of emerging issues, interacting with trade associations, and developing peer
roundtables with industry groups. Clearly, the centers have many means to gather information
about the needs of their customers. However, when looked at system-wide and in an evaluative
sense, this information is anecdotal and informal, and often documented minimally or in meeting
minutes.
NEWMOA conducts an Annual Survey of Assistance and P2 Program Priorities, which goes out
to the Northeast Assistance and P2 Roundtable Steering Committees made up of
representatives from the states. The survey asks about sector priorities, multi-sector technology
priorities, topical priorities, and service and outreach priorities. NEWMOA uses the results of this
survey to inform the topics they will focus on for both their regional roundtable activities and
their PPIN-funded work. For example, there is a direct link between the priority sectors reflected
in recent survey results and the topic hubs they have developed.
WRRC incorporates their needs assessment into existing functions, such as monthly
conference calls and semiannual meetings with their states. GLRPPR uses a variety of
methods to determine needs, including asking their steering committee, polling listserves,
monitoring blogs, and requesting feedback at conferences.
Zero Waste identifies customer needs using immediate feedback from workshops, P2Planner
information, feedback from a P2 program or local government program or pretreatment
coordinator, and/or incoming industry requests for information. Through workshops, Zero
Waste supplies the customer with information and follows up, partly in the form of assistance,
and partly as an evaluation. The process begins when Zero Waste speaks with assistance
providers to identify needs and develop tools, and conducts a workshop. This is followed by
broader market penetration via the Web. It is more difficult to determine need from Web usage;
however, Zero Waste is unique in that they know who is using certain of their Web tools
because they have a registration process for users which allows them to track their progress.
P2RIC annually consults their service provider directory of 133 programs to assess needs, and
identifies needs through semiannual roundtable meetings. Peaks to Prairies requests feedback
from state P2 coordinators, and also measures the use of their Web products to determine
demand. WRPPN uses a variety of methods to assess needs, including conference evaluations
and listserves. They also utilize their Advisory Board to annually compile a list of over 100
possible topics for trainings, eventually narrowing their scope to six trainings with 18 sessions
per year.
Evaluating the Effectiveness of the EPA PPIN Grant Program 37
-------
The model used by PPRC is similar to Zero Waste's model. They work with the regional
roundtable and a few TAPs and businesses to determine needs and develop an approach. The
center also conducts evaluation and gets outcome information at the same time as they conduct
their activities, and so can more effectively target the information they will provide in the future.
3.3.2 Needs of Businesses
Business respondents varied widely in their needs. Many indicated a need for cost information
to help them consider implementation of P2. One business expressed an interest in more topic
hubs or case studies pertaining to service sectors. Although service sector oriented information
exists, this respondent perceived it was lacking. Many businesses in the interviews and focus
groups requested vendor or product information. Respondents want the centers to be non-
commercial and unbiased, yet also provide reliable and up-to-date vendor information.
Additionally, many participants in both forums indicated a preference for process-specific
information, rather than the sector-based organization of the current topic hubs. Table 3-9
summarizes the needs that business representatives interviewed have with regard to P2Rx
centers.
Table 3-9: Interviewed Business Representatives' Reported Needs
Business Representative Reported Needs
University ES&H Director
Dry Cleaner
Paper Mill Environmental Manager
Naval Air Station
Cabinet Maker
Aerospace Parts Supplier
Ski Area Developer
Jewelry Maker
Networking to help him implement changes, case studies for
administrators and decision-makers
Web site key words, terminology, and units of measure that are
better suited to an industry audience, information that is easier to
find.
Information on return on investments, and alternatives and
product substitutions, product-specific information about
alternatives for toxic chemicals.
Information on prohibitions on products, chemical and process
change information, non-hazardous substitutes, and vendor
information on reputable recyclers. Prefers information organized
by topic or process (not by sector).
Information to help them keep their air emissions low enough to
avoid having to obtain a Title 5 permit under the Clean Air Act.
A non-commercial, unbiased information source with information
on products and case studies of successes. Also would like to
network with others.
Information on construction and demolition waste, purchasing,
recycling, and reuse options. All case- and state-specific.
Information about the waste effects of different processes
involved with jewelry making (i.e., plating, polishing, casting),
names of "green" vendors, and the waste streams of local
vendors. She thinks there would be tremendous response to a
jewelry-specific topic hub.
Evaluating the Effectiveness of the EPA PPIN Grant Program
38
-------
Focus group participants indicated that they need information organized by technology or
process, as well as organized by sector. This is consistent with comments in the interviews as
well. The example offered during the focus groups was for aqueous parts washing, as it is often
categorized in the automotive sector but in fact is applicable to other sectors as well.
An effective Web search engine was also cited as a critical element, allowing searching in ways
that cut across industries and technologies. Most people use Google for searches, and P2Rx
resources need to be accessible in this way. (Note that WRRC actively works at getting their
site to the top of Google results.) An idea was proffered to incorporate a knowledge
management system similar to what technical support sites have (i.e., log every question that is
asked and offer the ability to search the answers). This is analogous to ideas that came from the
center staff to better use the Rapid Response questions and answers as a way to identify the
needs that exist.
Further, industry focus group participants discussed the importance of new innovations and
technology, and the importance of having up-to-date information on the Web sites. It was noted
that new, cutting-edge technologies are often developed by for-profit companies, and that P2Rx
does not provide this type of information. The discussion continued by saying that, in fact,
center users are not even interested in an endorsement of certain technologies by the centers,
but only in an acknowledgement or a list of all the technology options. It was noted that other
government agencies and programs provide such listings (e.g., Occupational Safety and Health
Administration and HazComm).
3.3.3 Needs of TAPs
TAPs discussed three priorities in their information needs to promote P2 for their customers.
One involved their need for current, up-to-date information. Several TAPs indicated that
sometimes the topic hubs are outdated and contain links that do not work. Most TAPs said that
P2Rx provides the right content in the topic hubs, but that it needs to be more current. They
expressed a desire to be updated on new technologies, regulations, and best management
practices, and indicated that they need this information without commentary or opinion. Some
TAPs suggested broader use of the listserves or more emphasis on providing news in order to
supply TAPs with the "latest and greatest" P2 technologies and best practices.
Another priority involved help with promoting P2, and
the need for help in persuading businesses to adopt P2
practices. TAPs indicated that the best way to "sell" P2
to businesses is to present money-saving options and
incentives. Although P2Rx would not likely be able to
provide grants, the centers could provide more
information on the availability of grants or other funding, loans, rebates, or tax incentives to help
businesses make equipment changes or encourage their adoption of P2.
TAPs noted three priority needs:
Help with measuring effectiveness
Another priority was assistance with measurement and estimation of the effectiveness of P2,
which relates directly to the marketing priority by aiding in demonstrating the benefits of P2. A
Evaluating the Effectiveness of the EPA PPIN Grant Program 39
-------
few TAPs suggested the creation of a calculator-like tool that would show the financial savings
or waste reduction that could be achieved by implementing P2 practices, or simply a collection
of common metrics. Several of the TAPs said that a measurement tool would help them not only
with assistance to businesses, but with their own grants, as they are required to report on their
outcomes. TAPs suggested that P2 professionals have done some of these standard
calculations, but the formulas, metrics, and conversion factors cannot be found in one place. As
an example, if a TAP is working with the hospitality industry, he would like some common
metrics to qualify the results. For example, if a hotel switches to low-flow shower heads and
there are a known number of guest rooms, what is the reduction in water use? Providing a
conversion factor such as water use reduced per occupied room with a low-flow shower head
would be beneficial to TAPs. This information is available, but is difficult to find, and could be
useful to have in a topic hub setting or built into the P2 Results tool.
Several TAPs in the focus groups indicated a desire for increased searchability of the P2Rx
center Web sites and resources. These TAPs suggested a move toward a more user-friendly
format, involving tagged keywords and searchable resource text; however, these features are in
fact currently available on the Topic Hubs, although not all centers have provided the option to
browse by keyword. There seem to be two different strategies TAPs employ when seeking
information: either browse through a few trusted sources, or conduct a broad search via Google.
For those with a couple of frequently used sources, P2Rx and the local center are often
included in those sources; however, those who prefer a broad-based Internet search often do
not use the centers unless their resource is on the front page of a Google search.
Although not a need that can be met by P2Rx, a sentiment commonly expressed by TAPs was
that their biggest need is increased staff time and monetary resources to handle an increased
demand for their services. The centers can help ease the TAPs' burden by providing
information resources and conducting more time-consuming research so that TAPs can use
their time for direct assistance. Other suggestions from TAPs included more focus on vendors;
a shift in focus from production industries to the service industry; and production of templates for
documents such as checklists, guidance documents, and best practices.
Several TAPs indicated that the networking opportunities offered by P2Rx, such as roundtables
and regional meetings, conferences, Web conferences, and listserves, were P2Rx's most useful
services because they allow for sharing of information between states and provide opportunities
for TAPs to stay in touch with other professionals in the field.
Some TAPs utilized the topic hubs frequently, while others found them to not always have up-to-
date or relevant information. Several TAPs, particularly those from state programs, indicated
that they organize their own assistance by sector; therefore, the topic hub format is useful and
appropriate. Many TAPs thought the topic hubs were a helpful place to begin research on a new
sector with which they were unfamiliar. However, the organization by sector or the choice of the
sectors themselves was criticized by some TAPs, particularly local TAPs who tend to work with
smaller businesses, because the information in the topic hubs tends to be focused on larger-
scale industries or sectors they did not feel applied to their work.
Evaluating the Effectiveness of the EPA PPIN Grant Program 40
-------
TAPs also indicated a preference for case studies of successes that can be shared with the
businesses they serve because they provide evidence of a real-life application and are often an
effective tool in promoting P2. Some TAPs felt that local case studies were more useful,
especially when the business that participated in the case study was willing to become a
"mentor" to other businesses interested in implementing a similar P2 practice.
TAPs stressed the importance of regional or even state-specific information, and indicated that
the value of the centers lies in their ability to provide regionally relevant materials. Some focus
group participants indicated that even with the P2Rx centers' regional focus, the usefulness of
their information is limited at a certain point because TAPs need specific information based on
state or local regulations. Additionally, TAPs emphasized the need to tailor information to their
customers' needs and to the specific regulations under which they are governed.
One TAP mentioned that she was required to write a grant in order to begin working with a new
industry sector, and indicated that P2Rx information resources are useful for putting directly into
her grants. Others also mentioned that they use P2Rx information internally as well, and refer
people within their assistance organizations to P2Rx resources.
The TAPs interviewed generally fed results data into the P2 Results tool, and had mixed
opinions on its usefulness. Some, particularly state TAPs, found the tool to be a useful way to
track their outcomes for their own state or grant requirements. Others found that P2 Results did
not quite fit the measures they were already collecting or did not allow them to tailor the tool to
their own state or program. A few commented that the tool would be more useful if
accompanied by additional assistance in calculating the numbers to feed into the tool (e.g.
calculators or metrics, see Section 3.3.3 for more details). Thirty percent of TAPs who
responded to the 2007/2008 Behavior Survey rated the P2 Results tool as "Useful" or "Very
Useful."
Interviewees were split on their use of Rapid Response resources; some rely on Rapid
Response for help with either quick questions or more in-depth research, while others do not
utilize Rapid Response because they prefer to do their own research. One TAP who had used
the Rapid Response mentioned that she has in the past submitted inquiries that have gone
unanswered. Twenty nine percent of TAPs who responded to the 2007/2008 Behavior Survey
thought that Rapid Response was "Useful" or "Very Useful."
Evaluating the Effectiveness of the EPA PPIN Grant Program 41
-------
3.4 Outreach to Customers
Evaluation Question 6: Are the centers effective in reaching businesses through TAPs?
a) What do centers do to reach businesses through TAPs?
Evaluation Question 7: Are the centers effective in reaching businesses directly?
a) What do centers do to reach businesses directly?
In most cases the centers conducted outreach to businesses and TAPs in much the same way.
For many, the TAP/center interaction worked well, leveraging resources and expertise to
effectively serve the business community.
Businesses we spoke to felt that the information on the center Web sites was geared much
more to people in the regulatory or academic field than to people like themselves. In terms of
the way the centers function, they serve as more of a "connector" to others, including to the
TAPs. In some cases the business will contact their state TAP first, who will put them in touch
with the center for particular assistance. There was some fluidity in terms of who goes where
first, but what did come across is the difference in the type of services provided by the centers
versus the TAPs. The centers try to be a broad brush, reaching an entire region with information
delivered at a distance and through others. The TAPs, at the state or local level, are situated to
use the information the centers provide to do on-site or other direct assistance. There seems to
be a clearly defined set of roles for each. There are some cases where the center was a more
palatable organization for a businessperson to contact, as they are not (generally) associated
with a regulatory agency. Regardless of how the centers were accessing their clients, many
businesses felt that the centers were not promoting themselves effectively.
TAPs help the center determine what is needed by businesses, and they are also the delivery
mechanism to businesses. TAPs expressed that businesses are better served by a state or
local entity to provide direct assistance to end users. A few TAPs felt that if the centers were to
focus their efforts on serving businesses directly it would be a duplication of the services TAPs
already provide. Having the centers work directly with business overlaps with the TAPs' work
and may compete for funds and customer base, thereby making both organizations less
effective. The TAPs want to have the support the centers can offer them in providing services in
their state.
The findings show that historically the centers as a whole were not designed to help business
directly, but rather they were designed to help TAPs. When a center does work with business,
outreach is done in one of several ways: (1) through TAPs who may refer them; (2) in response
to incoming questions or requests for information (i.e., reactively); (3) through planned
conferences or workshops or (4) done no differently than other general outreach. For example,
workshops may be geared to a particular industry, but both businesses and TAPs are likely to
attend. Overall, direct work with businesses is most often done through or for groups.
Information on the Web sites does not all appear to be specifically applicable to businesses.
However, there is a wealth of information there and businesses clearly use it, as indicated in the
2007/8 Behavior Survey.
Evaluating the Effectiveness of the EPA PPIN Grant Program 42
-------
To help answer how the centers are reaching their customers, one can look to the 2007/208
Survey results in Table 3-10. Most users were referred to P2Rx by a conference or meeting, a
colleague, or an e-mail. These results are consistent with the 2002 Survey, which showed that
most users were referred by a colleague or through e-mail or a listserve. Table 3-10 shows that
more TAPs were referred by colleagues than were businesses. In total, 352 businesses (end
users) and 317 TAPs (intermediate users) responded to the 2007/2008 Behavior Survey.
Table 3-10: Referrers to P2Rx
Referrer
Assistance program
Brochure or newsletter
Colleague
Conference or meeting
E-mail message or listserve
Link from another website
Other (please specify)
Search engine
Vendor
Number of responses
Business
4%
4%
15%
37%
15%
7%
8%
11%
0%
309
TAPs
4%
1%
29%
33%
14%
6%
9%
5%
1%
295
Total
4%
2%
22%
35%
14%
7%
9%
8%
0%
604
Source: 2007/2008 Behavior Survey
The centers are more effective at reaching businesses through TAPs. For determining
effectiveness, however, they may have more success getting results directly from businesses to
the extent they can because it is more difficult to measure the effectiveness of reaching
businesses through TAPs. In terms of reaching businesses directly, many centers felt that
limited funding inhibited them from doing what was needed to effectively change their focus
from TAPs to businesses, in terms of determining needs and doing one-on-one assistance.
any case it seems too soon to evaluate the centers' effectiveness in reaching businesses
directly, as this is a new requirement as of last year.
In
3.4.1 Center Perspective on Customer Outreach
For the most part, centers' direct work with businesses is on an ad hoc basis because the
centers have been designed to provide information to TAPs, and also because of limited time
and staff. Centers primarily react to businesses' requests, and do not often specifically target
businesses with information. Of course, the centers do not all operate the same, as was
described previously. Some work more with businesses than others, and some have more
concertedly shifted to doing so in the past year. We can summarize these relationships based
on the interview findings as follows:
• NEWMOA is constrained by their relationship with their member states, and
consequently has difficulty transitioning to performing direct technical assistance
because it will overlap with their state's activities.
• WRRC still focuses primarily on TAPs, and works with businesses directly through Ask
Rudy and other Web related services.
• GLRPPR works primarily with TAPs, but has worked directly with a few businesses on
an ad hoc basis.
Evaluating the Effectiveness of the EPA PPIN Grant Program
43
-------
Zero Waste works with both TAPs and businesses, and reported reaching over 1,000
businesses in brief interventions, but working extensively with around 30 TAPs.
P2RIC still works mostly with TAPs, but is transitioning to working with businesses
because of changing EPA requirements.
Peaks to Prairies makes more effort to reach end users than TAPs; however their end
users are not traditional businesses but primarily schools, parks, tribes, residential
owners, designers and builders.
WRPPN focuses their efforts on TAPs, and only interacts directly with businesses
through the conferences the center sponsors.
PPRC calls their relationship with TAPs a "wholesale/retail" relationship, wherein PPRC
is the wholesaler of information, and the TAPs are the retailer.
A vast majority of centers said that they were set up to function Many centers wer& originally
by working through TAPs (as information providers), and
several interviewees (both industry and centers) said that the
materials on the Web sites were geared to this audience.
Many centers found it difficult to change their focus from TAPs
... . . , . . . A i_ presents some difficulty.
to businesses, and to move forward in customer outreach
designed to provide
information to TAPs, and
changing their focus to
directly assisting businesses
without the funds to hold discussions, focus groups, etc., with
businesses to determine their needs. In the past they have frequently relied on state TAPs and
others to inform them of their users' needs. Two centers noted that while their focus is on state
TAPs, everything they do is accessible to everyone via the Web. P2RIC said that their focus on
TAPs is changing due to changing PPIN grant requirements, but noted that the new RFP for
center grants seems not to emphasize a focus on working directly with businesses.
WRRC reported doing nothing different for the two audiences. They reach both audiences the
same way (i.e., making sure they're on top of the Google search return page). While PPRC
develops information with the final user (business) in mind, they use it to help the intermediate
customer reach them. Zero Waste directs their work at both TAPs and businesses with equal
effort. They see their relationship with TAPs and businesses as a continuum with a feedback
loop. Additionally, they commented that they have more success looking directly at business for
results; it is harder to measure the effectiveness of reaching businesses through TAPs, but they
are starting to do that now. Businesses know of them largely through referrals from TAPs, and
most of their interactions with business are with groups at workshops or from people who phone
in for assistance.
PPRC aligns with the work of TAPs, often coming in after the TAPs to provide longer-term
value-added services. They also provide assistance when the facility is uncomfortable having a
state agency provide service. Additionally, PPRC has some specific technical expertise that the
TAP may lack. PPRC stays in contact the business community, associations, MEPs, etc., as a
way to stay apprised of needs. Similar to other centers, however, they do not document this
very well and have no quantifiable information on results of these activities. The center also
leverages the fact that industry obtains information from other sources such as vendors and
service providers, and they work with these entities by arming them with information. For
example, industrial launderers are not traditionally considered a TAP, but in fact they visit many
businesses collecting rags and coveralls. If information is given to the industrial launderers, they
Evaluating the Effectiveness of the EPA PPIN Grant Program 44
-------
can provide it to their customers and make suggestions (e.g., replace solvents). Industrial
launderers have a vested interest in some of these activities because they're handling the
materials. This is an interesting example of a supply chain model.
P2RIC also plays a role in assisting both TAPs and businesses. Sometimes they act as a "sales
rep" for TAPs when they work with businesses, informing them about the services of the TAP or
sending their materials to business contacts. Conversely, TAPs use P2RIC to review materials
and make them more relevant. The TAP'S Web site links to P2RIC, encouraging their customers
to visit the P2RIC Web site. Similar to all other centers, P2RIC's interaction with business is
often through groups, including industrial councils and organizations. They also network with a
variety of individual business representatives who teach classes or speak at events.
Peaks to Prairies develops toolkits and other products for businesses and other end users that
P2 coordinators or TAPs then distribute. In some cases the centers are making business-
oriented materials and tools available, and this is how they are reaching businesses. However,
these materials can be used by either audience segment, and are often passed through TAPs to
businesses.
One center works with business associations, but as a source of information and reviewer of
content rather than as a customer. They interact with business as a customer only if the
business comes to them through Rapid Response. Another center said that their contact with
final customers was through conference attendees only. One center was established by a
stakeholder group that explicitly wanted an information hub, and was never intended to provide
direct technical assistance. An interesting point was made that the regional centers simply do
not have enough resources to provide direct technical assistance to a multi-state region.
In general, center-run conferences are not specifically promoted to businesses, although they
are welcome to attend. Center listserves generally include industry individuals, but the
percentage of recipients represented by business is unclear.
This is not to say that work with end users does not occur across the network, and with success.
Such direct work with businesses, when it does happen, is typically through business groups,
alliances or associations, and services are provided for the group as a whole. The information is
disseminated to groups rather than to individual businesses. For example, GLRPPR played a
role in fostering the development of the Midwest Product Stewardship Council (MPSC). The
center provided technical support for the MPSC, and had the group present at their conferences
and serve on the organizing panel. Peaks works with tribes (as end users) through the National
Pollution Prevention Roundtable's Tribal P2 Workgroup. Some entities, such as pretreatment
programs, are both an intermediate and a final customer in that they may use information from
the centers to reduce their own releases, or pass it on to their dischargers as a technical
assistance service. Zero Waste reported working closely with pretreatment coordinators.
Although some centers work more directly with businesses to provide services, they all rely on
TAPs at various stages of the process. For the most part, TAPs provide critical information to
direct the center to what is needed, and they are also the delivery mechanism.
It seems to be too soon to evaluate the centers' effectiveness in reaching businesses directly,
as this is a new requirement last year. In general we heard that for groups they work with
Evaluating the Effectiveness of the EPA PPIN Grant Program 45
-------
regularly, centers may get feedback from facility managers indicating a product was useful.
TAPs have also provided positive feedback (mostly anecdotal and verbal) indicating program
effectiveness.
3.4.2 Outreach to Businesses
Businesses expressed various opinions about the effectiveness of the centers at reaching them
directly, as well as of the effectiveness of working with TAPs. One business said the state TAP
acts as conduit between him and the center and he often sees them both at the same event,
while others preferred to work with centers directly, or frequently consult TAPs first and are
sometimes referred to the centers. Several businesses mentioned a preference for working with
the centers rather than with government TAPs, due to their management's discomfort with
working with an entity they perceive to be a regulatory agency, even in an assistance role. On
the other hand, some businesses indicated a tendency to first reach out to a government TAP
for assistance, after which they may be referred to the centers. Table 3-11 summarizes ways in
which the P2Rx centers reached the business representatives interviewed.
Table 3-11: Center Outreach to Interviewed Businesses
Business Representative Method of Contacting Center
University ES&H Director
Paper Mill Environmental
Manager
Naval Air Station
Cabinet Maker
Aerospace Parts Supplier
Ski Area Developer
Gets information both from the center directly and from TAPs. Had the
impression that the center may be less effective at reaching customers
who are geographically further away.*
Was referred to the center by his TAP. Attends training and networking
events if it coincides with another trip. Suggests webinars for those in
remote locations. Views centers as coordinator rather than technical
resource.
Worked with the center directly after receiving information about them
during an inspection. Does not work with TAPs. Thinks that many are
not aware of the centers' services.
Referred to the center through the state agency. Typically goes to the
state agency first, as they implement the state P2 planning
requirements and serve as a TAP. Sometimes will work directly with
the center, as when they received on-site technical assistance in use of
laser spray guns. Thinks that many are not aware of the centers'
services.
Worked directly with center because the state TAP is also the
implementerof the state P2 planning requirements and management
preferred to work with a group perceived as independent and working
for the best interest of the company.
Connected with the center via Web search — linked to the center
through resources on their associated university's Web site.
* This respondent is co-located with center
Three out of five business focus group participants had been in contact with a TAP. They
indicated that generally this contact involved them reaching out to the TAP for information,
rather than the other way around. However, the businesses represented in the focus groups
were not generally the industries typically served by TAPs, i.e., several were involved with
management consulting or information technology. Marketing was a major topic of the focus
Evaluating the Effectiveness of the EPA PPIN Grant Program
46
-------
groups; specifically, the centers' failure to do enough of it. One commenter said that the "P2Rx
has a marketing problem" and that they only found the Web site by stumbling upon it. Another
person goes to a lot of EPA conferences and had never heard of P2Rx or been referred to it.
They said many of these conferences are "preaching to the choir" and that the centers needed
to market their resources more widely, since it seems they most often only promote at their own
events. One participant suggested advertising in Business Week or Fortune Magazine in order
to reach businesses directly.
3.4.3 Outreach to TAPs
TAPs help the center determine what is needed by businesses, and they are also the delivery
mechanism to businesses. This is working effectively. The TAP participants in the interviews
and focus groups generally questioned the logic of having the
centers work directly with businesses. TAPs see the centers as a
source of information and networking aimed at TAPs, and thought
that the centers working directly with business would be
duplicative of the TAPs' efforts.
TAPs stress the importance
of one-on-one contact with
businesses to successfully
promote adoption ofP2
practices.
TAPs emphasized the need for one-on-one contact with businesses in order to be successful in
promoting P2, particularly with smaller businesses. They indicated that a local or state
organization may be more effective than a regional one in reaching businesses simply for
reasons of proximity.
Evaluating the Effectiveness of the EPA PPIN Grant Program 47
-------
3.5 Attribution of Outcomes
Evaluation Question 5: To what extent do businesses that use P2Rx centers change their practices or
adopt P2 practices?
a) Do businesses use P2Rx information as a resource to solve pollution problems or to help them adopt
P2Rx practices?
b) What causes/influences businesses to seek P2Rxinformation?
Evaluation Question 6b: Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers [through TAPs]?
Evaluation Question 7b: Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received [directly] from P2Rx centers?
Attribution of outcomes is perhaps the most challenging task for P2Rx. EPA sponsored the
recent nationwide 2007/2008 Behavior Survey to try to establish a connection between the
center activities and outcomes. While informative, this survey is not reliably representative of
P2Rx users, and does not isolate the effects of P2Rx activities from other influences on P2
adoption. The survey results certainly provided an indication that P2Rx is affecting short-term,
intermediate, and long-term outcomes; however, with the existing information, P2Rx is not
currently able to establish a definitive connection between their activities and behavioral or
environmental outcomes. However the evidence from the interviews and surveys shows that
there is some connection between the information provided by P2Rx and increased awareness
and long-term outcomes.
TAPs who were interviewed or participated in focus groups had mixed opinions on the
measureable impacts of P2Rx on outcomes. Nearly all TAPs consulted were concerned about
the difficulty in relating P2Rx resources or information to outcomes because of the number of
steps between the information and the outcome. Additionally, TAPs asserted that P2Rx is only
one of the many information sources they utilize in assisting customers, and to attribute an
outcome to a particular piece of information is very difficult. However, many TAPs indicated that
they thought P2Rx contributed to intermediate outcomes in the sense that P2Rx resources
serve to educate TAPs.
All the businesses we spoke with during interviews found at least certain center services
extremely helpful, the focus group participants less so. Measurable environmental outcomes
were cited by most of the interviewees, although not all of the outcomes were well documented.
Many of the outcomes would be characterized as awareness and behavior change outcomes
more than quantifiable waste reduction. The causes for business in both the interviews and
focus groups to seek P2Rx information are the same as those that cause them to seek any P2
information, i.e., cost savings, need for safer substitutes, responding to customer demand,
regulatory compliance, etc.
Behavior changes were captured in the 2007/2008 Survey, in the question asking what actions
were taken due to information obtained from P2Rx. Table 3-12 shows that TAP respondents
attribute more actions to P2Rx information than do businesses, while businesses were more
Evaluating the Effectiveness of the EPA PPIN Grant Program 48
-------
likely to report that no process changes occurred. Note that some items in this list are changes
in awareness (i.e., short-term outcomes) while others are actions taken (i.e., intermediate-term
outcomes).
Table 3-12: Actions Taken as a Result of P2Rx Information
Actions Taken
Changed a pollution prevention process or practice (i.e.,
implemented pollution prevention)
Identified a pollution prevention opportunity
Purchased new process equipment to prevent pollution
Implemented energy conservation measures
Switched to renewable energy
Contacted a vendor
Implemented material or waste recycling system
Changed handling of waste or emission
Installed pollution control equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken
Other (please specify)
Number of responses
Business
28%
45%
20%
22%
8%
14%
36%
17%
8%
2%
21%
7%
188
TAPs
53%
65%
23%
27%
4%
28%
38%
36%
11%
10%
14%
15%
188
Total
41%
55%
22%
25%
6%
21%
37%
27%
10%
6%
18%
11%
376
Source: 2007/2008 Behavior Survey
Additional responses to the Survey indicated that respondents thought P2Rx had impacted their
awareness. Table 3-13 lists the answers to the following 2007/2008 Behavior Survey question
about intermediate outcomes, which is worded differently for business and for TAPs:
Business: "[P2Rx] helps me to improve my awareness of environmental practices to
reduce pollution. (Check only one answer.)"
TAPs: "Access to [P2Rx] information has improved my ability to provide technical
assistance to my clients. (Check only one answer)".
Table 3-13: 2007/2008 Behavior Survey Responses: Improved Awareness/Ability to Provide Assistance
Improved Awareness/Ability Business TAPs Total
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
Number of responses
29%
41%
15%
0%
0%
14%
306
20%
44%
22%
1%
1%
10%
290
25%
43%
18%
1%
1%
12%
596
Both business and TAPs respondents were favorable about the usefulness of the Web site.
Seventy percent of business respondents agreed or strongly agreed that the site improves their
awareness of environmental practices, and 64% of TAPs respondents agreed or strongly
Evaluating the Effectiveness of the EPA PPIN Grant Program
49
-------
agreed that the site improves their ability to provide technical assistance. Very few respondents
disagreed, though some were neutral, and a substantial number did not use the site.
Further, respondents reported on long-term outcomes (pollution reduction) as shown in Table
3-14, which summarizes answers to the question which were asked differently for businesses
and TAPs:
Business: "Please identify whet her you reduced or eliminated pollutant(s) as a result of
[P2Rx] use. (Check all that apply.)"
TAPs: "Please identify whether your client(s) reduced or eliminated pollutant(s) as a
result of[P2Rx] use. (Check all that apply.)"
Table 3-14: Pollutant Reduction as a Result of P2Rx Use
Pollutant Reduction Business TAPs Total
Reduced non-hazardous waste (solid waste)
Eliminated non-hazardous waste (solid waste)
Reduced hazardous waste or hazardous materials
Eliminated hazardous waste or hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
Other (please specify)
Number of responses
26%
7%
29%
7%
7%
3%
12%
5%
20%
31%
19%
197
36%
13%
37%
13%
15%
5%
26%
10%
34%
8%
16%
208
31%
10%
33%
10%
11%
4%
20%
7%
27%
19%
18%
405
TAP respondents generally indicate more attribution of outcomes to P2Rx use than do
businesses. Of note are the high responses from both audiences for "Reduced non-hazardous
waste" and "Reduced hazardous waste or hazardous materials." A fairly high percentage of
businesses responded "None" as compared to TAPs, and high percentages in both audiences
responded "Don't know."
In addition to behavior changes and waste reduction, respondents to the 2007/2008 Behavior
Survey also indicated cost savings outcomes as a result of P2Rx information use. Table 3-15
presents the answers to the following 2007/2008 Behavior Survey, which was asked differently
for business and TAP respondents:
Business: "Has using [P2Rx] or the action(s) taken above resulted in cost savings to
you? (Cost includes time as well as dollars.) (Check only one answer.)"
TAPs: "Has using [P2Rx] or the action(s) taken above resulted in cost savings to your
client? (Cost includes time as well as dollars.) (Check only one answer.)"
Evaluating the Effectiveness of the EPA PPIN Grant Program
50
-------
Table 3-15: Cost Savings to Business - Directly or Through TAPs
Cost Savings Businesses TAPs Total
Do not know
No
Yes
Number of responses
49%
27%
24%
224
63%
9%
29%
219
56%
18%
26%
443
Source: 2007/2008 Behavior Survey
Table 3-16 presents answers to a question asked of TAPs in the 2007/2008 Behavior Survey:
"Has using [P2Rx] saved you time or money in serving your client(s) ? (Check only one
answer.)"
Table 3-16: Cost Savings to TAPs
Direct Cost Savings TAPs
Do not know
No
Yes
Number of responses
41%
10%
49%
22 1
Source: 2007/2008 Behavior Survey
About a quarter of respondents indicated cost savings for businesses, either directly or through
TAPs. However, large percentages of respondents did not know whether using P2Rx had
resulted in cost savings. Half of TAP respondents reported cost savings for their own
organizations due to use of P2Rx.
3.5.1 Center Analysis
Centers currently collect outcome information in various ways. Various anecdotal sources are
utilized, such as speaking with advisory boards and using conference evaluations, to indicate
that a center's end users are achieving waste reduction.
Zero Waste collects information about outcomes through the P2Planner on its Web site. When
someone registers on the Zero Waste Web site, they start to provide information (name,
business, Toxic Release Inventory (TRI) IDs, etc.) to the online assistance tool, and what P2
activity they are going to try to implement. Two years later, Zero Waste can go to another
database (e.g., a regulatory database like TRI) and find the user (e.g., with TRI ID). In TRI,
facilities may report their P2 activities on the Form R, and the center can use this information to
see what they changed. The P2Planner has 5,000 potential reporters; around 500 use the tool,
roughly 90 had results, and some of those the results were related back to center activities.
These data can be further filtered to see what services the P2 reporters used and what changes
they made (e.g., Zero Waste does not count the reduction if the reporter reduces something
different from what they say in their P2 plan). This quality control system results in the collection
of measures information from a tiny fraction of entities, but those changes resulted in reduction
of huge amounts of waste. It is important to note that P2 Planning is a requirement in the state
Evaluating the Effectiveness of the EPA PPIN Grant Program
51
-------
of Texas, and this requirement drives much of the activity on the part of industry that enables
Zero Waste to collect outcome data from the P2 Planner.
Peaks to Prairies collects information about outcomes by finding out from state P2 coordinators
what information they're using and how they're using it. Peaks to Prairies expects their toolkit for
schools to change behavior and prevent hazards, by helping schools inventory materials and
develop new purchasing plans. For parks, they look at park-level environmental management
systems. For builders, architects, and homeowners, they look for implementation of green
building strategies. They do not document these outcomes, and unless they have direct
interaction with their audience, they do not know what is being done with the information.
Several centers, including GLRPPR, WRRC, and PPRC, maintain online archives of questions
and answers that have been addressed by the center. GLRPPR's online Help Desk Archive also
includes the question "Was this answer helpful to you?" and any comments. PPRC tracks the
requester's name so that follow-up may be conducted. While information in this format is difficult
to browse through, an online archive captures valuable information and reflects the level of
effort expended in responding. An archive such as this, whether for Help Desk or Rapid
Response, can be used to share technical information and resources, as well as for
measurement purposes
The centers can track users' visits to their Web site looking for information, but gauging what
users do after that is almost impossible. One center attempts to document outcomes by
measuring the "stickiness" factor on their Web site, which is a measure of how long people stay
on the site. (The average time is getting longer.) They could potentially monitor the people who
make requests through Rapid Response/Ask Rudy, and then go back to them and ask for
information. But if the requesters come in through the Web, the center does not know who they
are. The center does know that their Web site has 4 million users with a lot of repeat users, that
30 percent of their users have their site bookmarked, and that when their site is down they get
numerous complaints Another center gauges effectiveness by the fact that their posted calendar
events seem to draw attendees, and that grant announcements are read by Web users.
One popular way to measure changes in awareness or knowledge is to do pre- and post-testing
at conferences, but of course this only evaluates the value of that particular event. Last year
WRPPN's conference evaluations asked what percentage of outcomes the attendees could
attribute to information from the conference. The answers to this multiple choice question were
evenly split between "0-10%" and "over 10%," with no one responding "0."
Short-term and intermediate outcomes are more clearly influenced by the centers, but final
outcomes are extremely difficult to assess. One center commented that it is rare to give a
customer a case study or other piece of information and have them make a change because of
it. This center believes they have had increased awareness, but this belief is based on indirect
and circumstantial evidence. In general, many centers feel it is not useful for EPA to ask them
to document implementation of P2, as there are multiple steps between the centers and P2
implementers and to attribute the implementation to P2Rx information is very difficult.
Evaluating the Effectiveness of the EPA PPIN Grant Program 52
-------
There is a continuum of costs and benefits involved in collecting measures information. For
example, it is extremely resource intensive to follow up on one-on-one technical assistance by
calling or e-mailing the recipient. However, when the recipient does respond, the center typically
gets deep, compelling results. A workshop uses fewer resources than one-on-one assistance,
but is still resource intensive. Zero Waste did a follow up study of workshop attendees that
documented $250,000 in savings and tens of thousands of pounds of waste reduced.
Zero Waste commented that obtaining data from EPA databases such as for Toxic Release
Inventory reporting, is the least resource-intensive means of collecting results information, and
perhaps the most efficient, but also only tells part of the story. This center actively mines state
and local databases for information to put into P2 Results. Zero Waste spent 2 weeks trolling
databases and found tens of millions of pounds of reductions that can be associated with
activities reported in P2 Plans. Results from databases have less depth, but are more in line
with EPA's strategic goals.
Many outcomes are illustrated through case studies prepared by P2Rx centers, and some of
these outcomes can be attributed to the assistance of a P2Rx center. A sample of case studies
provided by Zero Waste showed outcomes in the form of waste reduction, cost savings, energy
conservation, and the use of P2 to achieve regulatory compliance. These outcomes were
attributed in the case studies to attendance at workshops, use of the P2Planner, and use of
information provided by Zero Waste. PPRC provided several case study examples where the
center leveraged their EPA PPIN funding with funding from the National Institute of Standards
and Technology and the state to do on-site "Lean & Environment" activities. Using value stream
mapping and kaizen (improvement) events as well as other methods, they were able to achieve
and document significant long term outcomes. If such information is well documented, case
studies can provide a compelling sample of outcomes related to P2Rx assistance.
In order to get measurable results, it is helpful for centers to engage with users who are willing
to commit to follow-up. For example, in an exchange with businesses at a meeting, it was
suggested that one center develop a Green Meeting Guide for hotels. The center now requires,
as a precondition to any engagement, a commitment to measure; they are willing to develop the
guide if the hotel with which they are working can provide measures on what they saved by
using the guide.
The primary means used by the centers to get feedback of any kind comes from state and local
agencies, Advisory or other Boards, EPA regions or regional roundtables, and/or steering
committees. Centers have direct access to and ongoing relationships with these groups of
individuals, and it is straightforward to solicit information from them. In contrast, soliciting
feedback from end users is more difficult (unless they are affiliated with these groups), and
therefore is quite limited. This is an important factor when considering the goal of working
directly with end users because a robust mechanism for eliciting feedback from them, e.g., on
outcome measures, does not currently exist. Ironically, the near-term priority to make P2Rx
information resources more directly accessible to businesses in the Strategic Plan is to
"dialogue with assistance programs to define additional ways of modifying P2Rx resources so
that they are more directly accessible and usable by business." Maintaining this role of
assistance providers runs counter to the intention to work directly with business. Further, in
Evaluating the Effectiveness of the EPA PPIN Grant Program 53
-------
order to effectively redirect center efforts toward business, a concerted effort must be made to
determine the needs of this audience.
Many different factors can affect outcomes. For example, greenhouse gas reduction is
becoming a priority for many businesses, yet it is impossible to attribute this focus to a particular
resource or to years of information disseminated to the public. One center cited an example of a
facility that began to make environmental changes as a result of the owner's death from cancer.
The center had been providing them with information for a year, but that one event changed
their willingness to act. It is also important to remember that businesses get information from a
number of other sources as well as the centers, such as consultants, vendors, or industry
associations, making it difficult to attribute changes to any one source.
Many centers said that measuring outcomes and causality is extremely difficult, and is more so
given the current funding levels for the programs. Most centers said they felt they were effective
and have had an impact. They cited a variety of anecdotal evidence, or said that this was an
impression based on discussions with customers and other associates.
3.5.2 Business Analysis
Businesses reported a
number of outcomes, some
of which can be directly
attributed to the assistance
The business representatives we interviewed reported a variety of
outcomes, including waste management of electronics,
implementation of dry cleaning with CO2, reuse of expired dyes,
reduction in pesticide use, increased production due to
implementation of a lean environmental system, and preparation of
an SPCC Plan. Many of the outcomes involved awareness and
behavior changes, such as training of sprayers that led to reduced reworking of the painted
products. Some of the reported outcomes can be related directly back to the assistance of the
center, while for others the cause of the change is more ambiguous.
Many business representatives noted that their decision to implement P2 is most often
motivated by cost savings, although for some P2 is part of their job as environmental managers
at their companies, while for others P2 is simply a personal interest. Many say that in general,
businesses can most successfully implement education of their employees, as purchasing
technologies is a harder sell with their management. Often businesses now are interested in
being "green," but they do not want to lose money or be on the "bleeding edge" of technological
advances, as one participant pointed out. Table 3-17 summarizes the use of P2Rx information
and outcomes attributable to P2Rx center services from business representatives interviewed.
Evaluating the Effectiveness of the EPA PPIN Grant Program 54
-------
Table 3-17: Interviewed Businesses' Reported Outcomes Attributable to P2Rx
Business Representative
University ES&H Director
Dry Cleaner
Paper Mill Environmental
Manager
Naval Air Station
Cabinet maker
Aerospace Parts Supplier
Ski Area Developer
Jewelry Maker
Reported Outcomes
Obtained information on many things including volume minimization for
chemical labs, replacements for toxic products, identifying strategic
planning needs for implementing sustainable strategies. Waste reduction
outcomes have not been documented because he lacks the resources.
Starts with the center site when interested in making an environmental
change, and the site offers or links to information and contacts that help him
implement changes.
Reduced tons of waste through byproduct synergy/waste exchange. The
center networked the companies together. One outcome was sending
expired FDA-approved dyes to a farmer to add to pesticides to see where
they sprayed and reduce overspray. There is no benefit to the company in
reporting results.
Have made some good chemical substitutions, but have not documented
outcomes.
Lean environmental system provided a payback of over $1 million per year.
Substantially increased production efficiency; going from 900 cabinets per
day with three shifts, to 1 ,000 cabinets per day in two shifts without laying
off any employees. Results of their value stream mapping/kaizen are
documented in Washington Department of Ecology document (pub #06-04-
024, final report #07-04-033).
With information from the center, placed paper in their spray booths to avoid
having to clean booths with hazardous solvents, improved maintenance on
their electrostatic spray guns for greater efficiency in paint application.
Used materials provided by the center to prepare an SPCC Plan.
Eliminated metal plating process because of the information she received
from the center.
3.5.3 TAP Analysis
The primary sentiment from the TAPs concerning outcomes was that they know businesses are
changing their practices because they see it on the ground and conduct follow-up with the
businesses they have assisted. However, all but one TAP thought it would be nearly impossible
to attribute certain outcomes to P2Rx or a particular center. One
TAP did indicate that she uses only certain sources for her
information and can remember which source certain ideas came
from, and could ultimately attribute an outcome to P2Rx if she got
the idea from its resources; however, this opinion was the
minority. All of the TAPs thought P2Rx had been effective in
educating and increasing the knowledge of TAPs, but most felt
TAPs know that businesses
are changing their practices,
however they consider it
difficult or impossible to
attribute results to specific
information.
that the information resources were not necessarily causing the reductions; rather, site visits
and training were really causing outcomes. TAPs expressed concerns about drawing a
relationship between information resources and having businesses implement P2, and indicated
that there seemed to be too many steps between the information and the outcome to be able to
tie the outcome back to the information.
Evaluating the Effectiveness of the EPA PPIN Grant Program
55
-------
The TAPs primarily have anecdotal information from their experience and conversations with
businesses to inform their opinions on outcomes. All TAPs seem to do some form of follow-up
with their customers, but they vary in the information they are able to collect. One TAP indicated
a success rate of almost 100 percent in getting information back on the follow-up visit 6 months
after assistance, while others seem to have less formal follow-up procedures or less success in
collecting information. This information collection is important, because the TAPs usually must
report their outcomes for grant purposes.
TAPs interviewed had mixed opinions on the use of the P2 Results aggregator tool to track their
measurements. The focus group participants were largely critical of the tool's usability and
usefulness, while some in the interviews thought the tool was helpful for tracking their results.
All TAPs indicated that they need to collect and track their results in order to measure their own
effectiveness and justify their programs' existence; however many felt that the current tool was
difficult to use. Others suggested that the problem did not lie with the tool itself, but rather that
the difficulty was in collecting results information from businesses, particularly small businesses
who may not track their reductions and changes. Some also expressed concern that the P2
Results tool seemed geared toward bigger businesses and industry, and that it might not be
appropriate for reporting the results from small business because of the metrics used.
Evaluating the Effectiveness of the EPA PPIN Grant Program 56
-------
3.6 Performance Measurement
Evaluation Question 8: What performance measures do P2Rx centers need to collect in order to
demonstrate/ document client behavior and other outcomes?
a) How would the centers that currently do not have the capacity to make/gather the desired
measurements build their capacity?
b) How can the information best be collected based on available technology, funds, and survey approval
requirements?
Centers currently collect an array of data and other information to help them assess their
accomplishments. All of these tools provide useful information and are elements of a
comprehensive measurement strategy. There are limitations associated with each, but they all
have value. Overall, information about behavior change and other outcomes for end users is
less available than for intermediate users. Information collection includes:
• Activity Measures, providing quantitative data on the activity output of each center. In
doing so, the number of people reached through these activities (e.g., workshops, etc) is
provided. The affiliation of people reached through direct assistance is captured.
• Web Measures, providing information on Web site activity, such as number of user
sessions.
• Surveys, both national and regional, providing quantitative information from respondents
on usage patterns, satisfaction, outcomes, and needs, among other things. Surveys are
rarely done by the centers, and so the two national surveys are most heavily relied upon.
• Case studies, telling a story about particular outcome results achieved as a result of
center intervention. While these are anecdotal, they are perhaps the only currently
existing mechanism for linking center activity to quantifiable outcomes.
• User needs assessments, conducted in various ways and frequencies, providing
feedback on topics the centers should focus on. Typically such assessments appear to
focus on subject matter more than on ways to convey it or satisfaction of users, and also
on intermediate users more than on end users.
• Pre- and Post-Tests at conferences and workshops, capturing information about the
satisfaction of that particular event at meeting outcome objectives.
• P2 Results captures outcome and activity information for purposes of reporting.
It is difficult to estimate the true impact that P2Rx centers have on key outcomes for TAPs or for
end users. The design of this evaluation is not able to isolate the affect of P2Rx on the target
populations from other factors that may influence the adoption of pollution prevention practices
beyond the educated opinions of key informants in the study. Moreover, the informants do not
represent a systematic or large enough sample of intermediate or end users to allow us to
characterize the average opinions of the target populations. However, we have used the
testimony of informants and selected survey results to frame meaningful hypotheses and gain
insights about whether and how P2Rx centers fulfill the promise of the logic models.
Information for this section of the report came primarily from centers. Referring to the logic
models, one respondent commented that the centers cannot get to "outcomes" because they
get stuck at the "customer" column of the logic model. In other words, if they cannot clearly
identify their customers, they will never know their outcomes. This is an effective
Evaluating the Effectiveness of the EPA PPIN Grant Program 57
-------
characterization of the outcomes challenge for the Web-based aspects of the centers. In
summary, recipients are known for listserves, workshops, mailings, and direct assistance; while
they are unknown for Web access. Even when recipients are known, in some cases they are
not compiled and there is often no feedback loop to determine effectiveness. One center
commented that now that they have a common logic model, it would be a good exercise to use
it to discuss performance measures with the network as a whole.
Clearly, centers' capabilities with respect to measurement fall on a wide spectrum. One center
appeared to be doing next to nothing to get information into P2 Results (though they did
previously), commenting that it takes active effort and that they do not currently have the staffing
to do this. The evaluation findings show that P2 Results is being used primarily as a repository
for P2 data collected from P2Rx centers, TAPs, and others, and the cause of the reported P2
results is not captured in the data system by many centers.
In contrast, PPRC said that now more than ever they rely on the P2 Results database. As they
do more and more outreach, they will collect more and more information. They do this by
making an agreement with the company up front. They may offer an incentive such as waiving a
registration fee for a webinar. They then follow up 6 months after the webinar to find out if
anything changed. They may waive the fee in exchange for a single number or data point,
although they may not always be able to tie the data back causally to the outreach. They view
this as an in-kind contribution from the company; center services are free, so they ask for
something in return. The fact that Washington requires P2 Planning provides a more highly
motivated audience for this type of involvement.
The idea of asking for an in-kind contribution of measurement information as a condition of
engagement was raised by two centers. It was also noted that this could be a disincentive to
businesses that come to them but can also get the information elsewhere for free.
Zero Waste conducts performance measurement while they are engaged in an activity, rather
than conducting surveys. The center feels they lose credibility when they conduct surveys after
the fact, and are more effective when evaluation is done as in integral part of the assistance
process. While they categorize all of their activities as technical assistance, they broadly define
them as workshops, online assistance, telephone assistance, etc., rather than direct, on-site
technical assistance. Zero Waste's approach is to deliver P2 training workshops in partnership
with a TAP to meet an identified need. During the workshop, attendees participate in exercises
to identify their primary waste streams and potential projects, and to practice selling the idea to
their company. In this way, Zero Waste is performing a needs assessment for their purposes.
Attendees make commitments at these workshops and Zero Waste works with their partners to
follow up, usually with a phone call. The follow up is also an opportunity for the center to better
target their assistance, as well as to collect case studies. One important distinction of their
program is that P2 Results8 data is collected as part and parcel of the technical assistance
service they provide. It is an exchange the business has with the center in return for assistance.
The information that the center develops from this interaction can then be more widely
P2 Results is a data system for collecting, analyzing, and presenting the results of P2 for the regions
and the country.
Evaluating the Effectiveness of the EPA PPIN Grant Program 58
-------
distributed using the Web. The center is interested in sharing its method with other centers to
try to capture greater results.
EPA Region 9 (WRPPN) has some unique challenges in that they have numerous local P2
programs, including 35 distinct air districts in California alone, from which they would like to
collect information. Many haven't been measuring results at all, and many don't communicate
with each other. Further, a vast majority of these programs are not funded by EPA and so are
not required to submit measurement information. Some other regions may include states that
are predominantly funded by EPA.
It was felt that Rapid Response could be better utilized for measurement. For example, a
request for information that comes in via e-mail gives the center an e-mail address they can use
for follow-up to gather outcome information. Another idea is to do a pre- and post-test series of
questions when assistance is provided. A protocol for the entire network would be helpful here
for consistency. NEWMOA has a good precursor data gathering system that would gather data
from Rapid Response to go into P2 Results. They also have a tool that makes it easier for TAPs
and others to track their information and feed into P2 Results.
To define a good measure for efficiency, EPA needs to determine what the priority is for their
grant expenditures - for example, whether intermediate outcomes are sufficient, or if long-term
outcomes are the priority, whether it is important to see reduction in all states, for both small and
large business, etc., or just to see gross size of reductions. If the latter, centers will focus
exclusively on the areas most likely to experience waste reductions.
Some TAPs suggested ways that they might be able to better capture the effect of the
information provided by P2Rx. One suggestion involved adding questions to their exit interview
or follow-up that address whether the business used P2Rx resources. Another suggestion
involved having the interns who work for the assistance program track the resources they use in
a bibliography, and measure how frequently they use P2Rx resources, for what purposes, and
to what end.
A TAP participant in a focus group suggested that as information providers, P2Rx should
benchmark themselves against industry standards by comparing themselves to other similar
information providers. Another TAP suggested having a box on the Web site that said "if you've
accessed information on this Web site, would you mind leaving your e-mail address so we can
contact you and see how you've used it?"
A comprehensive and effective evaluation mechanism that examines causality will need to
involve all three entities: the P2Rx center, the TAP, and the end user, eliciting information from
each of these groups.
Evaluating the Effectiveness of the EPA PPIN Grant Program 59
-------
4 Conclusions and Recommendations
The following conclusions and recommendations are based upon the facts and opinions offered
by key informants, existing data and documentation, and the professional knowledge of the Abt
Associates' evaluators. Strategies that have been used by centers with reasonable evidence of
success are included in the recommendations for others to consider. Other recommendations
are based on our expert judgement as applied to the data, facts and opinions provided through
the evaluation. Several recommendations address aspects of multiple questions and they are
repeated in the context of the evaluation question. The results are grouped together by broad
themes, as well as by that theme's corresponding evaluation question(s). The evaluation
questions are included in parentheses.
4.1 Center Operation Models (Question 1)
Evaluation Question 1
1) What model(s) do the P2Rx centers use to encourage exploration and adoption ofP2
practices?
a) How are centers currently operating and what model(s) are they using to disseminate
information?
b) Which of the centers' model(s) is most efficient in information sharing and/or delivery?
c) When comparing among centers, what opportunities for centers' information delivery to
become more efficient are observed?
d) What opportunities are there for centers to become more efficient in information delivery
by sharing tasks?
The centers use a wide variety of models to fulfill the mission of P2Rx. Variation exists in
organizational structure, physical location (e.g., co-located with a TAP, university, or regulatory
agency), size of staff, availability of resources, sources and use of funding, activity emphasis,
and customer base. With this variation, each center possesses different qualities and meets
their objectives in ways that are both unique and similar, depending on the element. The
evaluation is inconclusive as to which single model may be most efficient but indicates that an
eight-center network is an effective method to reach the network's objectives. P2Rx as an
interactive network does appear to strengthen the abilities of any single provider, but there are
also inherent inequalities. Overall, as an information "wholesaler" with responsibility for
compiling and providing information to more localized entities (e.g., TAPs), a regional model for
the centers does appear to be effective. The existence of a national network makes it possible
for the centers to deliver more and better information to their customers, as the centers can
focus on unique regional needs while tapping into national resources. The centers do have
strong and constructive relationships with organizations and individuals within their region.
Considerable efficiency has been gained over the years in centralizing certain functions in one
or two centers. Successful examples include the case study database supported by Zero
Waste, centralized IT support for certain activities and overall Web site maintenance, the
centralized management of the Programs Database, and of course the library at WRRC. Web
Evaluating the Effectiveness of the EPA PPIN Grant Program 60
-------
support and programming was cited as an obvious opportunity to centralize so that the network
capitalizes on the strength of certain centers' IT expertise. For the most part, this seems to be
working well and to be enthusiastically supported by center staff. One observation is that it is
not clear if or how a center is compensated for taking on a particularly labor-intensive task, or if
resource limitations could be a disincentive for doing so. Further, there may be an inherent
contradiction in sharing tasks in order to gain efficiencies. For example, if services or functions
are centralized in a particular center (e.g., library services or Web support), that center will be
using a greater portion of their resources supporting that centralized function. The result may be
that they are less able to do other functions, such as providing direct assistance to businesses,
or possibly to measure outcomes.
Recommendations
• P2Rx should continue to work together as a network of distinct programs, and should share
responsibilities where appropriate as well as centralize tasks where appropriate. Continue
to make well-reasoned decisions about what tasks are best suited to be done in a
centralized way. Specifically, there appears to be benefit in centralizing the functions for
collecting case studies, maintaining the Programs Directory, performance measurement,
and some of the coding responsibilities.
• It is an understandable concern that the centers spend considerable time communicating
with one another, an activity that could impinge on the time they have to spend reaching out
to end users. To some extent this is a necessary part of networking. However,
communication should be value-added and done efficiently. Suggestions include the use of
facilitators, limits on the number of conference calls, and setting clear objectives for each
meeting.
• Consider making the PPIN grant process more streamlined and/or award multi-year grants.
This would reduce the time centers spend on writing proposals and allow for longer-range
activity planning.
4.2 P2Rx Information Users (Question 2)
Evaluation Question 2
2) What groups/types of customers access P2Rx information ?
How do the customers vary according to method of outreach by the centers?
• Via the Web site
• Via TAPs
• Via direct contact (e.g., telephone, correspondence, or other communication)
• Via P2Rx workshops, trainings, and regional meetings
The evaluation characterized the groups of customers that access P2Rx information, but was
not able to determine how the groups or types of customers vary according to the method of
outreach by the centers. Some determinations about the customers of P2Rx can be made
based on the data reviewed for the evaluation. Information that would characterize the users of
the Web sites, TAPs, telephone or e-mail correspondence, or face-to-face sessions, either does
not exist or has not been collated.
Evaluating the Effectiveness of the EPA PPIN Grant Program 61
-------
Through interviews we learned that businesses access P2Rx information primarily via Web
sites, direct contact, or training sessions. Data available from Activity Measures show listserves
and newsletters going out to the largest number of customers, as compared to meetings and
direct contact. Information on the identity of these recipients is kept by some centers, but not
across the network. The same is true for Activity Measures, which show number of meetings
and number of attendees at meetings but does not capture the identity of attendees. Web
measures do not capture the type of user, but do show a significant increase in usage of the
Web site over the years. 2007/2008 Behavior Survey data gave an indication of the types of
users that were using the P2Rx Web sites, and how they were using them. Activity Measures
show that the majority of customers for direct contact are from government, but with a declining
percentage of requests from government entities, and an increasing percentage of business and
industry requests. In contrast, the 2007/2008 Behavior Survey shows more than twice as many
TAPs using the P2Rx Web sites than businesses. Overall, this information is inconclusive in
responding to the evaluation question.
Recommendations
• Use a consistent method of characterizing customers for all of P2Rx. Identification
according to the two groups used in the 2007/8 Behavior Survey is recommended, as it
addresses how the customer uses P2Rx and not just who they are. If an additional level of
detail is desired, affiliation can be used as subcategories.
• Rapid Response (and other telephone or e-mail-based on-demand services) offers a
valuable opportunity to capture, tabulate and analyze information on types of customers and
types of request. Centers should collect and analyze information on the identity of these
customers. Some centers already collect this information, but it does not appear to be
analyzed. Centers that do not currently capture this information should consider doing so.
• Characterize and analyze the identity of conference attendees. While the number of
attendees is captured for the P2Rx Activity Measures, the identity of these participants is not
examined in a comprehensive way.
• Make the Web sites more interactive so that centers can gather information to characterize
their users. Many centers suggested this. Ideally, this would be interactive so that users can
also indicate what product they are using. The most popular suggestion that came from
several centers was to place a pop-up on the Web site asking a single question, such as
what sector or audience segment the user belongs to. Their usage would then be tracked
using a cookie. This would allow the center to identify their user group, as well as track their
repeat usage. Wthout follow-up the centers still wouldn't know about impact, but they would
know about reach. This recommendation would, however, require the centers to invest more
resources in changing their Web structure, as well as tracking users.
• Require or offer registration for use of the site. This would provide the center with
information they could use to identify their users and follow up with them. Consideration can
be given to have registration for discrete parts of the site rather than the entire site.
Evaluating the Effectiveness of the EPA PPIN Grant Program 62
-------
4.3 Usefulness of P2Rx Products (Questions 3 and 4)
Evaluation Questions 3 and 4
3) What products, services, and content do businesses need to make a decision/consider the
adoption of P2 practices?
a) Are the centers providing the right products, services, and content to businesses to
promote the adoption ofP2 practices?
4) What products, services, and content do TAPs need to promote businesses' adoption ofP2
practices?
c) Are the centers providing the right products, services, and content to TAPs to promote
the adoption ofP2 practices by businesses?
Businesses expressed three main priorities in their information needs:
• Up to date technical content that is easy to find on the Web using search engines and is
organized by process, topic or technology in addition to by sector
• Information on cost of alternatives, viable substitutes, product-specific vendor
information, military prohibitions, and case studies and success stories.
• Services to help them with the cost and technical aspects of implementing P2, ways to
address or preclude regulatory requirements, and networking opportunities.
TAPs expressed three main priorities in their information needs:
• Up-to-date information with working links, maintaining the existing sector-orientation;
• Help with marketing and "selling" P2 to their customers; and
• Assistance with measurement of P2 effectiveness and cost savings.
While many centers do have well-established networks with TAPs and other stakeholders to
determine customer needs and whether the centers are meeting them, it is neither systematic
nor consistent across centers. Further, more feedback mechanisms exist for TAPs and
government entities than for industry. The information obtained from the evaluation, particularly
the interviews and survey results, indicates that the centers appear to be providing some of the
right products, services and content to businesses to promote P2, but the information is
anecdotal. There exist numerous opportunities to address customer needs further and to
provide more useful and appropriate products, services, and content.
Recommendations
• Make better use of steering committees, advisory boards, board of directors, and/or state or
regional roundtables or other stakeholder groups to solicit information about needs and
whether needs are being met by P2Rx. This should have two key components: (1) an
attempt should be made to do this more consistently across centers, possibly having a core
set of questions that are asked and an agreed-upon periodicity; and (2) questions should
include those regarding effectiveness, not just priorities and needs. This will primarily
provide information from intermediate users.
• If P2Rx does, in fact, want to be able to address more directly the needs of end users, a
consistent network-wide mechanism must be established to identify these needs.
Mechanisms for collecting information from end users may include:
Evaluating the Effectiveness of the EPA PPIN Grant Program 63
-------
o Collecting info from Rapid Response and other on-demand services as a way to
evaluate need. This opportunity should be exploited by all centers to characterize
the needs of the user community, and possibly hone in on repeat requests that can
be addressed system-wide.
o As a way to gauge the usefulness of products, and possibly in addition to collecting
more user information, use a pop-up window on the Web sites to ask a single
question of the user after they access a particular resource to learn if it met their
needs (e.g. "How useful was this fact sheet to you?").
o Conduct a survey or focus groups of a particular, well-defined, user group to
determine their needs.
o Use the recurring process implemented by Zero Waste, i.e., use assistance
providers to identify needs and develop tools, conduct a workshop, obtain
commitments for action from attendees, follow up six months later, and collect results
information. The information from the workshop is placed on the Web for broader
market penetration. This model can be used for either an end-user an intermediate-
user audience.
If the emphasis is, in fact, on providing services to end-users, the information on the center
Web sites needs to be redirected toward process rather than organized by sector based on
the findings of this evaluation.
If the emphasis is on providing services to TAPs, maintain the existing sector-orientation.
Add information that would help them persuade businesses to adopt P2, and a
measurement tool.
Expend adequate resources to keep information up-to-date on center Web sites.
4.4 Adoption of P2 Practices by Businesses (Question 5)
Evaluation Question 5
9) To what extent do businesses that use P2Rx centers change their practices or adopt P2
practices?
a) Do businesses use P2Rx information as a resource to solve pollution problems or to
help them adopt P2Rx practices?
b) What causes/influences businesses to seek P2Rx information?
The evaluation provided an indication that businesses use information and services from P2Rx
to change their awareness and practices, leading to environmental outcomes. Survey results
and case studies are currently the best source of information to examine this nationally.
Interviews for this evaluation identified several cases in which businesses claimed to have
changed practices as a result of using P2Rx information; however, this evidence is not well
documented. The evaluation did determine that although some businesses use P2Rx
information as a trusted resource for solving pollution problems, they use many other resources
as well, making it difficult to attribute results to P2Rx. The causes for business to seek P2Rx
information are the same as those that cause them to seek any P2 information, i.e., cost
savings, need for safer substitutes, responding to customer demand, regulatory compliance,
etc. Overall, the design of this evaluation was not able to isolate the affect of P2Rx on the
Evaluating the Effectiveness of the EPA PPIN Grant Program 64
-------
target populations from other factors that may influence the adoption of pollution prevention
practices.
Recommendations
• Continue to collect case studies. Consider compiling and tabulating them to show
cumulative cause and effect of center activities. However, reliance on information collected
as a result of direct assistance to end users will have limitations with respect to the number
of customers reached and the resulting quantity of outcomes.
• Perform follow up with the target audience to determine if approaches are effective at
making change happen. This can be follow up from Rapid Response-type intervention,
workshop attendance, or any other venue where you know the identity of the customer.
P2Rx should develop some standard protocols to be used by all centers.
• Collect outcome information as part of the activity to best engage the captive audience.
This can be done by collecting information from workshop participants or asking for a
commitment to action. This model is used by Zero Waste.
• Dedicate discrete resources to do follow up for activities. At their discretion, the centers
might consider hiring an individual to support the entire network as another shared activity.
4.5 Outreach to Businesses Directly and Through TAPs (Questions 6
and?)
Evaluation Questions 6 and 7
6) Are the centers effective in reaching businesses through TAPs?
a) What do centers do to reach businesses through TAPs?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
7) Are the centers effective in reaching businesses directly?
a) What do centers do to reach businesses directly?
b) Can any outcomes (short-term, intermediate, or long-term) be attributed to information
received from P2Rx centers?
While EPA has directed the centers to shift their focus to targeting businesses and other end
users, the centers are implementing this to a greater or lesser extent across the network. There
does not seem to be a clear delineation between how centers reach businesses directly as
opposed to through TAPs. Some of the centers are TAPs as well. For many, the TAP/center
interaction worked well, leveraging resources and expertise to effectively serve the end user.
The evaluation concludes that centers more effectively serve intermediate customers than end
users for the following reasons:
• Not all centers have the resources (technical and/or financial) to provide assistance directly
to end users. In many cases it is considered out of scope or not a part of their stated
mission. In these cases, or unless the emphasis is removed from serving the TAP
community, additional resources would seem to be needed for such a retooling.
Evaluating the Effectiveness of the EPA PPIN Grant Program 65
-------
• Serving end users would appear to duplicate the work of the TAPs. This should be
considered in the context of efficient use of government funds, as well as the specific niche
intended to be filled by each of these two sets of service providers. While there appears to
be some reluctance to work with TAPs when they are connected to a regulatory agency, this
does not seem to be a major concern. One of the P2Rx centers also resides within a
regulatory agency.
• In general, the interplay between the centers and TAPs appears to be highly effective, i.e.,
the centers as researchers and information providers, and the TAPs to provide end users
with useful information for implementation.
• Businesses clearly see benefits in using P2Rx services. This would seem inevitable with a
resource as rich as P2Rx. The question should be asked if that means the centers should
actively direct their efforts towards end users, as opposed to it being a fortunate, but
passive, side benefit.
• The network does not have adequate tools in place to characterize the end user audience,
determine its needs, or determine if P2Rx is providing the right products, services and
content to them. Obtaining this information from end users is more difficult than from
intermediate users, and may require an ICR.
• All centers currently focus on intermediate users, but only a portion of them direct their
efforts to any degree to end users.
• Similarly, the recommendation from the 2002 Customer Satisfaction Survey was to focus on
the primary audience of technical assistance providers.
In general, the centers are more effective at reaching businesses through TAPs. For
determining effectiveness on long-term performance measures, however, it is simpler to get
results directly from businesses as opposed to trying to measure the effectiveness of reaching
businesses through TAPs. If the centers would like to better reach end-users, findings
suggested the centers should better promote their services in business-oriented forums. The
fact that the centers were a neutral, non-regulatory entity encourages businesses to seek them
out.
Conclusions regarding Outcomes are addressed under Question 5 above.
Recommendations
• Continue to focus on intermediate users rather than on end users. Work with end users as
a secondary audience as appropriate for each individual center.
• If interested in reaching the end user, promote and market center resources more widely
and beyond the usual venues. For example, advertise in business journals or on business
Web sites, attend business meetings and workshops in addition to or in place of P2Rx-
sponsored events.
4.6 Performance Measurement (Question 8)
Evaluation Question 8
Evaluating the Effectiveness of the EPA PPIN Grant Program 66
-------
8) What performance measures do P2Rx centers need to collect in order to demonstrate/
document client behavior and other outcomes?
a) How would the centers that currently do not have the capacity to make/gather the
desired measurements build their capacity?
b) How can the information best be collected based on available technology, funds, and
survey approval requirements?
Centers' capabilities to measure performance vary widely, however it is clear that additional
capacity is needed to establish a robust network-wide system of measurement. Individual
centers have different capacity to perform measurement based on their staffing and other
factors. For example, Zero Waste and PPRC have better capacity to obtain outcome
information from Texas and Washington due to the P2 Planning requirements in those states.
The P2 Results system is considered to be useful by some stakeholders, but not by others.
Centers have conducted surveys, both in person and online, of various audiences for various
purposes over the years. Conference pre- and post-surveys can provide an indication of the
effectiveness of that particular event and perhaps the future needs of the audience. However,
questions about outcomes can only be answered prospectively, unless follow up is conducted.
The usefulness of national or regional surveys such as those that have been conducted of users
at various times will vary depending on how they are conducted, how many respond, the ability
to characterize the respondents, and what is done with the results. The recently conducted
2007/2008 Behavior Survey is useful, but it cannot be said to be reliably representative of the
customers of P2Rx. The findings show that short-term and intermediate outcomes are more
clearly influenced by the centers, but final outcomes are extremely difficult to assess.
Recommendations
The two tables below present recommendations for improving the usefulness of existing
measures and gathering new measures to better assess the centers' performance.
Table 4-1: Recommendations for Existing Measures
Existing
Measure
Activity
Measures
Description
Currently collect:
Technical Assistance Requests—
number, type of organization, and
location
Newsletters—number
Listserves—number and number of
subscribers
Meetings/trainings organized—
number and number of attendees
Meetings attended and talked about
P2Rx—number and number of
attendees
Regional conference calls—number
Recommendations and Affected
Outcomes
• Continue to collect Activity Measures
=> Defines Outputs and some
Customers
• For Newsletters, Listserves and
Meetings, identify recipients/ attendees
using commonly agreed upon identifiers
=> Defines Customers
• For Technical Assistance Requests, use
contact information to conduct follow up
=> Defines Outcomes
Web
Measures
Currently collect:
• User sessions
• Continue to collect Web Measures
=> Defines Outputs
Evaluating the Effectiveness of the EPA PPIN Grant Program
67
-------
• Page views
• Visitors that visit more than 10 times
• User sessions for specific products
• Identify user with a single question upon
entry - or - via registration
=> Defines Customers
• Use cookies to track which pages these
users view
=> Links Outputs to Customers
• Attach cookies to a pre- and post-test for
users. If someone accesses a document,
for example, they can be asked "Was this
information useful to you? Yes/No."
=> Links Customers to Short-Term
Outcomes
Case Studies
Currently collected to varying degrees
among centers. P2RIC and Peaks to
Prairies currently contribute to Zero
Waste's case study database
Continue to collect case studies
=> Defines Activity, Output
Consider summarizing the activities and
customers and tabulating the quantitative
data to show cumulative cause and effect
of center activities
=> Links Customers to Outcomes (with
causality)
P2 Results
Centers currently conduct or facilitate
data entry from others
Enhance P2 Results so that the results
reporting by TAPs and others can be
attributed to specific center activities.
Recommend a more in-depth analysis of
P2 Results to generate detailed
recommendations
=> Intermediate and Long-Term
Outcomes (with causality)
National
Surveys
Currently have results of two National
Surveys
Conduct center-specific analysis on
2007/2008 Behavior Survey
=> Customers and Outcomes
User Needs
Assessments
Currently conducted to varying degrees
among centers
Survey established stakeholder groups
periodically and consistently (NEWMOA
model), or assess needs during
workshops (Zero Waste model). In either
case, standardize questions to obtain
needed information
Pre- and Post-
Conference
Surveys
Currently conducted to varying degrees
among centers
See below
Table 4-2: Recommended New Measures
Recommended
Measures
Pre- and Post-
Conference
Surveys
Recommendation and Affected Outcomes
• Standardize use of pre- and post-test surveys at conferences and workshops for all
centers
• Ask respondents to identify themselves by two groups, similar to what was used in
the 2007/2008 Behavior Survey
Evaluating the Effectiveness of the EPA PPIN Grant Program
68
-------
=> Defines Customers
In addition to asking questions about the effectiveness of the specific event (for
center use), ask general and standard questions about needs and
recommendations for P2Rx as a whole
=> Short-Term Outcomes, Needs Assessment
Have questionnaires be collected by EPA and analyzed.
=> Defines Customers, Short-Term Outcomes, Needs Assessment Network-wide
Conduct follow up with attendees to record changes
=> Intermediate and Long-Term Outcomes
National and
Regional
Surveys
Conduct a random assignment survey of a specified target population
Note: May be used to provide a probability sample and would allow more
statistically valid inferences to be drawn from the survey. Would likely require an
ICR.
=> Short-Term, Intermediate, Long-Term Outcomes
Pre- and Post-
Outreach
Surveys
Provide a resource to an established group of people (e.g., partners) that has
agreed to provide results information. For example, a particular information packet
can be distributed to a partner audience (e.g., schools in a state or district,
businesses in a consortium) whose identities are known. Develop boilerplate
language for agreements.
Care should be taken in this case to address potential bias, and ideally a control
group would be used of similar individuals who did not receive the resource.
Survey the audience pre- and post-assistance
=> Short-Term, Intermediate, Long-Term Outcomes
Measurement
Prerequisite
Require a commitment to measure as a precondition of any provision of services, as
some centers are already doing or considering
=> Long-Term Outcomes
Direct Contact
Follow up
Perform follow up with all customers for whom contact information is known to
gather outcome information (e.g., Rapid Response and other telephone or e-mail
based on-demand services). Attempt to make this an intrinsic part of the activity for
maximum resource efficiency.
=> Short-Term, Intermediate, Long-Term Outcomes
Other recommendations:
• Conduct further study to determine how similar information providers measure their
performance. This was initially proposed to be covered in this evaluation, but was ultimately
excluded from the scope of the study. Several potential sources of information are cited in
the literature review developed by Fred MacVaugh for P2RIC, including work done by the
Centers for Disease Control and United Way. These organizations and others, such as
university extension programs, have examined ways to link information dissemination with
behavior change and other related outcomes.
• Do not hesitate to capture behavior change or other intermediate outcomes if ultimate
outcomes are not available. Behavior changes can be assumed to be correlated in a
general way with long-term outcomes, even if these outcomes are not quantified.
Evaluating the Effectiveness of the EPA PPIN Grant Program
69
-------
Appendix A - Center Logic Models
During the interviews conducted by Abt Associates, center directors were consulted regarding
the generic P2Rx Logic Model, and whether the Resources, Activities, Outputs, and Customers
listed in the generic Logic Model applied to their particular programs. Outcomes were not
included as part of this discussion. The product of this consultation is a collection of eight
center-specific logic models that show the emphasis each center places on certain logic model
components.
The following is a key to the color-coding used in each of the logic models:
Bold Underline: a center's most important resource, activity, output, or customer
Bold Font: an important resource, activity, output, or customer
Normal Font: a component that the center does, but does not emphasize
a component that the center does not use, do, or produce
a component added by the center and not included on the generic logic
model
Evaluating the Effectiveness of the EPA PPIN Grant Program
70
-------
Northeast Waste Management Officials' Association (NEWMOA) Logic Model—Information Dissemination
iin an effective partnership of states to explore, develop, promote, and implement environmentally sound solutions fortht
materials and waste, and for the remediation of contaminated sites, in order to achieve a clean and healthy environment.
Resources
Funding
/•EPA PPIN w
•State PPIN
Match
•Other State /
Local Funding
(future)
'•EPA R1 SRAP'
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
Partners
•National P2Rx
•Regional States
•Boards of
Directors /
Committees
•Academia
Activities
Outputs
Conduct
evaluation and
measurement
Create and
conduct trainings
and seminars
•P2 Results
•Case studies of
successes
Trainings,
seminars,
conferences
Provide and
participate in
networking
opportunities
Provide technical
assistance
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls
/ meetings
Technical Response
(e.g. Rapid Response)
Design and
maintain websites
•Center websites
•Ancillary websites
Collect and
produce
information
resources and
content
Market resources
IT Tool
Development
•Topic Hubs
•Newsletters
•Case Studies
(compile)
•Databases
•Innovative P2
Tech Profiles
•P2 News
•Calendar
Marketing materials
IT Tools
* Activities done with funding other than PPIN
Customers
Outcomes
Other
Centers
I
Intermediate
•TAPs
•Local / State
Governments
•Non-Profit
Organizations
•University
Technical
Centers
•Business
Associations
I
Final
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments
•Other
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content of, access
to, and delivery of
P2 information and
assistance is
efficient and of
high quality
Feedback for improved quality, access, and delivery of P2 information and assistance
71
-------
Waste Reduction Resource Center (WRRC) Logic Model—Information Dissemination
iiiiiL'.
Resources ^1
Funding
•EPA PPIN
•State PPIN
Match
1 J 1 1 r I I L < 1 r
Lf'i'.1 rill n hi Hi
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
Partners
•National P2Rx
•Regional States
•Boards /
C M in nil ">v
1 Activities ^1 Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
Provide technical
assistance
4
Design and
maintain websites
Collect and
produce
information
resources and
content
Market resources
— ^
In
•P2 Results
•Case studies of
si iccesses
Trainings,
seminars,
conferences
4
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls/
meetings
Technical Response
(e.q. Rapid
Response)
4
•Center websites
•Ancillary websites
4
•Topic Hubs
•Newsletters
•Case Studies
•Databases
Marketing materials
^^^^^1 Customers ^1 Outcomes
Other
Cente
i
Interm
•TAPs
•Local /
Govern
K ii
Organiz
•Univer
Technic
Centers
•Busine
Associa
i
Final
rs
r
ediate
ments
MI
dtions
sity
al
ions
r
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments
•Other
II
Inter-
mediate
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
^ '
ii
Inter-
mediate
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content o
to, and de
P2 informs
assistance
efficient ar
high qualit
i
, access
ivery of
ition and
is
id of
y
r
Cost savings for
final customers
Reduced energy
use, water use,
natural resource
use, and waste
produced
Feedback for improved quality, access, and delivery of P2 information and assistance
72
-------
Great Lakes Regional Pollution Prevention Roundtable (GLRPPR) Logic Model—Information Dissemination
Mission: The G
^^^^^^^^^^^H^^^^B^^^^^^^^^^fflj^^^^^^^^fflj^a^^^^^B
prevention (P2) professionals in the Great Lakes regions of the United States and Canada. H
Resources ^1
Funding
•EPAPPIN
•State PPIN
Match (Staff)
•Other State /
Local Funding
•SRAP from R5
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
Partners
•National P2Rx
•Regional States
•Boards /
Committees
(multiple
stakeholders)
•University of
Illinois
1 Activities ^1 Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
Provide technical
assistance
Design and
maintain websites
Collect and
produce
information
resources and
content
Market resources
•P2 Results
•Case studies of
successes
Trainings,
seminars,
conferences
4
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls /
meetings
Technical Response
(e.g. Rapid Response)
4
•Center websites
•Ancillary websites
4
•Topic Hubs
•Newsletters
•Case Studies
•Databases
•News
•Library
Marketing materials
'
^^^^^H Customers ^1 Outcomes I
Only on
Ad-Hoc
Basis
Other
Cente
^
Interm
•TAPs
•Local /
Governr
•Non-Pr
Organiz
•Univers
Technic
Centers
•Busines
Associa
(PNEAC
^
Final
•Industry
•Busines
•Utilities
•Schools
•Local /
Governr
•Other
rs
r
ediate
WMRC)
State
nents
ofit
ations
ity
al
5S
ions
)
r
i
3ses
State
nents
o E
f 0)
(0 **
Inter-
mediate
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
^ '
it
_c fl)
V) **
<- 13
** "o
= 0)
~ E
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content of, access
to, and delivery of
P2 information and
assistance is
efficient and of
high quality
T '
Cost savings for
final customers
Reduced energy
use, water use,
natural resource
use, and waste
produced
Feedback for improved quality, access, and delivery of P2 information and assistance
"Co-located w/ WMRC (TAP)—transitioning to become part of University of Illinois
73
-------
The Southwest Network for Zero Waste Logic Model—Information Dissemination
lowering environmental impact. It's a win-win, non-regulatory approach that benefits everyone.
Resources
Funding
•EPAPPIN
•State PPIN
Match
•Other State /
Local Funding
and State
Contracts
•In-Kind Match
for Services and
Funds
•Leveraged
Federal Funding
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
•State and
Federal
Databases
•Participating
Businesses
Partners
•National P2Rx
Network
•Regional States
•Advisory
Boards /
Committees /
Roundtables
•Military
•University
•Volunteers
•Regional
Consortia
Activities
Conduct evaluation
and measurement
Outputs
•P2 Results
•Case studies of
successes
Create and
conduct trainings
and seminars
Trainings,
seminars,
conferences
•Provide and
participate in
networking
opportunities
•Strategic
Planning
Provide technical
assistance
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls
/ meetings
Technical Response
(e.g. Rapid Response)
Design and
maintain websites
•Center websites
•Ancillary
websites
Collect and
produce
information
resources and
content; assess
needs
Market resources
•Topic Hubs
•Newsletters
•Case Studies
•Databases
Marketing materials
O)
JB .£
=£-£
.c o
§•!
O i_
M
Customers
Outcomes
Other
Centers
Intermediate
•TAPs
•Local / State
Governments
•Non-Profit
Organizations
•University
Technical
Centers
•Business
Associations
•Pretreatment
Programs
i- (0
£v
_= 0)
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
Final
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments
V)
fc.2
£ -a
c a>
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content of, access
to, and delivery of
P2 information and
assistance is
efficient and of
high quality
Feedback for improved quality, access, and delivery of P2 information and assistance
"Located at a University
74
-------
Pollution Prevention Regional Information Center (P2RIC) Logic Model—Information Dissemination
Missouri, Nebraska) that provide waste reduction services and expertise to business and industry.
Resources ^1
Funding
•EPAPPIN
•State PPIN
Match
•Other State /
Local Funding
•In-Kind Match
from NGOs,
TAPs, etc
•Revenue
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
•Science
Expertise
Partners
•National P2Rx
•Regional States
and Local P2
Programs
•Boards /
Committees
•Regional
Roundtable
1 Activities ^1 Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
Provide technical
assistance
Design and
maintain websites
Collect and
produce
information
resources and
content
Market resources
•P2 Results
•Case studies of
successes
Trainings, seminars,
conferences,
lectures, podcasts
£
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls
/ meetings
Technical Response
(e.g. Rapid Response)
4
•Center websites
•Ancillary websites
4
•Topic Hubs
(updates)
•Newsletters
•Case Studies
•Databases
•News
•Marketing materials
•Event Publicity
^^^^^1 Customers ^1 Outcomes 1
Other
Cente
^
Interm
•TAPs
•Local /
Govern
•Non-Pr
Organiz
•Univers
Technic
Centers
•Busines
Associa
•Industi
consort
•Chamb
Comme
^
Final
•Industry
•Busines
•Utilities
•Schools
•Local /
Governr
•Other
•Univers
rs
r
ediate
State
ments
ofit
ations
ity
al
5S
ions
y
ia
erof
rce
r
/
5ses
State
nents
ities
o E
Inter-
mediate
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
T '
li
Inter-
mediate
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content o
to, and de
P2 inform;
assistance
efficient ar
high qualit
i
, access
ivery of
stion and
is
id of
y
r
Cost savings for
final customers
Reducec
use, wat
natural r
use, and
produce
energy
er use,
s sou rce
waste
d
Feedback for improved quality, access, and delivery of P2 information and assistance
"Based in a Business Development Center at a University
75
-------
Peaks to Prairies Logic Model—Information Dissemination
gion 8 states, Peaks to Prairies encourages adoption of pollution prevention practices by citizens, small businesses,
:ion and contacts, encourages collaboration and leveraging of resources between programs, and builds information sy
dissemination.
Resources
Funding
•EPAPPIN
•State PPIN
Match
•Other State /
Local Funding
(Montana PPIS)
•Occasional
Grants (EPA
and other)
Expertise /
Skills
•Staff Skills
(Program
management /
product
development)
•P2 Knowledge
•Technical and
Web Expertise
Partners
•Montana State
University
•National P2Rx
•Regional State
P2 Programs
•Boards /
Committees
•National P2
Roundtable
•National Tribal
P2 Workgroup
Activities
Outputs
Conduct
evaluation and
measurement
Create and
conduct trainings
and seminars
•P2 Results
•Case studies of
successes
Trainings,
seminars,
conferences
Provide and
participate in
networking
opportunities
(Tribal and Parks)
Provide technical
assistance
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls /
meetings
Technical Response
(e.g. Rapid Response)
Design and
maintain websites
•Center websites
•Ancillary websites
Collect and
produce
information
resources and
content
•Topic Hubs
•Newsletters
•Case Studies
•Databases
•Toolkits
Market resources
(distribute)
Marketing materials
"Based in University
"Focus on Tribes, Parks
Customers
Other
Centers
Intermediate
•TAPs
•Local / State /
Federal
Governments
•Non-Profit
Organizations
•University
Technical Centers
•Business
Associations
•Consultants
Final
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments-
Parks
•Home builders.
architects.
home owners
•Tribal
Government
Outcomes
V)
i- ra
0) —
*- ^
£ a)
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
(0
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content of, access
to, and delivery of
P2 information and
assistance is
efficient and of
high quality
Cost savings for
final customers
Reduced energy
use, water use,
natural resource
use, and waste
produced
Feedback for improved quality, access, and delivery of P2 information and assistance
76
-------
Western Regional Pollution Prevention Network (WRPPN) Logic Model—Information Dissemination
Mission: The Western Regional Pollution Prevention Network (WRPPN) is a strategic alliance involving local, state, federal and tribal pollution prevention (P2) programs throughout EPA
Region 9. WRPPN was established in 1997 by the U.S. Environmental Protection Agency to improve communication and information dissemination among network members to maximize
efficiency of P2 implementation.
Resources ^1
Funding
•EPAPPIN
•State PPIN
Match (Office
Space)
•Other State /
Local Funding
•Partnership
Program Grant
Expertise /
Skills
•Staff Skills
•P2 Knowledge
•Technical and
Web Expertise
Partners
•National P2Rx
•Regional States
•Boards /
Committees
•California
Consortium of
P2 Committees
•Green
Business
Programs
1 Activities ^1 Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
With TAPs and
National P2Rx
Provide technical
assistance
Design and
maintain websites
Collect and
produce
information
resources and
content
Market resources
via listserv
— >
•P2 Results
•Case studies of
successes
Trainings.
seminars.
conferences.
webinars. subject -
specific committees
*
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls/
meetings
Technical Response
(e.g. Rapid Response)
*
•Center websites
•Ancillary websites
*
•Topic Hubs
•Newsletters
•Case Studies
•Databases
•Specialty Projects
Marketing materials
^^^^^1 Customers ^1 Outcomes
~4
Other
Cente
i
Interm
•TAPs
•Local /
Govern
•Non-Pr
Organiz
•Univer
Technic
Centers
•Busine
Associc
•Interna
(Mexico
rs
r
ediate
State
ments
ofit
ations
sity
al
ss
itions
tional
)
*
Primarily
conference
attendance
Final
•Industry
•Busines
•Utilities
•Schools
•Local /
Governr
•Univers
•DoE
1
3ses
5
State
nents
ities
t £
o t
f 0)
(0 **
, »
5.2
*J -o
= 0)
E
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
1 r
it
f 0)
(0 **
, £
5 .2
2 -o
= 0)
E
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content o
to, and de
P2 inform;
assistance
efficient ar
high qualit
i
, access
ivery of
stion and
is
id of
y
r
Cost savings for
final customers
Reducec
use, wat
natural r
use, and
produce
energy
er use,
esource
waste
d
Feedback for improved quality, access, and delivery of P2 information and assistanc
77
-------
Pacific Northwest Pollution Prevention Resource Center (PPRC) Logic Model—Information Dissemination
Mission: The Pacific Northwest Pollution Prevention Resource Center (PPRC) is a non-profit organization that is the Northwest's leading source of high quality, unbiased pollution prevention
information. PPRC works collaboratively with business, government, non-government organizations, and other sectors to promote environmental protection through pollution prevention.
Resources ^1
Funding
•EPA PPIN (1/4)
•State PPIN
Match
•Other State /
Local Funding
Grants and
Contracts
•SRAP Grant
Expertise /
Skills
•Internal Staff
Skills
•P2 Knowledge
•Technical and
Web Expertise
•State and Local
Expertise
Partners
•National P2Rx
•Regional State P2
Programs, Local
Government
•Boards /
Committees
•Green Business
Programs
•Non-
Environmental
Business
Programs (i.e.
MEPs)
1 Activities ^1 Outputs 1
Conduct evaluation
and measurement
Create and
conduct trainings
and seminars
Provide and
participate in
networking
opportunities
Provide technical
assistance
Design and
maintain websites
Collect and
produce
information
resources and
original content
Market resources
•P2 Results
•Case studies of
successes
Trainings, seminars.
conferences.
webinars
£
•Partnerships
•Listservs
•Conferences /
Roundtables
•Conference calls
/ meetings
Technical Response
(e.g. Rapid Response)
£
•Center websites
•Ancillary websites
t
•Topic Hubs
•Newsletters
•Case Studies
•Databases
Marketing materials
^^^^^H Customers ^1
Wholesale
Other
Centers
4
Intermediate
•TAPS
•Local / State
Governments
•Non-Profit
Organizations
•University
Technical
Centers
•Business
Associations
•Consultants
•Non-
Environmental
Programs
4
Final
•Industry
•Businesses
•Utilities
•Schools
•Local / State
Governments
•Other
Outcomes
|I
a .2
*- 'o
C 0)
E
Improvements in
information content,
delivery, sharing, and
access
Capacity to provide P2
resources and services is
enhanced
^
o E
.C 0)
(0 **
Inter-
mediate
r
Increased awareness of P2
Implementation of P2
plans and practices
Long-term
Content ol
to, and de
P2 informs
assistance
efficient ar
high qualit
^
, access
ivery of
stion and
is
id of
y
r
Cost savings for
final customers
Reducec
use, wat
natural r
use, and
produce
energy
er use,
ssource
waste
d
Feedback for improved quality, access, and delivery of P2 information and assistanc
Independent Non-Profit
78
-------
Appendix B - Data Collection Matrix
EVALUATION QUESTION
1.
1.a.
1.b.
1.C.
1.d.
2.
2.3
What model(s) do the
P2Rx centers use to
encourage exploration
and adoption of P2
practices?
How are centers currently
operating - what model(s)
are they using to
disseminate information?
Which of the centers'
model(s) is most efficient in
information sharing and/or
delivery?
When comparing between
centers, what opportunities
for centers' information
delivery to become more
efficient are observed?
What opportunities are there
for centers to become more
efficient in information
delivery by sharing tasks?
What groups / types of
customers access P2Rx
information?
How do the customers vary
according to method of
outreach by the centers?
•Via the website
•Via TAPs
•Directly (via telephone,
correspondence or other
communication)
DATA SOURCES
Behavior Survey
(2007/8)
-
Q6 list from
each center
Q6 & Q7,
compared
across centers
-
-
Q2&Q3
Q7 crossed with
Q2 (partial
answer)
Customer
Satisfaction
Survey
(Weinreich,
2004) 9
-
-
Q9, if attributed
-
-
Q2
-
Web
Measures10
Yes
-
-
-
-
Yes
Yes, Activity
Measures
Interviews -
(Eclnc, 2004)11
Center Contacts
Q1.3&4
Center Contacts
Q1.3-6
Managers Q 6
Center Contacts
Q14, 15
Coordinators
Q3?
Center Contacts
multiple
questions
Managers Q 3
Coordinators
Q5-8
Center Contacts
Q7-Q11, 15
-
-
Phone
Interviews
(2008) 12
P2Rx
P2Rx
-
P2Rx
P2Rx
-
-
Focus Groups -
Region 9& 10
-
-
-
-
-
-
-
Center-specific
Documentation
13
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
-
Generic and
Individual Logic
Models
Yes
Yes
-
Yes
-
-
-
12
Question numbers differ depending on which version of the Survey you are using.
0 Will use breakout of data by Center.
' Based on interview transcripts from lEc Inc. Information is somewhat dated and doesn't directly address our evaluation questions.
Interviews will be conducted with three sets of entities: P2Rx center directors (P2Rx), Technical Assistance Providers (TAPs), and Business Representatives (BR)
' Will vary by Center.
79
-------
EVALUATION QUESTION
3.
3.a
4.
4.a.
5.
5.3.
6.
6.3.
6.b.
7.
1.3.
7.b.
What products, services
and content do
businesses need to make
a decision / consider the
adoption of P2 practices?
Are the centers providing
the right products, services
and content to businesses
to promote the adoption of
P2 practices?
What products, services
and content do TAPs need
to promote businesses'
adoption of P2 practices?
Are the centers providing
the right products, services
and content to TAPs to
promote the adoption of P2
practices by businesses?
To what extent are P2Rx
center resources
influencing businesses'
adoption of P2 practices?
What causes / influences
businesses to seek P2Rx
information?
Are the centers effective
in reaching businesses
through TAPs?
What do centers do to reach
businesses through TAPs?
Can any outcomes (short-
term, intermediate, or long-
term) be attributed to
information received from
P2Rx centers?
Are the centers effective
in reaching businesses
directly?
What do centers do to reach
businesses directly?
Can any outcomes (short-
term, intermediate, or long-
term) be attributed to
information received from
P2Rx centers?
DATA SOURCES
Behavior Survey
(2007/8)
-
Ultimate user
survey - Q5, Q6,
(Q7), linked to
businesses in
Q2
-
Intermediate
user survey -
Q5, Q6, (Q7),
linked to TAPs
in Q2
Ultimate user
survey - Q8 &
Q9
-
Ultimate user
survey - Q4 &
Q5
-
Intermediate
user survey -
Q8-Q11
Ultimate user
survey - Q4 &
Q5
-
Both surveys -
Q8-Q10
Customer
Satisfaction
Survey
(Weinreich,
2004) 9
-
Q9-Q12
-
Q9-Q12
Q7e, k
-
-
-
Q7e,j, k
-
-
Q7e,j, k
Web
Measures10
-
-
-
-
-
-
-
-
-
-
-
Interviews -
(Eclnc, 2004)11
-
-
Center Contacts
Q11 -13
Center Contacts
Q11 -13, 15
-
-
Center Contacts
Q19, 20,22
Center Contacts
Q13, 18-23
-
Center Contacts
Q19, 22,23
Center Contacts
Q13, 18-23
-
Phone
Interviews
(2008) 12
TAP
BR
BR
TAP
BR
TAP
BR
TAP
BR
P2Rx
BR
P2Rx
P2Rx
TAP
P2Rx
P2Rx
BR
P2Rx
TAP
P2Rx
Focus Groups -
Region 9& 10
Business Group
Q7&Q8
Business Group
Q7&Q8
TAPs Group
Q2
TAPs Group
Q3&Q4
TAPs Group
Q5
-
Business Group
Q4
TAPs Group
Q4
Business Group
Q3
TAPs Group
Q5
Business Group
Q2
-
Business Group
Q3
TAPs Group
Q5
Center-specific
Documentation
13
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
See Appendix H
in Document
Generic and
Individual Logic
Models
-
-
-
-
-
-
-
-
-
-
-
-
80
-------
EVALUATION QUESTION
8.
8.a.
8.b.
What performance
measures do P2Rx
centers need to collect in
order to demonstrate /
document client behavior
and other outcomes?
How would the centers that
currently do not have the
capacity to make/gather the
desired measurements build
their capacity?
How can the information
best be collected based on
available technology, funds
and survey approval
requirements?
DATA SOURCES
Behavior Survey
(2007/8)
-
-
-
Customer
Satisfaction
Survey
(Weinreich,
2004) 9
-
-
-
Web
Measures10
-
-
-
Interviews -
(Eclnc, 2004)11
Managers Q4, 5
Coordinators
Q4
Center Contacts
Q26
Managers Q5
Managers Q5
Phone
Interviews
(2008) 12
P2Rx
P2Rx
P2Rx
Focus Groups -
Region 9& 10
TAPs Group
Q6
-
-
Center-specific
Documentation
13
See Appendix H
in Document
Generic and
Individual Logic
Models
-
-
-
81
-------
Appendix C - 2007/2008 Behavior Survey Analysis
The 2007/2008 Behavior Survey data have been merged together from several sources. Each
center had a separate survey administered electronically through a link on the center's website
using Survey Monkey. In addition, one center distributed and collected surveys in hard copy.
For each question, the results are presented for business and TAPs separately as determined
by the first question, as well as business and TAPs responses combined. Where [P2Rx] occurs
in the question text listed here, any of the centers may be substituted.
Centers
Table 1 gives a summary of how many responses were received by center. This includes all
surveys started. Not all surveys are complete. The figure for Zero Waste includes 207 hard
copy surveys.
Table 1: Responses by Center
Center
GLRPPR
NEWMOA
P2RIC
P2Rx
Peaks to Prairies
PPRC
WRPPN
WRRC
Zero Waste
Grand Total
Respondents
50
67
80
10
48
69
77
34
234*
669
* Includes 207 surveys administered in hard copy
1. Respondent Type
The survey respondents were divided into types based on the first question. Respondents are
either members of the regulated community ("business") or technical assistance providers
("TAPs").
Question 1
Please select one option from the following list:
• I primarily use [P2Rx] to find environmental assistance information to apply to my own
organization or facility. [Business]
• I primarily use [P2Rx] in my capacity as a provider of environmental assistance to others
outside my organization. [TAPs]
Of the responses, 352 (53%) were of type business, while 317 (47%) were of type TAPs. In
each of the following questions, business and TAPs responses will be analyzed both together
and separately.
Evaluating the Effectiveness of the EPA PPIN Grant Program
82
-------
2. Organization Type
Table 2 lists answers to the question "What type of organization do you represent?" Figures are
given by percent. Please note that respondents were allowed to choose more than one
response, so these numbers will not add up to 100%. The total number of responses to this
question is 275 for business, 281 for TAPs, and 556 total.
Table 2: Type of Organization (% of responses)
Type of Organization
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
Other
Business
11%
12%
14%
41%
12%
6%
5%
4%
13%
TAPS
9%
35%
28%
5%
6%
11%
11%
7%
33%
Total
10%
24%
21%
22%
9%
8%
8%
6%
23%
Among business respondents, manufacturing was the leading type of organization at 40% of
respondents, while among TAPs state and local government were the leaders at 35% and 28%,
respectively.
3. Frequency of Use
Table 3 lists the answers to the question "How often do you use [P2Rx]?"
Table 3: Frequency of Use (% of responses)
Frequency of Use
First time user
Every three months
Monthly
Weekly
Other (please specify)
Number of responses
Business
51%
15%
11%
8%
14%
311
TAPs
22%
24%
18%
15%
20%
295
Total
37%
19%
15%
12%
17%
606
There are a notable number of first-time users. The business community was highly skewed
toward new or infrequent users. TAPs users were more evenly distributed across usage levels.
Among users who specified "Other," most answered that they used the site less often than
every three months, or only when a newsletter arrived.
4. Referrers
Table 4 lists answers to the question "How did you hear about [P2Rx]? (Check only one
answer.)" Hard copy users were able to check more than one answer; therefore the numbers
will not add up to 100%.
Evaluating the Effectiveness of the EPA PPIN Grant Program
83
-------
Table 4: Referrers
Referrer
Assistance program
Brochure or newsletter
Colleague
Conference or meeting
E-mail message or listserve
Link from another website
Other (please specify)
Search engine
Vendor
Number of responses
Business
4%
4%
15%
37%
15%
7%
8%
11%
0%
309
TAPS
4%
1%
29%
33%
14%
6%
9%
5%
1%
295
Total
4%
2%
22%
35%
14%
7%
9%
8%
0%
604
Most users were referred to P2Rx by a conference or meeting, a colleague, or an e-mail. More
TAPs were referred by colleagues than were businesses.
5. Awareness (Outcomes)
Table 5 lists the answers to the following question, which is worded differently for business and
for TAPs:
Business: "[P2Rx] helps me to improve my awareness of environmental practices to reduce
pollution. (Check only one answer.)"
TAPs: "Access to [P2Rx] information has improved my ability to provide technical assistance to
my clients. (Check only one answer)".
Table 5: Awareness (Outcomes)
Improved awareness/ability
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
Number of responses
Business
29%
41%
15%
0%
0%
14%
306
TAPs
20%
44%
22%
1%
1%
10%
290
Total
25%
43%
18%
1%
1%
12%
596
Both business and TAPs respondents were favorable about the usefulness of the site. 70% of
business respondents agreed or strongly agreed that the site improves their awareness of
environmental practices, and 64% of TAPs respondents agreed or strongly agreed that the site
improves their ability to provide technical assistance. Very few respondents disagreed, though
some were neutral, and a substantial number did not use the site.
6. Usefulness of Each Resource
Tables 6.1-6.10 summarize the results of the following question: "How useful were [P2Rx]
resources in meeting your needs? (Please make one selection for each resource listed)."
Different centers' surveys included different resources; therefore, only "core" resources which
were shared among all the electronic surveys were included. Note that not all of these "core"
Evaluating the Effectiveness of the EPA PPIN Grant Program
84
-------
resources are offered by all centers, but were not omitted from the list of responses, thereby
possibly skewing the results. Three of these resources, Library, Industry Sector Information,
and Vendors, were not available as choices on the paper version of the survey.
Table 6.11 shows an overview of all resources, providing the percentage of users who
responded "Useful" or "Very useful" not counting those who responded "Unaware of-Do not use"
(i.e., all active users)
Table 6.1: Topic Hubs
Topic Hubs
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
37%
0%
8%
36%
19%
237
TAPS
24%
2%
13%
28%
33%
229
Total
31%
1%
10%
32%
26%
466
Topic Hubs were rated highly, as Useful or Very Useful. Consistent with other evaluation
findings, those who found them most useful were the TAPs audience, although business
respondents also found them to be useful.
Table 6.2: News
News
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
35%
0%
11%
32%
22%
236
TAPs
21%
2%
21%
35%
21%
229
Total
28%
1%
15%
33%
22%
465
While the percentage of businesses and TAPs that felt the News was Useful, more TAPs than
businesses considered it only Somewhat Useful.
Table 6.3: Industry Sector Information
Industry Sector Information
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
35%
2%
13%
32%
19%
135
TAPs
26%
3%
14%
28%
28%
181
Total
30%
3%
14%
30%
24%
316
Sector Information was rated highly, as Useful or Very Useful. Consistent with other evaluation
findings, there was a larger percentage of TAPs than businesses finding it Very Useful.
Evaluating the Effectiveness of the EPA PPIN Grant Program
85
-------
Table 6.4: Case Studies
Case Studies
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
41%
0%
10%
27%
21%
237
TAPs
29%
2%
19%
26%
25%
234
Total
35%
1%
14%
27%
23%
471
The TAPs and business audiences were fairly consistent in their rating of the usefulness of case
studies.
Table 6.5: Rapid Response
Rapid Response
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
56%
0%
12%
20%
11%
235
TAPs
53%
5%
13%
18%
11%
228
Total
54%
3%
13%
19%
11%
463
Results regarding the usefulness of Rapid Response were very consistent between the two
audiences, although many respondents were not aware of it or did not use this service.
Table 6.6: P2 Programs Directory
P2 Programs Directory
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
38%
1%
8%
32%
20%
236
TAPs
31%
3%
15%
33%
18%
229
Total
34%
2%
12%
32%
19%
465
Consistent with other evaluation findings, the P2 Programs Directory was not found to be Very
Useful by either audience.
Table 6.7: Library
Library
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
44%
4%
13%
21%
18%
135
TAPs
30%
4%
20%
26%
19%
181
Total
36%
4%
17%
24%
19%
316
Evaluating the Effectiveness of the EPA PPIN Grant Program
86
-------
The library was found to be Useful by both audiences.
Table 6.8: Vendor Database
Vendor Database
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
50%
5%
16%
20%
8%
135
TAPs
43%
8%
27%
15%
7%
181
Total
46%
7%
22%
17%
8%
316
Nearly half of respondents were not aware of or did not use the Vendor Database. Of those
who did use it, the largest number of TAP respondents found it to be Somewhat Useful, while
the largest number of business respondents found it to be Useful.
Interestingly, this somewhat contradicts some statements heard in focus groups or interviews,
indicating a desire for more vendor information. Perhaps the Vendor Database needs to be
more accessible or better promoted on the P2Rx websites, although the "Somewhat useful"
rating among those who do use it indicates that the Vendor Database may need improvements
in content as well.
Table 6.9: Listserves
Listserves
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
40%
0%
11%
31%
17%
235
TAPs
33%
5%
15%
26%
21%
23 1
Total
37%
2%
13%
29%
19%
466
Business and TAP respondents felt similarly about the usefulness of the listserves.
Table 6.10: P2 Results System (Measurement Tool)
P2 Results System (Measurement
Tool)
Unaware of-Do not use
Not useful
Somewhat useful
Useful
Very useful
Number of responses
Business
54%
2%
8%
25%
11%
239
TAPs
42%
9%
19%
18%
12%
226
Total
48%
5%
14%
22%
12%
465
Interestingly, a quarter of the business respondents find the P2 Results System useful. A large
percentage of TAP respondents considered it Somewhat Useful, Useful, or Very Useful.
Evaluating the Effectiveness of the EPA PPIN Grant Program
87
-------
Table 6.11: Percent of active users rating resource as useful or very useful
Resource
Topic Hubs
P2 Programs Directory
News
Industry Sector Information
Case Studies
Listserves
Library
Rapid Response
P2 Results system (Measurement
Tool)
Vendor Database
Business
87%
84%
83%
77%
67%
81%
70%
72%
79%
57%
TAPS
81%
73%
71%
76%
71%
70%
65%
61%
52%
40%
Total
84%
78%
77%
77%
76%
76%
67%
66%
64%
46%
Overall, the most highly rated resources are the Topic Hubs, P2 Programs Directory, News, and
Sector Information. In all cases, fewer business respondents were aware of the resources
P2Rx had to offer than were the TAPs. It is interesting to note that among active users,
business respondents do seem to be finding value in the resources that P2Rx has to offer,
consistent with the value that the TAPs are finding.
7. Types of Use
Tables 7.1 and 7.2 list answers to a question about how the sites were used. This question is
worded differently for business and TAPs:
Business: "How did you utilize the [P2Rx] website(s)? (Check all that apply.)"
TAPs: "What information did you use to improve or influence your clients' understanding of
environmental practices to reduce pollution? (Check all that apply.)"
Note that since more than one answer could be chosen, the numbers will not add up to 100%.
Table 7.1: Business Use of Websites
Use
Attended a training, workshop, or conference promoted by a P2Rx Center
Reviewed Topic Hub information (or other NEWMOA/P2Rx website information)
Asked for pollution prevention technical assistance
Contacted a technical assistance provider
Requested additional resources from a P2Rx Center
Other (please specify)
Number of responses
Business
55%
51%
12%
10%
9%
14%
203
Over half of business respondents indicated that they used the website to attend a training,
conference or workshop, or review Topic Hub or other information. However, much smaller
percentages of respondents used P2Rx websites for other purposes. Consistent with other
Evaluating the Effectiveness of the EPA PPIN Grant Program
88
-------
findings, events such as conferences and trainings are a very popular use of the P2Rx center
websites.
Table 7.2: TAPs Use of Websites
Use
Provided training, workshop, and/or conference information
Provided information on technical assistance providers
Provided Topic Hub information
Provided vendor information
Other pollution prevention technical assistance. If so, please specify.
Number of responses
TAPs
59%
48%
38%
24%
57%
208
A majority of TAP respondents also utilized P2Rx websites for trainings, conferences, or
workshops. TAP respondents indicated a greater percentage of utilization of other P2Rx
services than did businesses.
8. Behaviors (Outcomes)
Table 8 indicates which actions were taken by users as a result of P2Rx programs. The
question was worded differently for business and TAPs:
Business: "What action(s) have you taken to improve environmental practices, in whole or in
part, due to information you have found through [P2Rx]? (Check all that apply.)"
TAPs: "What action(s) have your clients taken to improve environmental practices, in whole or in
part, due to information you have found through [P2Rx]?"
Table 8: Actions Taken
Actions Taken
Changed a pollution prevention
process or practice (i.e., implemented
pollution prevention)
Identified a pollution prevention
opportunity
Purchased new process equipment to
prevent pollution
Implemented energy conservation
measures
Switched to renewable energy
Contacted a vendor
Implemented material or waste
recycling system
Changed handling of waste or
emission
Installed pollution control equipment
(e.g., scrubbers, control technique)
Installed a waste treatment system
Business
28%
45%
20%
22%
8%
14%
36%
17%
8%
2%
TAPs
53%
65%
23%
27%
4%
28%
38%
36%
11%
10%
Total
41%
55%
22%
25%
6%
21%
37%
27%
10%
6%
Evaluating the Effectiveness of the EPA PPIN Grant Program
89
-------
No process changes were taken
Other (please specify)
Number of responses
21%
7%
188
14%
15%
188
18%
11%
376
TAP respondents indicate that they can attribute more outcomes to P2Rx information than do
businesses. One possible explanation of this difference is that TAPs are being asked to report
outcomes to the centers for P2 Results. See Table 9 as well.
9. Condition (Outcomes)
Table 9 shows answers to the following question:
Business: "Please identify whether you reduced or eliminated pollutant(s) as a result of[P2Rx]
use. (Check all that apply.)"
TAPs: "Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of
[P2Rx] use. (Check all that apply.)"
Table 9: Pollutant Reduction
Pollutant Reduction
Reduced non-hazardous waste (solid
waste)
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
Other (please specify)
Number of responses
Business
26%
7%
29%
7%
7%
3%
12%
5%
20%
31%
19%
197
TAPs
36%
13%
37%
13%
15%
5%
26%
10%
34%
8%
16%
208
Total
31%
10%
33%
10%
11%
4%
20%
7%
27%
19%
18%
405
Again, TAP respondents generally indicate more attribution of outcomes to P2Rx use than do
businesses. Of note are the high responses from both audiences for "Reduced non-hazardous
waste" and "Reduced hazardous waste or hazardous materials." A fairly high percentage of
businesses responded "None" as compared to TAPs, and high percentages in both audiences
responded "Don't know."
10. Cost Savings (Outcomes)
Table 10.1 presents answers to the following questions:
Business: "Has using [P2Rx] or the action(s) taken above resulted in cost savings to you? (Cost
includes time as well as dollars.) (Check only one answer.)"
Evaluating the Effectiveness of the EPA PPIN Grant Program
90
-------
LAPs: "Has using [P2Rx] or the action(s) taken above resulted in cost savings to your client?
(Cost includes time as well as dollars.) (Check only one answer.)"
Table 10.1: Cost Savings for Business
Cost Savings
Do not know
No
Yes
Number of
responses
Business
49%
27%
24%
224
TAPs
63%
9%
29%
219
Total
56%
18%
26%
443
Nearly half of business respondents and over half of TAP respondents indicated that they do not
know whether P2Rx resources have contributed to cost savings, although around a quarter of
all respondents thought that P2Rx had contributed to cost savings for businesses.
Table 10.2 presents answers to the following question for TAPs only: "Has using [P2Rx] saved
you time or money in serving your client(s) ? (Check only one answer.)"
Table 10.2: Direct Cost Savings for TAPs
Direct Cost Savings
Do not know
No
Yes
Number of responses
TAPs
41%
10%
49%
22 1
Almost half of TAP respondents indicated that use of P2Rx had saved them time or money in
serving their clients, and 40 percent did no know, leaving only 10 percent who did not think they
had saved money or time using P2Rx.
Evaluating the Effectiveness of the EPA PPIN Grant Program
91
-------
Appendix D - Center-Specific 2007/2008 Behavior Survey
Results
Evaluating the Effectiveness of the EPA PPIN Grant Program 92
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use NEWMOA and/or
P2Rx to find environmental
assistance information to apply [
to my own organization or
facility.
I primarily use NEWMOA and/or
P2Rx in my capacity as a provider ,-
of environmental assistance to <-
others outside my organization.
Response Response
Percent Count
55.2%
44.8%
answered question
skipped question
37
30
67
Page: Questions for Regulated Community
2. 2. What type of organization d<
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
f> view other (please specify)
D you represent? (Check all that apply.)
Response
Percent
n 3.6%
| Zo.o /o
0.0%
| 25.0%
| | 10. /%
| 10.7%
3.6%
I 7.1%
1 17.9%
answered question
skipped question
Response
Count
1
8
0
7
3
3
1
2
5
28
39
3. 3. How often do you use NEWMOA and/or P2Rx? (Check only one answer.)
Response Response
Percent Count
~q 53.6% 15
14.3% 4
17.9% 5
14.3% 4
0.0% 0
answered question 28
skipped question 39
First time user
Weekly
Monthly
Every three months
Other (please specify)
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (1 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
4. 4. How did you hear about NEWMOA
Search engine
Link from another website
E-mail message or listserv
Conference or meeting
\f) view
Brochure or newsletter
X% 1 1
Colleague
Vendor
Assistance program
Other (please specify)
1
^^^^^^^^^^—
^~
~|
B
B
and/or P2Rx?
d
3
I
(Check only
one answer
.)
Response
Percent
17.9%
17.9%
3.6%
7.1%
7.1%
25.0%
0.0%
10.7%
10.7%
answered question
skipped question
Response
Count
5
5
1
2
2
7
0
3
3
28
39
5. 5. NEWMOA and/or P2Rx helps me to improve my awareness of
environmental practices
to reduce
pollution. (Check only one answer.)
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
1
n
d
— I
Do Not UseQ ^
Response
Percent
25.0%
46.4%
14.3%
3.6%
0.0%
10.7%
answered question
skipped question
Response
Count
7
13
4
1
0
3
28
39
Page: Questions for Regulated Community (continued)
6. 6. How useful were NEWMOA and/or
P2Rx resources in meeting
your
needs? (Please make one
selection for each resource listed.)
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
NEWMOA P2 & CA Metrics
Database Software
Mercury Reduction Programs
Database
Verj
usef
^^.^%
16.7%
5.6%
5.6%
16.7%
5.6%
5.6%
11.1%
11.1%
5.6%
0.0%
11.1%
f
ul
(2)
(3)
(1)
(1)
(3)
(1)
(1)
(2)
(2)
(D
(0)
(2)
Usef
50.0%
33.3%
44.4%
27.8%
33.3%
27.8%
16.7%
27.8%
11.1%
27.8%
27.8%
33.3%
ul
(9)
(6)
(8)
(5)
(6)
(5)
(3)
(5)
(2)
(5)
(5)
(6)
Somewhat
useful
11.1% (2)
16. 7% (3)
16. 7% (3)
16. 7% (3)
11.1% (2)
16. 7% (3)
27.8% (5)
11.1% (2)
11.1% (2)
11.1% (2)
11.1% (2)
27.8% (5)
Not
0.0
0.0
0.0
0.0
0.0
5.6
0.0
0.0
0.0
0.0
5.6
5.6
useful
% (0)
% (0)
% (0)
% (0)
% (0)
%(1)
% (0)
% (0)
% (0)
% (0)
%(1)
%(1)
Unaware
of-Do not
use
27.8% (5)
33.3% (6)
33.3% (6)
50.0% (9)
38.9% (7)
44.4% (8)
50.0% (9)
50.0% (9)
66.7% (12)
55.6% (10)
55.6% (10)
22.2% (4)
Response
Count
18
18
18
18
18
18
18
18
18
18
18
18
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (2 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
Northeast States Activities
(Projects) Database
1R70/
167/0
11.1% (2) 5.6% (1) 50.0% (9)
answered question
skipped question
18
18
49
7. 7. How did you utilize the NEWMOA and/or P2Rx website(s)? (Check all that apply.)
Response Response
Count
Reviewed Topic Hub
information (or other NEWMOA/[
P2Rx website information)
Attended a training, workshop, or
conference promoted by a P2Rx [
Center
Asked for pollution prevention
technical assistance
Requested additional resources
from a P2Rx Center
Contacted a technical assistance ,
provider'
*>
View
]
Percent
55.6%
22.2%
5.6%
11.1%
33.3%
16.7%
answered question
skipped question
10
4
1
2
6
3
18
49
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part, due to
information you have found through NEWMOA and/or P2Rx? (Check all that apply.)
Response
Changed a pollution prevention
process or practice (i.e., |
implemented pollution prevention)
Identified a pollution .—
prevention opportunity'—
Purchased new process .—
equipment to prevent pollution'—
Implemented energy conservation .—
measures'—
Switched to renewable energy
Contacted a vendorl
Implemented material or waste ,—
recycling system'—
Changed handling of waste or.—
emission'—
Installed pollution control
equipment (e.g., scrubbers, | |
control technique)
Installed a waste treatment system
No process changes were takenl |
-• view
Other (please specify)
Percent
Response
Count
33.3%
61.1%
22.2%
11.1%
0.0%
16.7%
27.8%
11.1%
5.6%
0.0%
5.6%
16.7%
11
0
3
0
1
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (3 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
answered question
skipped question
18
49
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of NEWMOA and/or
P2Rx use. (Check all that apply.)
Response Response
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or r
hazardous materialsl-
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste wate
Don't Know
None
^f view
Other (please specify)
Percent Count
11.1% 2
5.6% 1
55.6% 10
22.2% 4
5.6% 1
11.1% 2
16.7% 3
16.7% 3
5.6% 1
16.7% 3
5.6% 1
answered question 18
skipped question 49
10.10
(Cost
. Has using NEWMOA and/or P2Rx or the action(s) taken above
resulted in cost savings to you?
includes time as well as dollars.) (Check only one answer.)
Yes
No
Do not know
If yes,
|
|
I
please provide details.
Response
Percent
27.8%
33.3%
38.9%
^ view
answered question
skipped question
Response
Count
5
6
7
1
18
49
Page: Questions for Assistance Provider Community
11.2. What type of organization <
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Jo you represent? (Check all that apply.)
Response Res
Percent C
I 18.5%
| 51.9%
3.7%
I 11.1%
| 11.1%
I 14.8%
I 11.1%
ponse
ount
5
14
1
3
3
4
3
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (4 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
Wastewater treatment industry! 1
s|)view other (please specify) r-.
7.4%
3.7%
answered question
skipped question
2
1
27
40
12. 3. How often do you use NEW
First time user
Weekly
Monthly
Every three months
••.$> view other (please specify)
/MOA and/or P2Rx? (Check only one answer.)
Response
Percent
I 7.4%
| 18.5%
22.2%
| OO.O70
I 1 R £1%
answered question
skipped question
Response
Count
2
5
6
9
5
27
40
13. 4. How did you hear al
Search
Link from another \
E-mail message or
Conference or n
Brochure or ne\
Col
Assistance p
s|> view other (please sp
)out NEWMOA and/or P2Rx? (Check only one answer.)
Response
Percent
enginel ^ 11.1%
/vebsiteQ 3.7%
listservl | 14.8%
neetingl | 18.5%
vsletter
league) |
0.0%
40.7%
Vendor 0.0%
rogram 0.0%
ecify) p=^ H 10/0
answered question
skipped question
Response
Count
3
1
4
5
0
11
0
0
3
27
40
14. 5. Access to the NEWMOA and/or P2Rx information has improved my ability to provide technical
assistance to my clients. (Check only one answer.)
Response Response
Percent Count
Strongly Agree) ~j 18.5% 5
H 40.7% 11
I 33.3% 9
0.0% 0
0.0% 0
7.4% 2
answered question 27
skipped question 40
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use| |
Page: Questions for Assistance Provider Community (continued)
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (5 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
15. 6. How useful were NEWMOA and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
iioofiii Useful neafiii Not useful of-Do not r^mmt
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
NEWMOA P2 & CA Metrics
Database Software
Mercury Reduction Programs
Database
Northeast States Activities
(Projects) Database
M^««l
34.8%
13.0%
13.0%
21.7%
34.8%
8.7%
8.7%
17.4%
4.3%
17.4%
4.3%
21.7%
8.7%
Ml
(8)
(3)
(3)
(5)
(8)
(2)
(2)
(4)
(1)
(4)
(1)
(5)
(2)
21.7%
30.4%
30.4%
21.7%
13.0%
8.7%
21.7%
34.8%
26.1%
13.0%
13.0%
21.7%
4.3%
(5)
(7)
(7)
(5)
(3)
(2)
(5)
(8)
(6)
(3)
(3)
(5)
(D
u
26
21
34
26
30
30
30
30
21
26
34
17
34
i^t^i i
.1%
.7%
.8%
.1%
.4%
.4%
.4%
.4%
.7%
.1%
.8%
.4%
.8%
.41
(6)
(5)
(8)
(6)
(7)
(7)
(7)
(7)
(5)
(6)
(8)
(4)
(8)
0.
4.
4.
8.
4.
13
0.
4.
8.
13
13
4.
4.
0% (0)
3% (1)
3% (1)
7% (2)
3% (1)
.0% (3)
0% (0)
3% (1)
7% (2)
.0% (3)
.0% (3)
o /o \\)
o /o \\)
use
17. 4% (4)
30.4% (7)
17. 4% (4)
2 1.7% (5)
17. 4% (4)
39.1% (9)
39.1% (9)
13.0% (3)
39.1% (9)
30.4% (7)
34.8% (8)
34.8% (8)
47.8% (11)
answered question
skipped question
V^XSUI 11
23
23
23
23
23
23
23
23
23
23
23
23
23
23
44
16. 7. What information did
environmental practices to
Provided training, works
and/or conference inform
Provided information on tech
assistance pro\
Provided Topic Hub inforrr
Provided vendor inforrr
f> vie./... other pollution
prevention technical assistan
so, please sp
^^^^^^^M
you use to improve or influence your clients' understanding of
reduce pollution? (Check all that apply.)
Response
Percent
atlon1 1 66'2%
nipol
• L 30 4%
/iH^TQ ww.-r /u
lation | 26.1%
lation | 13.0%
I 1 /.4%
ce. If
ecify.
answered question
skipped question
Response
Count
15
7
6
3
4
23
44
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have y
due to information you hav<
Changed a pollution preve
process or practice
implemented pollution preve
Identified a polk
nrox/ontirm nnnnrt
our clients taken to improve environmental practices, in whole
3 found through NEWMOA and/or P2Rx? (Check all that apply.)
Response
Percent
ntion
/; Q 1 QC no/.
ntion)
ition i
." 60.0%
linitw 1 \*vr.vr/u
or in part,
Response
Count
7
12
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (6 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
Purchased new process ,—,
equipment to prevent pollution'—'
Implemented energy conservation .—
measures'—
Switched to renewable energyl |
Contacted a vendor
Implemented material or waste ,—
recycling system'—
Changed handling of waste or.—
emission'—
Installed pollution control
equipment (e.g., scrubbers, Q
control technique)
Installed a waste treatment systeml |
No process changes were taken
\ vew
Other (please specify)
5.0%
10.0%
5.0%
0.0%
10.0%
20.0%
5.0%
5.0%
20.0%
5.0%
answered question
skipped question
1
4
1
20
47
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of NEWMOA
and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or
hazardous materials'—
Eliminated hazardous waste or,—
hazardous materials'-
Reduced fugitive air emissionsi
Eliminated air emissions
Reduced waste water pollutionl |
Eliminated pollutant waste water
Don't Know|
None|~~|
• vew
Other (please specify)
10.0%
5.0%
30.0%
10.0%
10.0%
0.0%
5.0%
0.0%
45.0%
5.0%
10.0%
answered question
skipped question
20
47
19.10.
client?
Has using NEWMOA and/or P2Rx or the action(s) taken above resulted
(Cost includes time as well as dollars.) (Check only one answer.)
Yes| 1
No
If yes, please provide details.
in cost savings to your
Response Response
Percent Count
30.0% 6
0.0% 0
70.0% 14
'•*§) view
1
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (7 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
answered question 20
skipped question 47
20.11. Has using NEWMOA and/or P2Rx saved you time or money in serving your client(s)? (Check
only one answer.)
Response Response
Percent Count
Yesl -
No
45.0% 9
] 20.0% 4
Do not knowl | 35.0% 7
If yes, please provide details.
answered question 20
skipped question 47
http://www.surveymonkey.com/sr.aspx?sm=nHW...cAo_2bXXnZdtlu5cLnW69ofsR5UWwiRvyW_2frUU_3d (8 of 8) [4/14/2008 4:16:23 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use WRRC and/or
P2Rx to find environmental
assistance information to apply [
to my own organization or
facility.
I primarily use WRRC and/or
P2Rx in my capacity as a
Response Response
Percent Count
50.0%
provider of environmental [
assistance to others outside
my organization.
50.0%
answered question
skipped question
17
17
34
Page: Questions for Regulated Community
2. 2. What type of organization do you represent? (Check all that apply.)
Response
Percent
Federal government 0.0%
State government! I 16.7%
Local government 0.0%
Manufacturing 25.0%
Service industry 25.0%
Educational institution 25.0%
Nonprofit organization 0.0%
Wastewater treatment industry 0.0%
V view Other (please specify) i — I 25 0°/
| | fcw»w /O
answered question
skipped question
Response
Count
0
2
0
3
3
3
0
0
3
12
22
3. 3. How often do you use WRR(
First time user
Weekly
Monthly
Every three months
Other (please specify)
1 and/or P2Rx? (Check only one answer.)
Response Res
Percent C<
I *>8 *?%
| 167%
| 167%
I 8.3%
0.0%
answered question
ponse
aunt
7
2
2
1
0
12
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (1 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
skipped question
22
4. 4. How did you hear about WRRC and/or P2Rx? (Check only one answer.)
Respon
Percen
Search engine
Link from another website
E-mail message or listserv_
Conference or meeting
Brochure or newsletter
Colleague
Vendor
Assistance program^
••$> view other (please specify) r-
| 33
H 25
8
0
8
8
se
t
.3%
.0%
.3%
.0%
.3%
.3%
0.0%
3 8.3%
=3 8.3%
answered question
skipped question
Response
Count
4
3
1
0
1
1
0
1
1
12
22
5. 5. WRRC and/or P2Rx helps me to improve my awareness of environmental practices to re
pollution. (Check only one answer.)
Response R
Percent
Agree I 25.0%
Neutral) | 8.3%
Disagree 0.0%
Strongly Disagree 0.0%
Do Not Usej | 8.3%
answered question
skipped question
duce
esponse
Count
7
3
1
0
0
1
12
22
Page: Questions for Regulated Community (continued)
6. 6. How useful were WRRC and/or P2Rx resources in
selection for each resource listed.)
meeting your needs? (Please make one
Unaware
oornewMai M.»A >_f • **.& r^.» .**.! rxesponse
• •oafiii Not useful of-Do not n /;••«*
userui uo^iui use wwuiii
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information (i.e.,
Best References Collection)
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
33.3%
33.3%
33.3%
44.4%
22.2%
22.2%
22.2%
33.3%
33.3%
33.3%
(3)
(3)
(3)
(4)
(2)
(2)
(2)
(3)
(3)
(3)
33.3%
33.3%
22.2%
33.3%
44.4%
33.3%
22.2%
22.2%
44.4%
22.2%
(3)
(3)
(2)
(3)
(4)
(3)
(2)
(2)
(4)
(2)
0.0% (0)
0.0% (0)
11.1% (1)
0.0% (0)
0.0% (0)
11.1% (1)
11.1% (1)
11.1% (1)
0.0% (0)
0.0% (0)
0
0
0
0
0
0
0
0
0
0
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
33.3%
33.3%
33.3%
22.2%
33.3%
33.3%
44.4%
33.3%
22.2%
44.4%
(3)
(3)
(3)
(2)
(3)
(3)
(4)
(3)
(2)
(4)
answered question
9
9
9
9
9
9
9
9
9
9
9
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (2 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
skipped question
25
7. 7. How did you utilize the WRRC and/or
Reviewed Topic Hub
information (or otner WRRC/i ~^^^—
P2Rx website information)
Attended a training, workshop, or
conference promoted by a P2Rx | |
Center
Asked for pollution prevention •
Requested additional resources •
frnm n P°Ry Prntrr
Contacted a technical assistance . .
provider' ^
sfiview other (please specify^ [— ^
P2Rx website(s)? (Check all that apply.)
Response
Percent
| 55.6%
11.1%
22.2%
| 33.3%
11.1%
11.1%
answered question
skipped question
Response
Count
5
1
2
3
1
1
9
25
Page: Questions for Regulated Community (continued)
28.6%
0.0%
Contacted a vendor
Implemented material or waste r-
recycling systemL
Changed handling of waste or r
emissionL
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken[
Other (please specify)
14.3%
14.3%
0.0%
0.0%
14.3%
0.0%
answered question
skipped question
0
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part, due to
information you have found through WRRC and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a pollution prevention
process or practice (i.e., 0.0%
implemented pollution prevention)
Identified a pollution ,
prevention opportunity
Purchased new process r 28 60/
equipment to prevent pollution1 '
Implemented energy conservation
measures
Switched to renewable energyl ^ 14.3%
=| 28.6%
27
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (3 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of WRRC and/or P2Rx
use. (Check all that apply.)
Response Response
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or r
hazardous materials'-
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution[
Eliminated pollutant waste water
Know
None
vew
Other (please
Percent Count
0.0%
0.0%
14.3%
0.0%
0.0%
0.0%
14.3%
0.0%
42.9%
14.3%
14.3%
answered question
skipped question 27
0
0
10.10. Has using WRRC and/or P2Rx or the action(s) taken above resulted in cost savings to you?
(Cost includes time as well as dollars.) (Check only one answer.)
Response Response
Percent Count
YesF ~~l 28.6% 2
28.6% 2
1 42.9% 3
If yes, please provide details. 0
answered question 7
skipped question 27
No
Do not know
Page: Questions for Assistance Provider Community
11. 2. What type of ore
Federal
State g
Local
Ms
Serv
Education,
Nonprofit c
Wastewater treatm
sf> view other (oleas
janization do you represent? (Check all that apply.)
Response
Percent
government | 6.3
UVclTllllclll •^^^^^^^^^^^^^^^^^^^^^^^^^™" 1 0.U
%
government 0.0%
mufacturing 0.0%
ice industry 0.0%
al institution^ 6.3%
jrganization 0.0%
ent industry 0.0%
e specify) i i <\ 2 5%
answered question
skipped question
Response
Count
1
12
0
0
0
1
0
0
2
16
18
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (4 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
12. 3. How often do you use WRRC and/or P2Rx? (Check only one answer.)
\f> view
First time
Weekly
Monthly
Every three months
Other (please specify)
Response Response
Percent Count
25.0%
43.8%
12.5%
12.5%
6.3%
answered question
skipped question
18
16
13. 4. How did you hear about WRRC and/or P2Rx? (C
Search engine) |
Link from another website! I
E-mail message or listserv
Conference or meetingl |
Brochure or newsletter
Vendor
Assistance program! I
•jHie- other (please specify) r— i
?heck only one answer.)
Response Response
Percent Count
18.8% 3
18.8% 3
0.0% 0
12.5% 2
0.0% 0
37.5% 6
0.0% 0
6.3% 1
6.3% 1
answered question 16
skipped question 18
14. 5. Access to the WRRC and/or P2Rx information has improved my ability to provide technical
assistance to my clients. (Check only one answer.)
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
Response
Percent
37.5%
56.3%
6.3%
0.0%
0.0%
0.0%
answered question
skipped question
Response
Count
6
9
1
0
0
0
16
18
Page: Questions for Assistance Provider Community (continued)
15. 6. How useful were WRRC and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
Topic Hubs 26.7% (4) 53.3% (8) 13.3% (2) 0.0% (0) 6.7% (1)
15
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (5 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
P2 Programs Directory 40.0% (6) 13.3% (2) 13. 3% (2)
News 0.0% (0) 40.0% (6) 33.3% (5)
Library 40.0% (6) 40.0% (6) 6.7% (1)
Industry Sector Information (i.e., ~o o0/ /c\ e-i w /o\ n no/ /m
Best References Collection) 33'3/0 (5) 53'3/0 (8) °-0% (0)
Vendor Database 13.3% (2) 13.3% (2) 33.3% (5)
Case Studies 33.3% (5) 46.7% (7) 6.7% (1)
Listservs 6.7% (1) 13.3% (2) 13. 3% (2)
Rapid Response 13.3% (2) 26.7% (4) 6.7% (1)
P2 "gSjgfSS, 26-™ <*> 26'™ W «.3% (2)
13. 3% (2) 20.0% (3)
6.7% (1) 20.0% (3)
0.0% (0) 13.3% (2)
6.7% (1) 6.7% (1)
13. 3% (2) 26. 7% (4)
6.7% (1) 6.7% (1)
13. 3% (2) 53.3% (8)
0.0% (0) 53.3% (8)
6.7% (1) 26.7% (4)
answered question
skipped question
15
15
15
15
15
15
15
15
15
15
19
16. 7. What information did you use to improve or influence your cli<
environmental practices to reduce pollution? (Check all that apply.)
Provided traininq, workshop, andA 1
Provided information on fprhniral •
provided Topic HUD information |
Provided vendor information I
<|) view other pollution
I
prevention technical assistance. If
so, please specify.
jnts1 understanding of
Response
Percent
53.3%
40.0%
60.0%
26.7%
26.7%
answered question
skipped question
Response
Count
8
6
9
4
4
15
19
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmer
due to information you have found through WRRC and/or P2Rx? (Cl
Changed a pollution prevention
implemented pollution prevention)
Identified a pollution , ,
nrAVAntion onnorfiinitw I
Piirphn^pd npw nrnpp^^ .
oni linmont tn nrov/ont nnlh itinn
Imnlpmpntpd pnprnv pnn^prv/ntinn .
mocici i roc
Switched to renewable energy) |
Contacted a vendor! ^
Imnlpmpntpd matprial orwa^tp •
Changed handling of waste or . — ,
tal practices, in whole
leek all that apply.)
Response
Percent
71.4%
78.6%
42.9%
35.7%
7.1%
14.3%
64.3%
50.0%
or in part,
Response
Count
10
11
6
5
1
2
9
7
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (6 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
Installed pollution control
equipment (e.g., scrubbers, [
control technique)
Installed a waste treatment system
No process changes were taken
view
Other (please specify)
21.4%
21.4%
7.1%
7.1%
answered question
skipped question
20
3
1
1
14
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of WRRC
and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
1 57.1% 8
| 42.9% 6
| 42.9% 6
^ 35.7% 5
| 28.6% 4
14.3% 2
| 42.9% 6
14.3% 2
| 28.6% 4
0.0% 0
7.1% 1
answered question 14
skipped question 20
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste r
(solid waste)L
Reduced hazardous waste or r
hazardous materialsL
Eliminated hazardous waste or r
hazardous materialsL
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
• vew
Other (please specify)
19.10.
client?
Has using WRRC and/or P2Rx
(Cost includes time as well as
Yes|
No
Do not know|
or the action(s) taken
dollars.) (Check only
1
I
If yes, please provide details.
above resulted in cost savings to your
one answer.)
Response Response
Percent Count
57.1% 8
0.0% 0
42.9% 6
^ view
answered question
skipped question
2
14
20
20.11. Has using WRRC and/or P2Rx saved you time or money in serving your client(s)? (Check only
one answer.)
Response Response
Percent Count
Yes
No
71.4%
7.1%
10
1
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (7 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
Do not know| I
If yes, please provide details.
21.4%
-•' view
answered question
skipped question
3
2
14
20
http://www.surveymonkey.com/sr.aspx?sm=6OJXR_2fXZwN2CXIPddiEWNwcBYRhLglO_2fjGwEgVh3dxk_3d (8 of 8) [4/14/2008 4:43:45 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use GLRPPR and/or
P2Rx to find environmental
assistance information to apply [
to my own organization or
facility.
I primarily use GLRPPR and/or
P2Rx in my capacity as a provider ,-
of environmental assistance to <-
others outside my organization.
Response Response
Percent Count
54.0%
46.0%
answered question
skipped question
27
23
50
Page: Questions for Regulated Community
2. 2. What type of organization d<
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
f> view other (please specify)
D you represent? (Check all that apply.)
Response
Percent
^•^ I iy.u%
I 19.0%
| 14.3%
| 28.6%
^ 4.8%
4.8%
4.8%
0.0%
=| 9.5%
answered question
skipped question
Response
Count
4
4
3
6
1
1
1
0
2
21
29
3. 3. How often do you use GLRP
First time user
Weekly
Monthly
Every three months
•^> view other (please specifv)
•PR and/or P2Rx? (Check only one answer.)
^^^^»- |
1
^ '
1
Response
Percent
33.3%
19.0%
19.0%
9.5%
19.0%
Response
Count
7
4
4
2
4
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (1 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
answered question 21
skipped question 29
4. 4. How did you hear about GLRPPR and/or P2Rx? (Check only one answer.)
Response Response
Percent Count
Search engine
Link from another website
E-mail message or listserv|~ ~| 28.6% 6
Conference or meeting! I 19.0% 4
J 4.8% 1
I 14.3% 3
23.8% 5
] 9.5% 2
Brochure or newsletter
Colleague
Vendor 0.0% 0
Assistance program 0.0% 0
Other (please specify) 0.0% 0
answered question 21
skipped question 29
5. 5. GLRPPR and/or P2Rx helps me to improve my awareness of environmental practices to reduce
pollution. (Check only one answer.)
Response Response
Percent Count
Strongly Agreel —
Agree
3 19.0% 4
52.4% 11
19.0% 4
Disagree 0.0% 0
Strongly Disagree 0.0% 0
Do Not Usej | 9.5% 2
answered question 21
skipped question 29
Page: Questions for Regulated Community (continued)
6. 6. How useful were GLRPPR and/or P2Rx resources
selection for each resource listed.)
in meeting your needs? (Please make one
Unaware
oornewMai M.»A >_f • **.& r^.» .**.! txesponse
• •oafiii Not useful of-Do not n /;••«*
userui uo^iui use wwuiii
Topic Hubs
P2 Programs Directory (i.e.,
Contacts Database)
News
Library (i.e., P2 Infohouse)
Industry Sector Information (i.e.,
Sector Resources)
Vendor Database
Case Studies
Listservs (not including P2Tech
Rapid Response (i.e., Help Desk)
P2 Results Data System
(Measurement Tool)
P2Tech Listserv
23.1% (3)
7.7% (1)
46.2% (6)
23.1% (3)
23.1% (3)
7.7% (1)
15.4% (2)
23.1% (3)
23.1% (3)
7.7% (1)
15.4% (2)
38
46
23
30
23
15
46
15
15
23
15
.5%
.2%
.1%
.8%
.1%
.4%
.2%
.4%
.4%
.1%
.4%
(5)
(6)
(3)
(4)
(3)
(2)
(6)
(2)
(2)
(3)
(2)
7.7% (1)
23.1% (3)
7.7% (1)
15.4% (2)
15.4% (2)
30.8% (4)
7.7% (1)
30.8% (4)
15.4% (2)
15.4% (2)
30.8% (4)
0
0
0
0
0
0%
0%
0%
0%
0%
15.4%
0
0
0
7
0
0%
0%
0%
7%
0%
(0)
(0)
(0)
(0)
(0)
(2)
(0)
(0)
(0)
(1)
(0)
30.8%
23.1%
23.1%
30.8%
38.5%
30.8%
30.8%
30.8%
46.2%
46.2%
38.5%
(4)
(3)
(3)
(4)
(5)
(4)
(4)
(4)
(6)
(6)
(5)
13
13
13
13
13
13
13
13
13
13
13
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (2 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
GLRPPRBlog 15.4% (2) 15.4% (2) 23.1% (3)
Funding Opportunities List 15.4% (2) 15.4% (2) 7.7% (1)
Online Calendar 23.1% (3) 30.8% (4) 15.4% (2)
LINK Newsletter 23.1% (3) 23.1% (3) 23.1% (3)
Projects Database 7.7% (1) 15.4% (2) 15.4% (2)
0.0% (0) 46.2% (6)
7.7% (1) 53.8% (7)
0.0% (0) 30.8% (4)
0.0% (0) 30.8% (4)
7.7% (1) 53.8% (7)
answered question
skipped question
37
13
13
13
13
13
13
7. 7. How did you utilize the GLRPPR and/or P2Rx website(s)? (Check all that apply.)
Response Response
Reviewed Topic Hub
information (or other GLRPPR/[
P2Rx website information)
Attended a training, workshop, or
conference promoted by a P2Rx [
Center
Asked for pollution prevention r-
technical assistanceL
Requested additional resources r
from a P2Rx CenterL
Contacted a technical assistance
providen
•.. view
Other (please specify)
Percent Count
76.9% 10
30.8% 4
23.1% 3
7.7% 1
15.4% 2
7.7% 1
answered question 13
skipped question 37
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in
information you have found through GLRPPR and/or P2Rx? (Check all that apply.)
Respons
Percen
Changed a pollution prevention
process or practice (i.e.,
implemented pollution prevention)
Identified a nolliition .-
nrAVAntion onnorfiinitw
Purchased new process r
equipment to prevent pollutionL
Imnlpmpntprl pnprnv pnno-prv/ptinn —
mpaci i roc -
Switched to renewable energy
Contacted a vendor
Implemented material or waste r
rppvplinn QVQtpm -
Hhanopd handlinn of wa^tp or •-
pmiQQinn
1 18.
i e-»
DO.
=1 9.
1 ^R
| OD.
9.
9.
| 45.
| 18.
Installed pollution control
equipment (e.g., scrubbers, 0.
control technique)
Installed a waste treatment system 0.
No process changes were taken| | 9.
part, due to
>e Response
t Count
2%
6%
1%
4%
1%
1%
5%
2%
0%
0%
1%
2
7
1
4
1
1
5
2
0
0
1
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (3 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
\ vew
Other (please specify)
9.1%
answered question
skipped question
1
11
39
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of GLRPPR and/or
P2Rx use. (Check all that apply.)
Response Response
Percent Count
| 36.4% 4
3 18.2% 2
| 36.4% 4
9.1% 1
0.0% 0
9.1% 1
9.1% 1
9.1% 1
0.0% 0
3 18.2% 2
9.1% 1
answered question 11
skipped question 39
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste ,
(solid waste/
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or f
hazardous materials^
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None[
\3f view
Other (please specify)
10.10. Has using GLRPPR and/or P2Rx or the action(s) taken above resulted in cost savings to you?
(Cost includes time as well as dollars.) (Check only one answer.)
Response Response
Percent Count
YesF ~~l 18.2%
36.4%
45.5%
No
Do not know
If yes, please provide details.
answered question
skipped question
39
11
Page: Questions for Assistance Provider Community
11.
2. What type of organization <
Federal government
State government
Local government
Manufacturing
Jo you represent? (Check all that apply.)
Response
Percent
^^^^^ 25.0%
| 40.0%
10.0%
10.0%
Response
Count
5
8
2
2
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (4 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
s|> view other (please specify)
| | 10.0%
I 10.0%
^ 5.0%
0.0%
answered question
skipped question
2
2
1
0
3
20
30
12. 3. How often do you u,
First tin
IV
Every three
s|> view other (please sp
se GLRPPR and/or P2Rx? (Check only one answer.)
ne user |
weekly |
lonthly |
months
Response
Percent
15.0%
25.0%
30.0%
20.0%
ecify) [— =1 100o/0
answered question
skipped question
Response
Count
3
5
6
4
2
20
30
13. 4. How did you hear al
Search
Link from another >
E-mail message or
Conference or r
Brochure or ne\
Col
Assistance p
.|> view other release so
Dout GLRPPR and/or P2Rx? (Check only one answer.)
Response
Percent
engineQ 5.0%
website! I 15.0%
listservl | 20.0%
yvsletter
league) |
Vendor
rograml |
15.0%
0.0%
25.0%
0.0%
15.0%
ecify) r-| 50o/0
answered question
skipped question
Response
Count
1
3
4
3
0
5
0
3
1
20
30
14. 5. Access to the GLRPPR and/or P2Rx information has improved my ability to provide technical
assistance to my clients. (Check only one answer.)
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
Response
Percent
30.0%
30.0%
30.0%
0.0%
0.0%
10.0%
answered question
skipped question
Response
Count
6
6
6
0
0
2
20
30
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (5 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
Page: Questions for Assistance Provider Community (continued)
15. 6. How useful were GLRPPR and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
?y. Useful S°"le«1at Not useful of"oo not
useful useful ,,__
use
Topic Hubs 50.0% (8) 12.5% (2) 18.8% (3) 0.0% (0) 18.8% (3)
P2 Programs Directory (i.e., 9= no/ /4\ A* oo/ m « -w f-n « -w m I««O//Q\
Contacts Database) 25.0 /o (4) 43.8/o(7) 6.3% (1) 6.3% (1) 18.8% (3)
News 12. 5% (2) 25.0% (4) 31.3% (5) 6.3% (1) 25.0% (4)
Library (i.e., P2 Infohouse) 18. 8% (3) 18.8% (3) 37.5% (6) 0.0% (0) 25.0% (4)
Industry Secto^nJorm^nJLe^ 37>fi% (6) 37>fi% (6) QQ% ((J) QQ% ((J) 25Q% (4)
Vendor Database 12.5% (2) 12.5% (2) 18.8% (3) 12.5% (2) 43.8% (7)
Case Studies 25.0% (4) 37.5% (6) 18. 8% (3) 0.0% (0) 18.8% (3)
Listservs (not including P2Tech , 2 5o/0 (2) 18.8%(3) 12.5% (2) 18.8% (3) 37.5% (6)
Rapid Response (i.e., Help Desk) 0.0% (0) 18.8% (3) 0.0% (0) 25.0% (4) 56.3% (9)
P2 TMealu^ementTod) 6-3%(1) 18-8%(3) 18.8% (3) 25.0% (4) 31.3% (5)
P2Tech Listserv 25.0% (4) 12.5% (2) 18. 8% (3) 6.3% (1) 37.5% (6)
GLRPPR Blog 6.3% (1) 18.8% (3) 12.5% (2) 25.0% (4) 37.5% (6)
Funding Opportunities List 6.3% (1) 43.8% (7) 18.8% (3) 6.3% (1) 25.0% (4)
Online Calendar 6.3% (1) 31. 3% (5) 12.5% (2) 12.5% (2) 37.5% (6)
LINK Newsletter 6.3% (1) 56.3% (9) 12.5% (2) 0.0% (0) 25.0% (4)
Projects Database 0.0% (0) 31.3% (5) 25.0% (4) 12.5% (2) 31.3% (5)
answered question
skipped question
Count
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
34
16. 7. What information did you use to improve or influence your clients' understanding of
environmental practices to reduce pollution? (Check all that apply.)
Response
Percent
Providpd tr^ininn work^hon ^nd/
Provided information on tprhnir^l
Provided vendor information| I 31.3%
.|> view other pollution
II 1 R R%
|| I O . O /O
prevention technical assistance. If
so, please specify.
answered question
skipped question
Response
Count
7
/
7
/
8
5
3
16
34
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmental practices, in whole
due to information you have found through GLRPPR and/or P2Rx? (Check all that apply.)
or in part,
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (6 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
Changed a pollution prevention
process or practice (i.e., [
implemented pollution prevention)
Identified a pollution
prevention opportunity!
Purchased new process,
equipment to prevent pollution!
Implemented energy conservation ,
measures!
Switched to renewable energy
Contacted a vendor
Implemented material or waste r
recycling systeml-
Changed handling of waste or r
emissionl-
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken[
,J view
Other (please specify)
Response Response
Percent Count
40.0% (
46.7% 7
33.3% 5
33.3%
13.3% 2
33.3% 5
33.3% 5
26.7% 4
0.0% 0
0.0% 0
20.0% 3
26.7%
answered question
skipped question
4
15
35
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of GLRPPR
and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
20.0% ;
13.3% ;
3 26.7%
13.3% ;
13.3% 2
6.7% 1
| 33.3% 5
6.7% 1
| 46.7% 7
13.3% 2
13.3%
answered question 15
skipped question 35
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste ,
(solid waste/
Reduced hazardous waste or f
hazardous materials^
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
•.. vew
Other (please specify)
19.10. Has using GLRPPR and/or P2Rx or the action(s) taken above resulted in cost savings to your
client? (Cost includes time as well as dollars.) (Check only one answer.)
Response Response
Percent Count
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (7 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
Yes
No
Do not know
If yes,
1
1
|
please provide details.
33.3%
13.3%
53.3%
^ view
answered question
skipped question
5
2
8
2
15
35
20.11. Has using GLRPPR and/or P2Rx saved you time or money in serving your client(s)? (Check
only one answer.)
Response Response
Percent Count
Yes| I 73.3%
No| | 6.7%
Do not knowl =| 20.0%
If yes, please provide details.
11
1
3
8
answered question
skipped question
15
35
http://www.surveymonkey.com/sr.aspx?sm=p4g89CT537vOyrEIistFxlaiZ6d6ecbrl9MGxFU3WJI_3d (8 of 8) [4/14/2008 4:44:20 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use the Zero Waste
Network and/or P2Rx to find
environmental assistance [
information to apply to my own
organization or facility.
I primarily use the Zero Waste
Network and/or P2Rx in my
capacity as a provider of [
environmental assistance to
others outside my organization.
Response Response
Percent Count
40.7%
59.3%
answered question
skipped question
11
16
27
-2
Page: Questions for Regulated Community
2. 2. What type of organization do you represent? (Check all that apply.)
Response
Percent
Federal governmentl | 10.0%
State government 0.0%
Local government | 50.0%
Manufacturing 10.0%
Service industry 10.0%
Educational institution 10.0%
Nonprofit organization 0.0%
Wastewater treatment industry 0.0%
.fHiew other (please specify) . . mrw
answered question
skipped question
Response
Count
1
0
5
1
1
1
0
0
1
10
15
3. 3. How often do you use the Zero Waste Network and/or P2Rx? (Check only one answer.)
Response Res|
Percent Cc
First time user | 50.0%
Weekly | 10.0%
Monthly 0.0%
Lvery three months! I 20.0%
Dense
>unt
5
1
0
2
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (1 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
vew
Other (please specify)
20.0%
answered question
skipped question
2
10
15
4. 4. How did you hear ab<
Search
Link from another \
E-mail message or I
Conference or n
Brochure or ne\
Co
Assistance p
•-.f) view other (please sp
Dut the Zero Waste Network and/or P2Rx? (Check only
Re
P
engine |
website |
istserv |
neetmg |
one answer.)
sponse Response
ercent Count
20.0% 2
10.0% 1
30.0% 3
20.0% 2
vsletter 0.0% 0
league 0.0% 0
Vendor 0.0% 0
rograml | 10.0% 1
ecify) [-=[ 10.0% 1
answered question 10
skipped question 15
5. 5. The Zero Waste Network and/or P2Rx helps me to improve my
practices to reduce pollution. (Check only one answer.)
Strongly Agree
Agree
Neutral I
Disagree
Strongly Disagree
Do Not Use| |
^^^^^^^^^m^^^^^^^^^^m
awareness of environmental
Response
Percent
20.0%
20.0%
30.0%
0.0%
0.0%
30.0%
answered question
skipped question
Response
Count
2
2
3
0
0
3
10
15
Page: Questions for Regulated Community (continued)
6. 6. How useful were the Zero Waste Network and/or P2Rx resources in meeting your needs? (Please
make one selection for each resource listed.)
Ve7 . Useful Somewhat
useful "»«'"' useful
Topic Hubs 28.6% (2) 14.3% (1) 0.0% (0)
P2 Programs Directory 14.3% (1) 14.3% (1) 0.0% (0)
News 14. 3% (1) 0.0% (0) 14. 3% (1)
Library 14.3% (1) 0.0% (0) 0.0% (0)
Industry Sector Information 14.3% (1) 14.3% (1) 0.0% (0)
Vendor Database 14.3% (1) 0.0% (0) 0.0% (0)
Case Studies 14.3% (1) 28.6% (2) 0.0% (0)
Listservs 28.6% (2) 0.0% (0) 0.0% (0)
Rapid Response 14.3% (1) 0.0% (0) 0.0% (0)
Unaware
Not useful of-Do not
use
0.0% (0) 57.1% (4)
0.0% (0) 71. 4% (5)
0.0% (0) 71. 4% (5)
0.0% (0) 85.7% (6)
0.0% (0) 71. 4% (5)
0.0% (0) 85.7% (6)
0.0% (0) 57.1% (4)
0.0% (0) 71. 4% (5)
0.0% (0) 85.7% (6)
Response
Count
7
7
7
7
7
7
7
7
7
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (2 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
P2 Results Data System
(Measurement Tool)
P2 Planner
P2 Options
Spanish Materials
RENEW (Resource Exchange)
Workshops/Presentations
Site Visits
14.3% (1) 0.0% (0) 0.0% (0) 0.0% (0) 85.7% (6)
14.3% (1) 14.3% (1) 0.0% (0) 0.0% (0) 71.4% (5)
14.3% (1) 0.0% (0) 0.0% (0) 0.0% (0) 85.7% (6)
14.3% (1) 0.0% (0) 0.0% (0) 0.0% (0) 85.7% (6)
14.3% (1) 14.3% (1) 0.0% (0) 0.0% (0) 71.4% (5)
14.3% (1) 0.0% (0) 0.0% (0) 0.0% (0) 85.7% (6)
14.3% (1) 0.0% (0) 0.0% (0) 0.0% (0) 85.7% (6)
answered question
skipped question
18
7. 7. How did you utilize the Zero Waste Network and/or P2Rx website(s)? (Check all that apply.)
Response Response
Count
Reviewed Topic Hub information
(or other Zero Waste Network/[
P2Rx website information)
Attended a training, workshop, or
conference promoted by a P2Rx [
Center
Asked for pollution prevention
technical assistance
Requested additional resources
from a P2Rx Center
Contacted a technical assistance r
provider!-
\=s view
Other (please specify)
Percent
28.6%
28.6%
0.0%
0.0%
14.3%
57.1%
answered question
skipped question
18
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental
information you have found through the Zero Waste Network
Changed a pollution prevention
process or practice (i.e., | -^^— |
implemented pollution prevention)
Identified a nollution nrevention • i
Purchased new process
equipment to prevent pollution
Imolemented enerov conservation i i
Switched to renewable energy) |
Contacted a vendor! I
Imnlpmpntprl mntprinl or wn^tp .
Phnnnprl hnnrllinn of wn^tp or .
practices, in whole or in part, due to
and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
28.6%
14.3%
0.0%
14.3%
14.3%
14.3%
14.3%
14.3%
2
1
0
1
1
1
1
1
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (3 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken
Other (please specify)
0.0%
0.0%
71.4%
0.0%
answered question
skipped question
18
9. 9. Please identify whether you reduced or eliminated
Network and/or P2Rx use. (Check all that apply.)
Rprli ipprl nnn hn~7nrrlni i^ wn^tp .
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or • — — ,
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions! H
Eliminated air emissions
Reduced waste water pollution! ~\
Eliminated pollutant waste water
Don't Know
None)
Other (please specify)
pollutant(s) as a result of the Zero
Response
Percent
28.6%
0.0%
28.6%
0.0%
14.3%
0.0%
14.3%
0.0%
0.0%
| 71.4%
0.0%
answered question
skipped question
Waste
Response
Count
2
0
2
0
1
0
1
0
0
5
0
7
18
10. 10. Has using the Zero Waste Network and/or P2Rx or the action(s) taken above resulted in cost
savings to you? (Cost includes time as well as dollars.) (Check only one answer.)
Response Response
Percent Count
Yes
No
Do not know
| 14.3%
| 28.6%
| O / . 1 /O
If yes, please provide details.
answered question
skipped question 18
1
2
4
0
7
Page: Questions for Assistance Provider Community
11.2. What type of organization do you represent? (Check all that apply.)
Response Response
Percent Count
Federal government 0.0%
State government
Local government
Manufacturing
Service industry
Educational institution
I 33.3%
| 40.0%
0.0%
13.3%
13.3%
0
5
6
0
2
2
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (4 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
Nonprofit c
Wastewater treatm
f> view other (oleas
rganizationQ
snt industry^ 3
6.7%
13.3%
9 specify) ^ 6 1%
answered question
skipped question
1
2
1
15
10
12. 3. How often do you use the Zero Waste Network and/or P2Rx? (Check only one answer.)
First time user
Weekly
Monthly
Every three months
Other (please specify)
Response
Percent
13.3%
20.0%
13.3%
53.3%
0.0%
answered question
skipped question
Response
Count
2
3
2
8
0
15
10
13. 4. How did you hear aboi
Search en
Link from another we
E-mail message or list
Conference or mee
Brochure or newsl
Collea
Ve
Assistance proc
sf> view other (please sped
jt the Zero Waste Network and/or P2Rx? (Check only
Res
Pe
gine|
DsiteQ
serv |
ting |
etter
iguei |
one answer.)
>ponse Response
srcent Count
6.7% 1
6.7% 1
20.0% 3
26.7% 4
0.0% 0
26.7% 4
ndor 0.0% 0
jrarrQ 6.7% 1
fy) Q 6.7% 1
answered question 15
skipped question 10
14. 5. Access to the the Zero Waste Network and/or P2Rx information has improved my ability to
provide technical assistance to my clients. (Check only one answer.)
Response Response
Percent Count
| 26.7% 4
| 46.7% 7
I 26.7% 4
0.0% 0
0.0% 0
0.0% 0
answered question 15
skipped question 10
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
Page: Questions for Assistance Provider Community (continued)
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (5 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
15. 6. How useful were the Zero Waste Network and/or
(Please make one selection for each resource listed.)
P2Rx resources in meeting your needs?
Unaware
useful se u useful lwl ""'"' "'"use""1 Count
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
P2 Planner
P2 Options
Spanish Materials
RENEW (Resource Exchange)
Workshops/Presentations
Site Visits
46.2%
30.8%
23.1%
30.8%
30.8%
7.7%
53.8%
30.8%
7.7%
23.1%
7.7%
30.8%
7.7%
30.8%
23.1%
30.8%
(6)
(4)
(3)
(4)
(4)
(1)
(7)
(4)
(1)
(3)
(1)
(4)
(1)
(4)
(3)
(4)
30
23
46
30
46
30
30
23
23
38
30
30
0.
30
30
7.
.8%
.1%
.2%
.8%
.2%
.8%
.8%
.1%
.1%
.5%
.8%
.8%
0%
.8%
.8%
7%
(4)
(3)
(6)
(4)
(6)
(4)
(4)
(3)
(3)
(5)
(4)
(4)
(0)
(4)
(4)
(1)
0.0% (0)
15.4% (2)
15.4% (2)
15.4% (2)
7.7% (1)
30.8% (4)
7.7% (1)
OO -1 O/ /Q\
ZO. 1 /O (O)
15.4% (2)
7.7% (1)
7.7% (1)
7.7% (1)
7 "7 O/ / d \
. / /O \ \ I
0.0% (0)
15.4% (2)
7 "7 O/ / d \
. / /O \ \ I
0
0
0
0
7
7
0
0
0
0
0% (0)
0% (0)
0% (0)
0% (0)
7% (1)
7% (1)
0% (0)
0% (0)
0% (0)
0% (0)
15.4% (2)
0
7
0
0
0
0% (0)
7% (1)
0% (0)
0% (0)
0% (0)
23.1%
30.8%
15.4%
23.1%
7.7%
23.1%
7.7%
23.1%
53.8%
30.8%
38.5%
30.8%
76.9%
38.5%
30.8%
53.8%
(3)
(4)
(2)
(3)
(1)
(3)
(1)
(3)
(7)
(4)
(5)
(4)
(10)
(5)
(4)
(7)
answered question
skipped question
13
13
13
13
13
13
13
13
13
13
13
13
13
13
13
13
13
12
16. 7. What information did
environmental practices to
Provided training, workshop
or conference inforrr
Provided information on tech
assistance pro\
Provided Topic Hub inform
Provided vendor inforrr
,|> view other pollution
prevention technical assistan
so, please sp
you use to improve or influence your clients' understanding of
reduce pollution? (Check all that apply.)
Response
Percent
lotion ' 4b'2/0
/idcr- ' '30-8%
ation | 53.8%
1 23.1%
ce. If
ecify.
answered question
skipped question
Response
Count
6
4
7
5
3
13
12
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have y
due to information you hav<
apply.)
Changed a pollution preve
process or practice
implemented pollution preve
our clients taken to improve environmental practices, in whole
3 found through the Zero Waste Network and/or P2Rx? (Check
Response
Percent
ntion
(i.e., | 53.8%
ntion)
or in part,
all that
Response
Count
7
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (6 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
Identified a pollution
prevention opportunity
Purchased new process,
equipment to prevent pollution^
Implemented energy conservation ,
measures^
Switched to renewable energy
Contacted a vendor[
Implemented material or waste ,
recycling system^
Changed handling of waste or,
emission^
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken
Other (please specify)
| 84.6%
30.8%
46.2%
0.0%
30.8%
46.2%
53.8%
7.7%
7.7%
15.4%
0.0%
answered question
skipped question
11
4
6
0
4
6
7
1
1
2
0
13
12
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of the Zero
Waste Network and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
Reduced non-hazardous waste, , Qn Q0/
(solid waste)' ^^ 30.8 /o
Eliminated non-hazardous waste •
(solid waste)1
Reduced hazardous waste or
hazardous materials^
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
\*f view
Other (please specify)
30.8%
0.0%
7.7%
7.7%
38.5%
23.1%
23.1%
0.0%
23.1%
answered question
skipped question
0
1
1
5
3
3
0
13
12
19.10. Has using the Zero Waste Network and/or P2Rx or the action(s) taken above resulted in cost
savings to your client? (Cost includes time as well as dollars.) (Check only one answer.)
Response
Percent
Yesj | 61.5%
No 0.0%
^= I 38.5%
Response
Count
Do not know[
If yes, please provide details.
answered question
8
0
5
13
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (7 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
I skipped question 12
20.11. Has using the Zero Waste Network and/or P2Rx saved you time or money in serving your client
(s)? (Check only one answer.)
Response Response
Percent Count
Yesj ] 46.2% 6
No| ] 7.7% 1
=^= 1 46.2% 6
Do not know
If yes, please provide details.
answered question 13
skipped question 12
http://www.surveymonkey.com/sr.aspx?sm=JFDhjw3o2wQKOFFmXq_2fzeZAj87SXzmuMRen9wJA_2f66I_3d (8 of 8) [4/14/2008 4:44:52 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use P2RIC and/or
P2Rx to find environmental r-
assistance information to apply to <-
my own organization or facility.
I primarily use P2RIC and/or
P2Rx in my capacity as a
Response Response
Percent Count
provider of environmental [
assistance to others outside
my organization.
47.5%
52.5%
answered question
skipped question
38
42
80
Page: Questions for Regulated Community
2. 2. What type of organization d<
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
f> view other (please specify)
D you represent? (Check all that apply.)
Response
Percent
1 3.1%
| 9.4%
| 37.5%
^ 6.3%
6.3%
12.5%
^ 12.5%
Q 6.3%
=| 12.5%
answered question
skipped question
Response
Count
1
3
12
2
2
4
4
2
4
32
48
3. 3. How often do you use P2RIC
First time user
Weekly
Monthly
Every three months
•^> view other (please specifv)
I and/or P2Rx? (Check only one answer.)
Response
Percent
| 21.9%
| | 9.4%
| 21.9%
| 31.3%
1 15.6%
Response
Count
7
3
7
10
5
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (1 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
answered question
skipped question
32
48
4. 4. How did you hear abi
Search
Link from another >
E-mail message or 1
Conference or r
Brochure or ne\
Co
Assistance p
view other (please sp
out P2RIC and/or P2Rx? (Check only one answer.)
Response
Percent
enginel | 9.4%
website| | 6.3%
istserv | 53.1
7o
neeting) | 6.3%
yvsletter 0.0%
lleaguel =| 12.5%
Vendor 0.0%
rogram 0.0%
answered question
skipped question
Response
Count
3
2
17
2
0
4
0
0
4
32
48
5. 5. P2RIC and/or P2Rx helps me to improve my awareness of environmental practices to reduce
pollution. (Check only one answer.)
Response Response
Percent Count
Strongly Agree| | 12.5% 4
Anroo I f\~% 1°A
Neutral I 28.1%
Disagree 0.0%
Strongly Disagree 0.0%
Do Not UseQ 6.3%
answered question
skipped question
17
9
0
0
2
32
48
Page: Questions for Regulated Community (continued)
6. 6. How useful were P2RIC and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
Unaware
S.i Useful S°™ejj;!]at Not useful of-Do not RerSP°"tSe
US6TUI US6TUI LrOUnt
use
Topic Hubs
P2 Programs Directory (i.e.,
Service Provider Directories)
News (i.e., P2/Environmental
News)
Library (i.e., Virtual Library)
Industry Sector Information
Vendor Database
Case Studies
Listservs (i.e., Discussion Groups)
Rapid Response (i.e., Research
Assistance)
7.4% (2)
3.7% (1)
22.2% (6)
11.1% (3)
7.4% (2)
3.7% (1)
7.4% (2)
11.1% (3)
3.7% (1)
33
25
51.
14
11
11
29
51.
14
.3%
.9%
9%
.8%
.1%
.1%
.6%
9%
.8%
(9)
(7)
(14)
(4)
(3)
(3)
(8)
(14)
(4)
18. 5% (5)
18. 5% (5)
14. 8% (4)
14. 8% (4)
14. 8% (4)
7.4% (2)
7.4% (2)
22.2% (6)
3.7% (1)
0
0
3
3
3
0
0
0
0
0%
0%
7%
7%
7%
0%
0%
0%
0%
(0)
(0)
(1)
(1)
(1)
(0)
(0)
(0)
(0)
40.7%
51.9%
7.4%
55.6%
63.0%
77.8%
55.6%
14.8%
77.8%
(11)
(14)
(2)
(15)
(17)
(21)
(15)
(4)
(21)
27
27
27
27
27
27
27
27
27
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (2 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
P2 R(eMeaIuDrIme^tTod)
P2RIC RSS News Feed
R7 Roundtable
P2 Marketing Tools
Calendars
0.0%
7.4%
11.1%
3.7%
11.1%
(0)
(2)
(3)
(1)
(3)
14
29
22
14
22
8
6
2
8
2
%
%
%
%
%
(4)
(8)
(6)
(4)
(6)
3
7
3
7
11
7%
4%
7%
4%
.1%
(1)
(2)
(1)
(2)
(3)
3.7% (1) 77.8% (21)
0.0% (0) 55.6% (15)
3.7% (1) 59.3% (16)
0.0% (0) 74.1% (20)
0.0% (0) 55.6% (15)
answered question
skipped question
27
27
27
27
27
27
53
7. 7. How did you utilize the P2RIC and/or P2Rx website(s)? (Check all that apply.)
Response Response
Percent Count
Reviewed Topic Hub
information (or other P2RIC/P ^ I 63.0% 17
P2Rx website information)
Attended a training, workshop, or
conference promoted by a P2Rx | 1 14.8% 4
Center
Asked for pollution prevention,—, 3 7°/ 1
technical assistance!—I °- /0
Requested additional resources, , 11 -ID/ o
from a P2Rx Center' ' " •'/0 J
Contacted a technical assistance a
providerL-l 6''/0 n
^>vi
view
Other (please specify) i i 148% 4
answered question 27
skipped question 53
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part, due to
information you have found through P2RIC and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a pollution prevention
process or practice (i.e., | | 15.4% 4
implemented pollution prevention)
Identified a pollution prevention, , ~A K0/ a
opportunity^ J J4-b/0 y
Purchased new process rq 3 8°/ 1
equipment to prevent pollution'—'
Implemented energy conservation, —, -ic/io/ A
measures'- 15-4/0 4
Switched to renewable energyl | 7.7% 2
Contacted a vendorl | 7.7% 2
Implemented material or waste, ^ 15 4
-------
Survey Results
No process changes were taken
Other (please specify) r
vew
50.0%
3.8%
answered question
skipped question
13
1
26
54
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of P2RIC and/or P2Rx
use. (Check all that apply.)
Response Response
Reduced non-hazardous waste r-
(solid waste)L
Eliminated non-hazardous waste pq
(solid waste)!—I
Reduced hazardous waste or.—
hazardous materials'—
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollutionl
Eliminated pollutant waste water
Don't Know|
Nonel
\ff view
Other (please specify)
Percent Count
3.8%
7.7%
0.0% 0
0.0% 0
0.0% 0
7.7% 2
0.0% 0
26.9% 7
46.2% 12
7.7%
answered question 26
skipped question 54
10.10. Has using P2RIC and/or P2Rx or the action(s) taken above resulted in cost savings to you?
(Cost includes time as well as dollars.) (Check only one answer.)
Response Response
^| Percent Count
Yes| | 15.4% 4
I 34.6% 9
| 50.0% 13
No
Do not know
If yes, please provide details.
answered question
skipped question
54
26
Page: Questions for Assistance Provider Community
11.2. What type of organization do you represent? (Check all that apply.)
Federal government
State government
Local government
Response
Percent
0.0%
30.8%
17.9%
Response
Count
0
12
7
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (4 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
\|> view other (please specify)
2.6%
H 2.6%
| 23.1%
I 25.6%
0.0%
3 2.6%
answered question
skipped question
1
1
9
10
0
1
39
41
12. 3. How often do you use
First time
We
Mor
Every three mor
s|> view other (please specif
P2RIC and/or P2Rx? (Check only one answer.)
Response
Percent
jser| | 10.3%
ekly| | 10.3%
ithlv 1 9R 9%
iths | 35.9%
y) i— i 15.4%
answered question
skipped question
Response
Count
4
4
11
14
6
39
41
13. 4. How did you hear about P2
Search engine
Link from another website
E-mail message or listserv
Conference or meeting
Brochure or newsletter
Colleague
Vendor
Assistance program
| view other (please specify)
RIC
n
J
n
and/or P2Rx? (Check only one answer.)
Response
Percent
5.1%
7.7%
| 25.6%
| 1 /.9%
0.0%
1 ^n ft%
2.6%
5.1%
5.1%
answered question
skipped question
Response
Count
2
3
10
7
0
12
1
2
2
39
41
14. 5. Access to the P2RIC and/or P2Rx information has improved my ability to provide technical
assistance to my clients. (Check only one answer.)
Strongly Agree
Agree
Neutral
Disagreel |
Strongly Disagree^
Do Not User~|
Response
Percent
5.1%
48.7%
33.3%
5.1%
2.6%
5.1%
answered question
skipped question
Response
Count
2
19
13
2
1
2
39
41
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (5 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
Page: Questions for Assistance Provider Community (continued)
15. 6. How useful were P2RIC and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
u^e7ul useful "I* Not useful of™ ***%™
use
Topic Hubs
P2 Programs Directory (i.e.,
Service Provider Directories)
News (i.e., P2/Environmental
News)
Library (i.e., Virtual Library)
Industry Sector Information
Vendor Database
Case Studies
Listservs (i.e., Discussion Groups)
Rapid Response (i.e., Research
Assistance)
P2 Results Data System
(Measurement Tool)
P2RIC RSS News Feed
R7 Roundtable
P2 Marketing Tools
Calendars
19.2% (5)
7.7% (2)
19.2% (5)
1 1 .5% (3)
1 1 .5% (3)
3.8% (1)
3.8% (1)
30.8% (8)
1 1 .5% (3)
7.7% (2)
1 1 .5% (3)
19.2% (5)
0.0% (0)
1 1 .5% (3)
23.1%
42.3%
53.8%
42.3%
34.6%
15.4%
30.8%
34.6%
23.1%
19.2%
19.2%
34.6%
19.2%
26.9%
(6)
(11)
(14)
(11)
(9)
(4)
(8)
(9)
(6)
(5)
(5)
(9)
(5)
(7)
30
23
19
23
11
26
26
19
19
30
26
23
30
30
.8%
.1%
.2%
.1%
.5%
.9%
.9%
.2%
.2%
.8%
.9%
.1%
.8%
.8%
(8)
(6)
(5)
(6)
(3)
(7)
(7)
(5)
(5)
(8)
(7)
(6)
(8)
(8)
3
3
3
0
3
3
3
3
3
7
3
0
3
7
8%(1)
8%(1)
8%(1)
0% (0)
8%(1)
8%(1)
8%(1)
8%(1)
8%(1)
7% (2)
8%(1)
0% (0)
8%(1)
7% (2)
23.1%
23.1%
3.8%
23.1%
38.5%
50.0%
34.6%
11.5%
42.3%
34.6%
38.5%
23.1%
46.2%
23.1%
(6)
(6)
(1)
(6)
(10)
(13)
(9)
(3)
(11)
(9)
(10)
(6)
(12)
(6)
answered question
skipped question
26
26
26
26
26
26
26
26
26
26
26
26
26
26
26
54
16. 7. What information di
environmental practices t
Provided training, work
and/or conference infor
Provided information on tec
assistance pr
Provided Topic Hub info
Provided vendor info
s|> view other pollutio
prevention technical assists
so, please 5
d you use to improve or influence your clients' understanding of
o reduce pollution? (Check all that apply.)
Response
Percent
cshop, i 1 fi: A0,
;hnical , , 00 co/
n\/inr^r" ^^.\^ /\j
-matinn 1 AK ^o/.
•mation | 11.5%
n
^^^_^^_| °°, 1 "/::
mce. If
specify.
answered question
skipped question
Response
Count
17
10
12
3
6
26
54
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmental practices, in whole
due to information you have found through P2RIC and/or P2Rx? (Check all that apply.)
or in part,
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (6 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
Changed a pollution prevention
process or practice (i.e., , ,
imnlpmpntpH nolhition
prevention)
Identified a pollution , ,
n FA Mention onnorfiinitx/l
Purchased new process . .
equipment to prevent pollution' '
Implemented energy conservation • ,
mocici irocl
Switched to renewable energy
Contacted a vendor! 1
ImDlemented material or waste •
rof*vf*linn Qv^toml '
Changed handling of waste or , ,
Installed pollution control
equipment (e.g., scrubbers, Q
control technigue)
Installed a waste treatment systemQ
s|)view other (please specify^ | 1
Response
Percent
45.8%
45.8%
8.3%
37.5%
0.0%
29.2%
45.8%
37.5%
4.2%
4.2%
20.8%
12.5%
answered question
skipped question
Response
Count
11
11
2
9
0
7
11
9
1
1
5
3
24
56
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of P2RIC
and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
Reduced non-hazardous waste . ^
(solid waste)^ I
Eliminated non-hazardous waste . .
(solid waste)' '
Reduced hazardous waste or
]
45.8°
8.3°
33.3°
Eliminated hazardous waste or
|—i
hazardous materials1—1
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
ve./...
Other (please specify)
4.2%
12.5%
4.2%
29.2%
4.2%
o / .£) /o
8QO/
.O /O
8QO/
.O /O
answered question
skipped question
11
2
8
1
3
1
7
1
9
2
24
56
19.10. Has using P2RIC and/or P2Rx or the action(s) taken above resulted in cost savings to your
client? (Cost includes time as well as dollars.) (Check only one answer.)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (7 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
Response Response
Percent Count
Yes| | 29.2% 7
Mop 4.2% 1
Do not know) | 66.7% 16
If yes, please provide details. 0
answered question 24
skipped question 56
20.11. Has using P2RIC and/or P2Rx saved you time or money in serving your client(s)? (Check only
one answer.)
Response Response
Percent Count
Yesl— 1 45.8% 11
] 8.3% 2
| 45.8% 11
If yes, please provide details. 0
answered question 24
skipped question 56
No
Do not know|
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9du4H4xH9Vflo5_2fldrSSk2Y_3d (8 of 8) [4/14/2008 4:45:18 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use Peaks to Prairies
and/or P2Rx to find
environmental assistance [
information to apply to my own
organization or facility.
I primarily use Peaks to Prairies
and/or P2Rx in my capacity as a
provider of environmental [
assistance to others outside my
organization.
Response Response
Percent Count
66.7%
33.3%
answered question
skipped question
32
16
48
Page: Questions for Regulated Community
2. 2. What type of organization d<
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
f> view other (please specify)
D you represent? (Check all that apply.)
Response
Percent
| 19.2%
| 26.9%
"3 3.8%
0.0%
| 19.2%
7.7%
q 11.5%
^\ 3.8%
| 1 34.6%
answered question
skipped question
Response
Count
5
7
1
0
5
2
3
1
9
26
22
3. 3. How often do you use Peaks
First time user
Weekly
Monthly
Every three months
» to Prairies and/or P2Rx? (Check only one answer.)
Respon
Percen
I 26
0
I 19
| 30
se Response
t Count
.9% 7
.0% 0
.2% 5
.8% 8
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (1 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
\f) view
uiner ipiease specuy )
\r r 11 | —|
23.1%
answered question
skipped question
6
26
22
4. 4. How did you hear abo
Search (
Link from another w
E-mail message or Ik
Conference or m
Brochure or new
Coll
\
Assistance pr
•-.f) view other (please spe
ut Peaks to Prairies and/or P2Rx? (Check only one answer.)
Response
Percent
snginel | 11.5%
ebsitel | 15.4%
stserv |
eeting) |
sletter
eaguej |
^endor
ogram
ciiy) |
26.9%
7.7%
0.0%
19.2%
0.0%
0.0%
19.2%
answered question
skipped question
Response
Count
3
4
7
2
0
5
0
0
5
26
22
5. 5. Peaks to Prairies and/or P2Rx helps me to improve my awareness of environmental practices to
reduce pollution. (Check only one answer.)
Response Response
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use[
Percent Count
26.9% 7
38.5% 10
15.4% 4
0.0% 0
0.0% 0
19.2% 5
answered question 26
skipped question 22
Page: Questions for Regulated Community (continued)
6. 6. How useful were Peaks to Prairies and/or P2Rx resources in meeting your needs? (Please make
one selection for each resource listed.)
u^e7ul
useful
Not useful ofnot
Topic Hubs 25.0% (5) 30.0% (6) 15.0% (3) 0.0% (0) 30.0% (6)
P2 Programs Directory 15.0% (3) 45.0% (9) 10.0% (2) 0.0% (0) 30.0% (6)
25'0% 5 20 0% 4 20 0% 4 °-°* ° 3
Library (i.e., Virtual Card
Catalogue, P2 Bookmarks, or 15.0% (3) 15.0% (3) 25.0% (5) 0.0% (0) 45.0% (9)
other P2Rx online Libraries)
Industry Sector Information (i.e.,
Featured Resources, Green 30.0% (6) 25.0% (5) 25.0% (5) 0.0% (0) 20.0% (4)
Parks)
20
20
20
20
20
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (2 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
Vendor Database
Case Studies (including Homes-
Across-America profiles)
Listservs
Rapid Response (i.e., Help Desk)
P2 Results Data System
(Measurement Tool)
Tribal Pollution Prevention
Resources
15.0%
45.0%
20.0%
15.0%
5.0%
0.0%
(3)
(9)
(4)
(3)
(1)
(0)
15
10
20
10
30
0% (3)
0% (2)
0% (4)
0% (2)
0% (6)
5.0% (1)
25
15
20
25
25
15
0%
0%
0%
0%
0%
0%
(5)
(3)
(4)
(5)
(5)
(3)
5
0
0
0
0
5
0%(1)
0% (0)
0% (0)
0% (0)
0% (0)
0%(1)
40.0% (8)
30.0% (6)
40.0% (8)
50.0% (10)
40.0% (8)
75.0% (15)
answered question
skipped question
20
20
20
20
20
20
20
28
7. 7. How did you utilize the Peaks to
Reviewed Topic Hub
information (or other Peaks to ,
information)
Attended a training, workshop, or
conference promoted by a P2Rx |
Center
Ac-ifpH for nnllution nrpvpntion •
from n P9Rv Ponton
Contacted a technical assistance .
provider'
^ uiner ^piease speciTyj
i
Prairies and/or P2Rx website(s)? (Check all that apply
Response
Percent
3 15.0%
I 20.0%
3 15.0%
10.0%
1 25.0%
answered question
skipped question
Response
Count
14
3
4
3
2
5
20
28
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part, due to
information you have found through Peaks to Prairies and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a pollution prevention
process or practice (i.e., [
implemented pollution prevention)
Identified a pollution
prevention opportunity
Purchased new process r-
equipment to prevent pollution'-
Implemented energy conservation r
Switched to renewable energy
Contacted a vendor
Implemented material or waste ,
recycling system'
Changed handling of waste or,
emission'
) _ |
]
31.6%
57.9%
10.5%
42.1%
5.3%
21.1%
42.1%
10.5%
11
2
8
1
4
8
2
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (3 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
Installed pollution control
equipment (e.g., scrubbers, | |
control technique)
Installed a waste treatment systeml |
No process changes were takenQ
Other (please specify) i=
view
5.3%
5.3%
21.1%
10.5%
answered question
skipped question
29
1
4
2
19
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of Peaks to
and/or P2Rx use. (Check all that apply.)
Response
Percent
Reduced non-hazardous waste . . ,6 „„,
Eliminated non-hazardous waste , , in R0/
(solid waste)1— =1 IU-0/0
Reduced hazardous waste or • -. 0 , rn/
hiTirHnMc- mitoriilc- ol.D/o
Eliminated hazardous waste or 0 00/
hazardous materials
Reduced fugitive air emissions! 1 15.8%
Eliminated air emissions 0.0%
Reduced waste water pollution) | 1 5.8%
Eliminated pollutant waste water ^| 5.3%
Don't Know| | 15.8%
\|J view other (please specify) i 1 -i ^ On/
answered question
skipped question
Prairies
Response
Count
7
2
6
0
3
0
3
1
3
4
3
19
29
10
to
. 10. Has using Peaks to Prairii
you? (Cost includes time as w
Yes
No
Do not know
If yes,
35 and/or P2Rx or the
ell as dollars.) (Check
I
I
please provide details.
action(s) taken above resulted in cost savings
only one answer.)
Response Response
Percent Count
36.8% 7
15.8% 3
47.4% 9
'•^) view
answered question
skipped question
4
19
29
Page: Questions for Assistance Provider Community
11.2. What type of organization do you represent? (Check all that apply.)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (4 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
•J> view other (please specify}
1
Eb
I
i
1
Response
Percent
14.3%
42.9%
0.0%
7.1%
14.3%
28.6%
21.4%
0.0%
14.3%
answered question
skipped question
Response
Count
2
6
0
1
2
4
3
0
2
14
34
12. 3. How often do you use Peal
First time user
Weekly
Monthly
Every three months
s|> view other (p^ase specify)
cs to Prairies and/or P2Rx? (Check only one answer.)
Response
Percent
21.4%
I 14.3%
21.4%
21.4%
21.4%
answered question
skipped question
Response
Count
3
2
3
3
3
14
34
13. 4. How did you hear about Peaks to Prairies and/or P2Rx? (Check only one answer.)
Response
Percent
Search engine 0.0%
Link from another website 0.0%
E-mail message or listservl | 35.7%
Conference or meeting 0.0%
Brochure or newsletter 0.0%
Colleague! | 42.9%
Vendor 0.0%
Assistance program 0.0%
• vew
Other (please specify)
21.4%
answered question
skipped question
Response
Count
34
14
14. 5. Access to the Peaks to Prairies and/or P2Rx information has improved my ability to provide
technical assistance to my clients. (Check only one answer.)
Response Response
Percent Count
Strongly AgreeQ 7.1 %
| 50.0%
I 35.7%
Agree
Neutral
Disagree
0.0%
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (5 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
Strongly Disagree
Do Not Use| |
0.0%
7.1%
answered question
skipped question
0
1
14
34
Page: Questions for Assistance Provider Community (continued)
15. 6. How useful were Peaks to Prairies and/or P2Rx resources in meeting your needs? (Please make
one selection for each resource listed.)
II rt o \A/3 FG
\/6 rv ^5^i IT1 G Wll 3 1 Fx6SD^)dS6
Topic Hubs
P2 Programs Directory
News (i.e., National P2 News or
Region 8 News and Events)
Library (i.e., Virtual Card
Catalogue, P2 Bookmarks, or
other P2Rx online Libraries)
Industry Sector Information (i.e.,
Featured Resources, Green
Parks)
Vendor Database
Case Studies (including Homes-
Across-America profiles)
Listservs
Rapid Response (i.e., Help Desk)
P2 Results Data System
(Measurement Tool)
Tribal Pollution Prevention
Resources
23.1% (3)
15.4% (2)
23.1% (3)
15.4% (2)
7.7% (1)
7.7% (1)
7.7% (1)
23.1% (3)
0.0% (0)
15.4% (2)
15.4% (2)
30
15
30
15
23
15
38
15
30
23
7.
.8%
.4%
.8%
.4%
.1%
.4%
.5%
.4%
.8%
.1%
7%
(4)
(2)
(4)
(2)
(3)
(2)
(5)
(2)
(4)
(3)
(1)
7.7%
23.1%
23.1%
15.4%
23.1%
15.4%
15.4%
30.8%
15.4%
15.4%
23.1%
(1)
(3)
(3)
(2)
(3)
(2)
(2)
(4)
(2)
(2)
(3)
0
0
0
0% (0)
0% (0)
0% (0)
15.4% (2)
0
0
0
0
0
7
0
0% (0)
0% (0)
0% (0)
0% (0)
0% (0)
7%(1)
0% (0)
38.5%
46.2%
23.1%
38.5%
46.2%
61.5%
38.5%
30.8%
53.8%
38.5%
53.8%
(5)
(6)
(3)
(5)
(6)
(8)
(5)
(4)
(7)
(5)
(7)
answered question
skipped question
13
13
13
13
13
13
13
13
13
13
13
13
35
16. 7. What information did you use to improve or influence your clients' understanding of
environmental practices to reduce pollution? (Check all that apply.)
Response
Percent
Provided training, workshop,
and/or conference information
Provided information on technical,
assistance providers^
Provided Topic Hub information
Provided vendor information
ve./...
Other pollution
[
prevention technical assistance. If
so, please specify.
3
61.51
38.5°
38.5°
15.41
J.81
answered question
skipped question
Response
Count
35
8
13
Page: Questions for Assistance Provider Community (continued)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (6 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
17. 8. What action(s) have your clients taken to improve environmental practices, in whole or in part,
due to information you have found through Peaks to Prairies and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a process or practicef ^
Identified a pollution ,—
prevention opportunity
Purchased new process f
equipment to prevent pollution^
Implemented energy conservation f
measures^
Switched to renewable energy
Contacted a vendor
Implemented material or waste
recycling system
Changed handling of waste or
emission
Installed pollution control
equipment (e.g., scrubbers,
I
]
50.0%
50.0%
8.3%
25.0%
8.3%
25.0%
25.0%
25.0%
control technique)
Installed a waste treatment system
No process changes were taken[
view
Other (please specify)
0.0%
0.0%
33.3%
16.7%
answered question
skipped question
0
0
4
12
36
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of Peaks to
Prairies and/or P2Rx use. (Check all that apply.)
Response Response
Percent Count
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste r
(solid waste)L
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or r
hazardous materialsL
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
view
Other (please specify)
50.0%
8.3%
50.0%
16.7%
8.3%
16.7%
25.0%
8.3%
33.3%
8.3%
8.3%
answered question
skipped question
1
2
3
1
4
1
12
36
19.10. Has using Peaks to Prairies and/or P2Rx or the action(s) taken above resulted in cost savings
to your client? (Cost includes time as well as dollars.) (Check only one answer.)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (7 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
Yes| I
No
Do not know)
If yes, please provide details.
Response
Percent
25.0%
0.0%
| 75.0%
*\§) view
answered question
skipped question
Response
Count
3
0
9
2
12
36
20.11. Has using Peaks to Prairies and/or P2Rx saved you time or money in serving your client(s)?
(Check only one answer.)
Response Response
Percent Count
Yes| | 41.7%
No| | 8.3%
Do not know! | 50.0%
If yes, please provide details.
answered question
skipped question
12
36
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9RRaNZpxsAqHlOP8gxAYo7Q_3d (8 of 8) [4/14/2008 4:45:49 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use WRPPN and/or
P2Rx to find environmental r-
assistance information to apply to <-
my own organization or facility.
I primarily use WRPPN and/or
P2Rx in my capacity as a
provider of environmental [
assistance to others outside
my organization.
Response Response
Percent Count
35.1%
64.9%
answered question
skipped question
27
50
77
Page: Questions for Regulated Community
2. 2. What type of organization do you represent? (Check all that apply.)
Response
Percent
Federal government) | 19.2%
State government! I 1 1 -5%
Local government | 30.8%
Manufacturing | 15.4%
Service industry 0.0%
Educational institution 0.0%
Nonprofit organization^ 3.8%
Wastewater treatment industry^ 3.8%
•~*r VIRW f^thar /rila^ca criQr*if\/\
^ uiner (please speciTy ) -i c >io/
answered question
skipped question
Response
Count
5
3
8
4
0
0
1
1
4
26
51
3. 3. How often do you use WRPF
First time user
Weekly
Monthly
Every three months
•^> view other (please specifv)
3N and/or P2Rx? (Check only one answer.)
Response
Percent
-^^M-H 26.9%
| 23.1%
| | 15.4%
I 15.4%
I 19.2%
Response
Count
7
6
4
4
5
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (1 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
answered question
skipped question
26
51
4. 4. How did you hear abi
Search
Link from another >
E-mail message or
Conference or n
Brochure or ne\
Co
Assistance p
view other (please sp
out WRPPN and/or P2Rx? (Check only one answer.)
Response
Percent
enginel | 11.5%
website 0.0%
listservl | 7.7%
leeting ] 34.6
yvsletter^| 3.8
lleague | 30.b
%
%
%
Vendor 0.0%
rogramQ 3.8%
ecify) |=| 77o/0
answered question
skipped question
Response
Count
3
0
2
9
1
8
0
1
2
26
51
5. 5. WRPPN and/or P2Rx helps me to improve my awareness of environmental practices to r
pollution. (Check only one answer.)
Response R
Percent
strongly Agree ] 46.2%
Agree I 30. 8%
Neutral | 19.2%
Disagree 0.0%
Strongly Disagree 0.0%
Do Not Usen 3.8%
answered question
skipped question
educe
esponse
Count
12
8
5
0
0
1
26
51
Page: Questions for Regulated Community (continued)
6. 6. How useful were WRPPN and/or P2Rx
selection for each resource listed.)
Very
useful
resources in meeting your needs?
Useful S~ at Not useful
(Please make one
sss "«»-
use
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
20.0%
15.0%
20.0%
30.0%
20.0%
0.0%
10.0%
25.0%
5.0%
0.0%
(4)
(3)
(4)
(6)
(4)
(0)
(2)
(5)
(1)
(0)
40.0%
45.0%
40.0%
20.0%
55.0%
40.0%
40.0%
40.0%
35.0%
25.0%
(8)
(9)
(8)
(4)
(11)
(8)
(8)
(8)
(7)
(5)
10
15
10
10
0.
20
10
10
20
15
.0%
.0%
.0%
.0%
0%
.0%
.0%
.0%
.0%
.0%
(2)
(3)
(2)
(2)
(0)
(4)
(2)
(2)
(4)
(3)
0.0% (0)
5.0% (1)
0.0% (0)
15.0% (3)
10.0% (2)
10.0% (2)
5.0% (1)
0.0% (0)
5.0% (1)
10.0% (2)
30.0%
20.0%
30.0%
25.0%
15.0%
30.0%
35.0%
25.0%
35.0%
50.0%
(6)
(4)
(6)
(5)
(3)
(6)
(7)
(5)
(7)
(10)
answered question
20
20
20
20
20
20
20
20
20
20
20
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (2 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
skipped question
57
7. 7. How did you utilize the WRPPN and/or P2Rx website(s)? (Check all that apply.)
Response Response
Reviewed Topic Hub information
(or other P2Rx website [
information)
Attended a training, workshop,
or conference promoted by a [
P2Rx Center
Asked for pollution prevention r-
technical assistanceL
Requested additional resources
from a P2Rx Center
Contacted a technical assistance r
providen-
Other (please specify)
X vew
Percent Count
50.0% 10
80.0% 16
25.0% 5
0.0% 0
20.0% 4
10.0% 2
answered question 20
skipped question 57
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part, due to
information you have found through WRPPN and/or P2Rx? (Check all that apply.)
Response
Changed a pollution prevention
process or practice (i.e., [
implemented pollution prevention)
Identified a pollution
prevention opportunity'
Purchased new process
equipment to prevent pollution1—'
Implemented energy conservation r
measures'-
Switched to renewable energy
Contacted a vendor[
Implemented material or waste r-
recycling system'-
Changed handling of waste or r
emission'-
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken[
^f view
Other (please specify)
Percent
Response
Count
47.4%
78.9%
5.3%
15.8%
0.0%
15.8%
5.3%
31.6%
0.0%
0.0%
15.8%
10.5%
answered question
skipped question
15
0
3
0
0
3
19
58
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (3 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of WRPPN and/or P2Rx
use. (Check all that apply.)
Response Response
Percent Count
=| 36.8% 7
10.5% 2
| 42.1% 8
10.5% 2
10.5% 2
5.3% 1
10.5% 2
5.3% 1
10.5% 2
21.1% 4
15.8% 3
answered question 19
skipped question 58
Reduced non-hazardous waste r-
(solid waste)L
Eliminated non-hazardous waste r
(solid waste)L
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
\ vew
Other (please specify)
10. 10
(Cost
. Has using WRPPN and/or
includes time as well as do
Yes
No
Do not know
If yes,
P2Rx or the action(s) taken above
Mars.) (Check only one answer.)
I
I
I
1
please provide details.
resulted in cost savings to you?
Response Response
Percent Count
52.6% 10
15.8% 3
31.6% 6
^ view
answered question
skipped question
4
19
58
Page: Questions for Assistance Provider Community
11.2. What type of organization <
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
Jo you represent? (Check all that apply.)
Response
Percent
Q 6.4%
29.8%
^ 4.3%
4.3%
"3 6.4%
q 12.8%
I 8.5%
Response
Count
3
14
14
2
2
3
6
4
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (4 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
jview other (please specify) ^^ R ,,„/
answered question
skipped question
3
47
30
12. 3. How often do you use WRPPN and/or P2Rx? (Check only one answer.)
•. vew
First time user
Weekly
Monthly
Every three months
Other (please specify)
Response Response
Percent Count
10.6%
25.5%
17.0%
25.5%
21.3%
answered question
skipped question
30
5
12
8
12
10
47
13. 4. How did you hear about WRPPN and/or P2Rx? (Check only one answer.)
Response
Percent
Search enginel | 10.6%
Link from another website ^ 4.3%
E-mail message or listserv| | 1 9. 1 %
Conference or meeting I 23.4%
Brochure or newsletter 0.0%
f~*rtlloani 10! —I "?1 Q°/>
Vendor 0.0%
Assistance program^ 4.3%
.^view other (please specify) i — i R Aof
answered question
skipped question
Response
Count
5
2
9
11
0
15
0
2
3
47
30
14. 5. Access
assistance to
to the WRPPN and/
my clients. (Check
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Do Not Use
or P2Rx information has improved my ability to provide t<
only one answer.)
Response
Percent
I OH 0»/.
| 46.8%
•^ | 12.8%
2.1%
2.1%
H 4.3%
answered question
skipped question
;chnical
Response
Count
15
22
6
1
1
2
47
30
Page: Questions for Assistance Provider Community (continued)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (5 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
15. 6. How useful were WRPPN and/or P2Rx resources
selection for each resource listed.)
usee7ul Useful
Topic Hubs42.5% (17) 22.5% (9)
P2 Programs Directory 20.0% (8) 40.0% (16)
News 25.0% (10) 40.0% (16)
Library 20.0% (8) 22.5% (9)
Industry Sector Information 32.5% (13) 20.0% (8)
Vendor Database 7.5% (3) 20.0% (8)
Case Studies 25.0% (10) 20.0% (8)
Listservs32.5% (13) 40.0% (16)
Rapid Response 15.0% (6) 12.5% (5)
P2 Results Data System 7 5% (3) 125%(5)
(Measurement Tool) v ' v '
in meeting your needs? (Please make
II rt o \A/3 FG
^5^i IT1 G Wll 3 1 i^
10.0% (4)
10.0% (4)
20.0% (8)
15.0% (6)
15.0% (6)
17. 5% (7)
25.0% (10)
10.0% (4)
17. 5% (7)
25.0% (10)
use
5.0% (2) 20.0% (8)
5.0% (2) 25.0% (10)
0.0% (0) 15.0% (6)
0.0% (0) 42.5% (17)
5.0% (2) 27. 5% (11)
7.5% (3) 47.5% (19)
2.5% (1) 27.5% (11)
0.0% (0) 17. 5% (7)
2.5% (1) 52.5% (21)
10.0% (4) 45.0% (18)
answered question
skipped question
one
ocnonQp
^OUvl IOC
40
40
40
40
40
40
40
40
40
40
40
37
16. 7. What information did you use to improve or influence your clients' understanding of
environmental practices to reduce pollution? (Check all that apply.)
Response
Percent
Provided training, workshop, , , 67 ,.„/
anrl/or ponffrfnpf* information oi.o/o
Providpd information on tprhniral • •
Provided Topic Hub information I 32.5%
PrnwiHoH wonrlnr infnrmatinn O^\ OPA
-.f> view other pollution
| 22.5%
prevention technical assistance. If
so, please specify.
answered question
skipped question
Response
Count
27
21
13
10
9
40
37
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmental practices, in whole or in part,
due to information you have found through WRPPN and/or P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a pollution prevention
process or practice (i.e., | ^ | 68.4% 26
implemented pollution prevention)
Identified a pollution, , 78 90/ on
prevention opportunity* ~^^^^^^^^^^^^^^^^^^^ "
Purchased new process. . ~R o0/ 14
equipment to prevent pollution* I oo.o/o
Implemented energy conservation, , Ofi Q0/ ..
measures'— ^ ^°-y/0 "
Switched to renewable energy^ 0.0% 0
Contacted a vendorF I 36.8% 14
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (6 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
Implemented material or waste . — . 0~ on.
rar»\/r»|inn cwctom ^w.o /o
Phnnnprl hnnrllinn of wn^tp or .
rmi— inn ' oD.O/o
Installed pollution control
equipment (e.g., scrubbers, | 15.8%
control technique)
Installed a waste treatment systeml | 1 3.2%
No process changes were takenQ 2.6%
.>) view Othor /^nlonco cnorifiA • •
^_ ^HllGl (Ulcdoc oUcUIIV) 4 Q AO/
answered question
skipped question
10
14
6
5
1
7
38
39
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result
and/or P2Rx use. (Check all that apply.)
Response
Percent
Reduced non-hazardous waste • , co ™/
^nliH wT?tr'\ ' Oz.D/o
Eliminated non-hazardous waste • — , 00 7fy
^nliri wn^t^ 1 £.3.1/0
Reduced hazardous waste or .
hn7nrrinn^ matrriaK' ' O/.»/o
Fliminntprl hn^nrrlnii^ vA/n^tp or .
ho-rarrlniic matorialcl ^.w.o/o
Reduced fugitive air emissions I 21.1%
Eliminated air emissions | 10.5%
Eliminated pollutant waste water) | 1 8.4%
Don't Know| | 21.1%
None^| 5.3%
•-=) view other (please specify) i 1 ^nKo/-
answered question
skipped question
of WRPPN
Response
Count
20
9
22
10
8
4
15
7
8
2
4
38
39
19. 10. Has using WRPPN and/or P2Rx or the action(s) taken above resulted in cost savini
client? (Cost includes time as well as dollars.) (Check only one answer.)
Response
Percent
Yes | 34.2%
No | 10.5%
If yes, please provide details. ^ V]*w
answered question
skipped question
gs to your
Response
Count
13
4
21
9
38
39
20. 11. Has using WRPPN and/or P2Rx saved you time or money in serving your client(s)? (Check
only one answer.)
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (7 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
Response
Percent
Yes | 63.2%
No | 10.5%
Do not know| I 26.3%
If yes, please provide details. ^ V]e'w
answered question
skipped question
Response
Count
24
4
10
19
38
39
http://www.surveymonkey.com/sr.aspx?sm=Nsd_...AHCjEzEy7kHYQ9WHFZfObYgabW_2bJV_2fXApPn8_3d (8 of 8) [4/14/2008 4:46:25 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
primarily use PPRC and/or P2Rx
to find environmental assistance
information to apply to my own
organization or facility.
I primarily use PPRC and/or
P2Rx in my capacity as a
Response Response
Percent Count
provider of environmental [
assistance to others outside
my organization.
37.7%
62.3%
answered question
skipped question
26
43
69
Page: Questions for Regulated Community
2. 2. What type of organization d<
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
f> view other (please specify)
D you represent? (Check all that apply.)
Response
Percent
| 8.3%
I 8.3%
| 33.3%
| 16.7%
I 12.5%
| 8.3%
I 8.3%
^ 4.2%
=| 12.5%
answered question
skipped question
Response
Count
2
2
8
4
3
2
2
1
3
24
45
3. 3. How often do you use PPRC and/or P2Rx? (Check only one answer.)
First time user
Weekly
Monthly
Every three months
Other (please specify)
Response
Percent
20.8%
4.2%
25.0%
50.0%
0.0%
answered question
skipped question
Response
Count
5
1
6
12
0
24
45
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (1 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
4. 4. How did you hear about
Search er
Link from another we
E-mail message or lis
Conference or me
Brochure or newsl
Collec
Ve
Assistance proc
\f> view other (please sped
t PPRC and/or P2Rx? (Check only one answer.)
Respon
Percen
gine 1 25
bsiteQ 4
tservQ 4
sting | 25
etter 0
igue| | 33
se
t
.0%
.2%
.2%
.0%
.0%
.3%
ndor 0.0%
}ramQ 4.2%
fy) n 4-2%
answered question
skipped question
Response
Count
6
1
1
6
0
8
0
1
1
24
45
5. 5. PPRC and/or P2Rx helps me to improve my awareness of environmental practices to rec
pollution. (Check only one answer.)
Response R
Percent
blrongly Agree '^^^^^•l 29.2%
Agree | 62.5%
Neutral) | 8.3%
Disagree 0.0%
Strongly Disagree 0.0%
Do Not Use 0.0%
answered question
skipped question
luce
esponse
Count
7
15
2
0
0
0
24
45
Page: Questions for Regulated Community (continued)
6. 6. How useful were PPRC and/or P2Rx resources in meeting your needs? (Please make one
selection for each resource listed.)
Unaware
Verv Useful Somewhat N t usefui Of.nn not Response
useful useiui useful useiui OT uo noi Count
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
22.2
16.7
44.4
11.1
22.2
11.1
22.2
11.1
11.1
11.1
%
%
%
%
%
%
%
%
%
%
(4)
(3)
(8)
(2)
(4)
(2)
(4)
(2)
(2)
(2)
61.
38
38
27
38
11
27
38
11
0.
1%
.9%
.9%
.8%
.9%
.1%
.8%
.9%
.1%
0%
(11)
(7)
(7)
(5)
(7)
(2)
(5)
(7)
(2)
(0)
5.6% (1)
5£* Q / / "1 \
. O / 0 1 I )
11.1% (2)
•\ -\ -\o/ /o\
I I . I /O \£.)
22.2% (4)
11.1% (2)
•\ -\ -\o/ /o\
I I . I /O \£.)
•\ -\ -\o/ /o\
I I . I /O \£.)
5.6% (1)
5C* O/ / "1 \
. O / 0 I 1 )
0
11
0
5
0
5
0
0
0
0
0% (0)
. 1 % (2)
0% (0)
6% (1)
0% (0)
6% (1)
0% (0)
0% (0)
0% (0)
0% (0)
use
11.1% (2)
27.8% (5)
5.6% (1)
44.4% (8)
16.7% (3)
61.1% (11)
38.9% (7)
38.9% (7)
72.2% (13)
83.3% (15)
answered question
skipped question
18
18
18
18
18
18
18
18
18
18
18
51
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (2 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
7. 7. How did you utilize the PPRC and/or P2Rx website(s)?
Reviewed Topic Hub
website information)
Attended a training, workshop, or
Center
Asked for pollution prevention ,-q
technical assistance' — '
Requested additional resources r^
from a P2Rx Center-^
Contacted a technical assistance , — ,
provider-^
V view other (please specify) •— i
(Check all that apply.)
Response
Percent
33.3%
5.6%
5.6%
5.6%
5.6%
answered question
skipped question
Response
Count
16
6
1
1
1
1
18
51
Page: Questions for Regulated Community (continued)
8. 8. What action(s) have you taken to improve environmental practices, in whole or in part,
information you have found through PPRC and/or P2Rx? (Check all that apply.)
Response
Percent
Changed a pollution prevention
implerru
equ
Impleme
Swil
Imple
Cha
ec
Installed
No prc
\f) view
process or practice (i.e.,
anted pollution prevention)
Identified a pollution r
nrpwpntinn nnnnrtiinitv
Purchased new process
pment to prevent pollution
nted energy conservation •-
m^^Qi ir^Q •
ched to renewable energy
Contacted a vendor
mented material or waste r
recycling system^-
"inprl hanrllinn nf wa^tp nr -
pmiccinn .
Installed pollution control
ppment (e.g., scrubbers,
control technique)
a waste treatment system
Dcess changes were taken :
Other (please specify) r
1 37.5%
| 68.8%
0.0%
| 18.8%
^ 6.3%
| 31.3%
=| 12.5%
1 0-1 OO/
0.0%
3 6.3%
~ 6.3%
answered question
skipped question
due to
Response
Count
6
11
0
3
1
5
2
5
0
1
1
3
16
53
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (3 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of PPRC and/or P2Rx
use. (Check all that apply.)
Reduced non-hazardous waste
(solid waste)
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or
hazardous materials
Eliminated hazardous waste or r
hazardous materialsL
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution[
Eliminated pollutant waste water
Don't Know
None
view
Other (please specify)
Response
Percent
37.5%
0.0%
37.5%
6.3%
0.0%
0.0%
18.8%
0.0%
12.5%
18.8%
12.5%
answered question
skipped question
Response
Count
0
16
53
10.10. Has using PPRC and/or P2Rx or the action(s) taken above resulted in cost savings to you?
(Cost includes time as well as dollars.) (Check only one answer.)
Response Response
^| Percent Count
Yesj = | 31.3% 5
I 25.0% 4
1 43.8% 7
If yes, please provide details. 0
answered question 16
skipped question 53
No
Do not know
Page: Questions for Assistance Provider Community
11.2. What type of organization <
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution
Nonprofit organization
Wastewater treatment industry
^ view other (please specify)
Jo you represent? (Check all that apply.)
Response
Percent
I 12.8%
I 46 2%
| 12.8%
0.0%
| 7.7%
| 10.3%
2.6%
H 2.6%
=| 12.8%
answered question
skipped question
Response
Count
5
18
5
0
3
4
1
1
5
39
30
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (4 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
12. 3. How often do you use PPR
First time user
Weekly
Monthly
Every three months
C and/or P2Rx? (Check only one answer.)
Response
Percent
^| 5.1%
| 10.3%
| 2b.2%
| 33.3%
J view other (please specify) i 1 ~o ->«/
answered question
skipped question
Response
Count
2
4
11
13
9
39
30
13. 4. How did you hear about PF
Search engine
Link from another website
E-mail message or listserv
Conference or meeting
Brochure or newsletter
Colleague
Vendor
Assistance program
> view other (please specify)
>RC
3
n
and/or P2Rx? (Check only one answer.)
Response
Percent
2.6%
:\D. \ /o
10.3%
3 12.8%
2.6%
I d1 0%
0.0%
5.1%
answered question
skipped question
Response
Count
1
2
4
5
1
16
0
2
8
39
30
14. 5. Access to the PPRC and/or P2Rx information has improved my ability to provide technical
assistance
to my clients. (Check only one answer.)
Strongly Agree |
Neutral |
Disagree
Strongly DisagreefJ
Do Not UseQ
Response
Percent
25.6%
46.2%
20.5%
0.0%
2.6%
5.1%
answered question
skipped question
Response
Count
10
18
8
0
1
2
39
30
Page: Questions for Assistance Provider Community (continued)
15. 6. How
useful were PPRC and/or P2Rx resources in meeting your
needs? (Please make
one
selection for each resource listed.)
Topic Hubs34.4% (11) 34.4% (11) 9.4% (3)
0.0% (0) 21. 9% (7)
32
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (5 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
P2 Programs Directory 3.1% (1) 37.5% (12) 28.1% (9) 0.0% (0) 31. 3% (10)
News 40.6% (13) 31 .3% (10) 18. 8% (6) 0.0% (0) 9.4% (3)
Library 9.4% (3) 15.6% (5) 25.0% (8) 9.4% (3) 40.6% (13)
Industry Sector Information 25.0% (8) 25.0% (8) 18. 8% (6) 0.0% (0) 31.3% (10)
Vendor Database 0.0% (0) 12.5% (4) 34.4% (11) 6.3% (2) 46.9% (15)
Case Studies 28.1% (9) 21. 9% (7) 21. 9% (7) 0.0% (0) 28.1% (9)
Listservs 12. 5% (4) 18.8% (6) 9.4% (3) 6.3% (2) 53.1% (17)
Rapid Response 12. 5% (4) 6.3% (2) 9.4% (3) 6.3% (2) 65.6% (21)
P2 ^eaL^ntToS) 6'3% & 15.6% (5) 21 .9% (7) 9.4% (3) 46.9% (15)
answered question
skipped question
32
32
32
32
32
32
32
32
32
32
37
16. 7. What information did you use to improve or influence your clients' understanding of
environmental practices to reduce pollution? (Check all that apply.)
Response
Percent
Prnx/iHpH tmininn wnrk^hnn ?inH/.
Provided information on .
tophnipal a^^i^tanpo nrnvirlor^ uu.u/o
Provided Topic Hub information| | 31.3%
Provided vendor information^ 3. 1 %
s|> view other pollution
| -^^— | 25.0%
prevention technical assistance. If
so, please specify.
answered question
skipped question
Response
Count
13
16
10
1
8
32
37
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmental practices, in whole
due to information you have found through PPRC and/or P2Rx? (Check all that apply.)
Response
Percent
Changed a pollution prevention
process or practice (i.e., | I 32.1%
implemented pollution prevention)
Identified a pollution . — , 5Q Q0/
nrpvpntinn onnortnnitw ou.u/o
Purrha^pd npw nrorpc5ci • i
Qni linm^nt tn nrox/ont nnlh itinrv * it.o/o
Imnlpmpntpd pnprnv ron^prvation • •
m^^Qi irr>*"J * T-.O /o
Switched to renewable energy 0.0%
Contacted a vendor) | 25.0%
Implemented material or waste • 1 9/- no/
rorvrlinn c-wctom ^O.U/o
PhfinnpH hfinHlinn nf vA/Pi^tp nr .
pmjccjnn '
or in part,
Response
Count
9
14
4
4
0
7
7
5
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (6 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system[
No process changes were taken[
vew
Other (please specify)
0.0%
3.6%
3.6%
28.6%
answered question
skipped question
41
0
1
1
8
28
18. 9. Please identify whether your client(s) reduced or eliminated pollutant(s) as a result of PPRC and/
or P2Rx use. (Check all that apply.)
Response Response
Count
Reduced non-hazardous waste .
(solid waste)*
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or
hazardous materials^
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water
Don't Know
None
• vew
Other (please specify)
Percent
17.9%
0.0%
25.0%
10.7%
10.7%
0.0%
10.7%
3.6%
60.7%
0.0%
7.1%
answered question
skipped question
0
7
3
0
3
1
17
0
28
41
19.10.
client?
Has using PPRC and/or P2Rx or the action(s) taken
(Cost includes time as well as dollars.) (Check only
No
Do not know)
If yes, please
1
provide details.
above resulted in cost savings
one answer.)
Response
Percent
25.0%
0.0%
| 75.0%
^ view
answered question
skipped question
to your
Response
Count
7
0
21
3
28
41
20.11. Has using PPRC and/or P2Rx saved you time or money in serving your client(s)? (Check only
one answer.)
Response
Percent
Yes[
NoQ
67.9%
3.6%
Response
Count
19
1
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (7 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
Do not know| I
If yes, please provide details.
28.6%
'^) view
answered question
skipped question
8
28
41
http://www.surveymonkey.com/sr.aspx?sm=Nsd_2frxsAHCjEzEy7kHYQ9QUCr8qQUG4KQtPHSjST9gM_3d (8 of 8) [4/14/2008 4:46:52 PM]
-------
Survey Results
View Summary
Filter Responses Download Responses Browse Responses »
Page: Opening Question
1.1. Please select one option from the following list:
I primarily use P2Rx to find
environmental assistance
information to apply to my own
organization or facility.
I primarily use P2Rx in my
capacity as a provider of
environmental assistance to <-
others outside my organization.
Response Response
Percent Count
60.0%
40.0%
answered question
skipped question
10
Page: Questions for Regulated Community
2. 2. What type of organization do you
Federal government
State government
Manufacturing
Service industry
Educational institution
Nonprofit organization)
Wastewater treatment industry
.-• view nthnr fnlnacn cnnrifvt •
I
represent? (Check all that apply.)
Response
Percent
0.0%
0.0%
0.0%
0.0%
0.0%
| 33.3%
0.0%
1 33.3%
answered question
skipped question
Response
Count
0
0
1
0
0
0
1
0
1
3
7
3. 3. How often do you use P2Rx? (Check only one answer.)
First time user[
Weekly
Monthly
Every three months
Response Response
Percent Count
66.7%
0.0%
0.0%
0.0%
• vew
Other (please specify)
33.3%
answered question
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (1 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
skipped question
4. 4. How did you hear about P2Rx? (Check only one answer.)
Respon
Percen
Search engine 0
Link from another website
E-mail message or listserv
Conference or meeting
Brochure or newsletter
Colleague
Vendor
Assistance program
\=) view Othor /nloaco cnorifvt .
r Filler ^fJIcaoc opclrllyf
^^
33
0
0
0
0
0
11
se
t
.0%
.3%
.3%
.0%
.0%
.0%
.0%
.0%
.3%
answered question
skipped question
Response
Count
0
1
1
0
0
0
0
0
1
3
7
5.5.
only
P2Rx helps me to improve my awareness
one answer.)
Neutral
Disagree
Strongly Disagree
Do Not Use
of environmental practices to reduce pollution. (Check
Response Response
Percent Count
3 33.3% 1
0.0% 0
0.0% 0
0.0% 0
0.0% 0
answered question 3
skipped question 7
Page: Questions for Regulated Community (continued)
6. 6. How useful were P2Rx resources in meeting your needs? (Please make one selection for each
resource listed.)
lino \A/ o t"f*
u«7u, *•" S°umse7r Not useful of-Do not •—-
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
P2 Results Data System
(Measurement Tool)
33.3%
33.3%
33.3%
33.3%
0.0%
0.0%
33.3%
33.3%
0.0%
33.3%
(1)
(1)
(1)
(1)
(0)
(0)
(1)
(1)
(0)
(1)
33.3%
33.3%
33.3%
33.3%
(1)
(1)
(1)
(1)
100.0% (3)
33.3%
0.0%
0.0%
0.0%
0.0%
(1)
(0)
(0)
(0)
(0)
0.
0.
0.
0.
0.
33
33
33
33
33
0%
0%
0%
0%
0%
.3%
.3%
.3%
.3%
.3%
(0)
(0)
(0)
(0)
(0)
(1)
(1)
(1)
(1)
(1)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
% (0)
% (0)
% (0)
% (0)
% (0)
% (0)
% (0)
% (0)
% (0)
% (0)
use
33.3% (1)
33.3% (1)
33.3% (1)
33.3% (1)
0.0% (0)
33.3% (1)
33.3% (1)
33.3% (1)
66.7% (2)
33.3% (1)
answered question
skipped question
3
3
3
3
3
3
3
3
3
3
3
7
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (2 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
7. 7. How did you utilize the P2Rx website(s)? (Check all that apply.)
Reviewed Topic Hub
information (or other PZKX | I
website information)
Attended a training, workshop, or
Center
Ac-ifpH for nnllution nrpvpntion • •
Requested additional resources
from a P2Rx Center
Contacted a technical assistance
provider
Other (please specify)
Response
Percent
66.7%
33.3%
33.3%
0.0%
0.0%
0.0%
answered question
skipped question
Response
Count
2
1
1
0
0
0
3
7
Page: Questions for Regulated Community (continued)
3. 8. What action(s) have you taken to improve environmental practices
information you have found through P2Rx? (Check all that apply.)
Changed a pollution prevention
process or practice (i.e., , ,
imnlpmpntpH nolhition
prevention)
Identified a pollution , ,
nrpvpntion nnnnrtiinitv '
Purchased new process
equipment to prevent pollution
Implemented energy conservation
measures
Switched to renewable energy
Contacted a vendor| I
Implemented material or waste , ,
Phnnnprl hnnrllinn of wn^tp or .
Installed pollution control
equipment (e.g., scrubbers,
control technique)
Installed a waste treatment system
No process changes were taken
..:,,..
" uiner (piease specuy)
^^^^^^^^^^^^^
, in whole or in part,
Response
Percent
66.7%
66.7%
0.0%
0.0%
0.0%
33.3%
66.7%
33.3%
0.0%
0.0%
0.0%
33.3%
answered question
skipped question
due to
Response
Count
2
2
0
0
0
1
2
1
0
0
0
1
3
7
9. 9. Please identify whether you reduced or eliminated pollutant(s) as a result of P2Rx use. (Check all
that apply.)
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (3 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
(solid waste)'
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or i i
hoTdrrlni ic mdtoridlcl 1
Fliminatpd hr^ardou^ wa^tp or • •
Reduced fugitive air emissions
Eliminated air emissions
Reduced waste water pollutionl I
Eliminated pollutant waste water
Don't Know
None
(|) view other (please specify^ . .
Response
Percent
— I 66.7%
0.0%
33.3%
33.3%
0.0%
0.0%
33.3%
0.0%
0.0%
0.0%
33.3%
answered question
skipped question
Response
Count
2
0
1
1
0
0
1
0
0
0
1
3
7
10.10. Has using P2Rx or the action(s) taken above resulted in cost savings to you? (Cost includes
time as well as dollars.) (Check only one answer.)
Yes
No[
Do not know[
Response
Percent
0.0%
33.3%
66.7%
If yes, please provide details.
answered question
skipped question
Response
Count
Page: Questions for Assistance Provider Community
11.2. What type of organization do you represent? (Check all that apply.)
Response
Percent
0.0%
1 75.0%
0.0%
0.0%
0.0%
I 25.0%
I 25.0%
0.0%
0.0%
answered question
skipped question
Federal government
State government
Local government
Manufacturing
Service industry
Educational institution[
Nonprofit organization[
Wastewater treatment industry
Other (please specify)
Response
Count
12. 3. How often do you use P2Rx? (Check only one answer.)
Response Response
Percent Count
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (4 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
First time user
Weekly
| 25.0% 1
| 75.0% 3
Monthly 0.0% 0
Every three months 0.0% 0
Other (please specify) 0.0% 0
answered question 4
skipped question 6
13. 4. How did you hear about P2Rx? (Check only one answer.)
Search engine
Link from another website! I
E-mail message or listserv
Conference or meeting
Brochure or newsletter
Colleague! I
Vendor
Assistance program] I
Other (please specify)
Response
Percent
0.0%
25.0%
0.0%
0.0%
0.0%
50.0%
0.0%
25.0%
0.0%
answered question
skipped question
Response
Count
0
1
0
0
0
2
0
1
0
4
6
14. 5. Access to the P2Rx information has improved my ability to provide technical assistance to my
clients. (Check only one answer.)
Response Response
Percent Count
Strongly Agree! I 50.0% 2
25.0% 1
25.0% 1
Agree
Neutral
Disagree 0.0% 0
Strongly Disagree 0.0% 0
Do Not Use 0.0% 0
answered question 4
skipped question 6
Page: Questions for Assistance Provider Community (continued)
15. 6. How useful were P2Rx resources in meeting
resource listed.)
your
ul
needs? (Please make one selection for each
"SET Not useful o^oTt «-£•"•
use
Topic Hubs
P2 Programs Directory
News
Library
Industry Sector Information
Vendor Database
Case Studies
Listservs
Rapid Response
100
33.
33.
33.
100
33.
33.
33.
33.
.0% (3)
w /O ill
w /O ill
3% (1)
.0% (3)
3% (1)
3% (1)
3% (1)
3% (1)
0.0%
33.3%
33.3%
66.7%
0.0%
0.0%
66.7%
33.3%
0.0%
(0)
(1)
(1)
(2)
(0)
(0)
(2)
(1)
(0)
0.
33
33
0.
0.
66
0.
33
33
0% (0)
.3% (1)
.3% (1)
0% (0)
0% (0)
.7% (2)
0% (0)
.3% (1)
.3% (1)
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
33.3%
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(1)
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
3
3
3
3
3
3
3
3
3
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (5 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
66-7% (2) 33'3% ^ a°% (0) a°% (0) 0.0% (0) 3
answered question 3
skipped question 7
16. 7. What information di
environmental practices t<
Provided training, work
and/or conference infer
Provided information on tec
assistance pr
Provided Topic Hub infer
Provided vendor infoi
^ view Other pollutio
prevention technical assists
so, please j
d you use to improve or influence your clients' unde
o reduce pollution? (Check all that apply.)
[shop, 1
mat! on I I
•*hnipn|
nation I
mation |
n
1
rstanding of
Response
Percent
100.0%
33.3%
100.0%
66.7%
33.3%
nee. If
specify.
answered question
skipped question
Response
Count
3
1
3
2
1
3
7
Page: Questions for Assistance Provider Community (continued)
17. 8. What action(s) have your clients taken to improve environmental practices, in whole or in part,
due to information you have found through P2Rx? (Check all that apply.)
Response Response
Percent Count
Changed a pollution prevention
process or practice (i.e., . . 100 no/
implemented pollution L I iuu.u/o
prevention)
Identified a pollution
. - .
prevention opportunity1 - '
Purchased new process 0 Q0/ 0
equipment to prevent pollution
Implemented energy conservation 0 Q0/ 0
measures ' °
Switched to renewable energy 0.0% 0
Contacted a vendor 0.0% 0
Implemented material or waste , , inn no/ o
recycling system' I luu.u/o 2.
Changed handling of waste or . - .
'
Installed pollution control
equipment (e.g., scrubbers, 0.0% 0
control technique)
Installed a waste treatment system 0.0% 0
No process changes were taken 0.0% 0
view
Other (please specify) | | 50.o%
answered question
skipped question 8
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (6 of 7) [4/14/2008 4:47:15 PM]
-------
Survey Results
18. 9. Please identify whether your client(s)
(Check all that apply.)
/Qolirl AA/act^^l
Eliminated non-hazardous waste
(solid waste)
Reduced hazardous waste or ,
ho7orHni IG mat Aria IcI
Eliminated hazardous waste or
hazardous materials
Reduced fugitive air emissions!
Eliminated air emissions
Reduced waste water pollution
Eliminated pollutant waste water)
Don't Know
None
Other (please specify)
reduced or eliminated pollutant(s) as a result of
Response
Percent
0.0%
| 100.0%
0.0%
| 100.0%
0.0%
0.0%
| 50.0%
0.0%
0.0%
0.0%
answered question
skipped question
P2Rx use.
Response
Count
2
0
2
0
2
0
0
1
0
0
0
2
8
19. 10. Has using P2Rx or the action(s) taken above resulted in cost savings to your client? (Cost
includes time as well as dollars.) (Check only one answer.)
Response Response
Percent Count
Yes| | 100.0% 2
No 0.0% 0
Do not know 0.0% 0
If yes, please provide details.
answered question
skipped question 8
20. 11. Has using P2Rx saved you time or money in serving
Ypcl
No
Do not know
If yes, please provide details.
your client(s)? (Check only on
Response
Percent
I 100 0%
0.0%
0.0%
^f) view
answered question
skipped question
e answer.)
Response
Count
2
0
0
1
2
8
http://www.surveymonkey.com/sr.aspx?sm=zJKRSdlqEqWy5UB3RI5iAxXZS3bURKfWrkfXFbOo7gs_3d (7 of 7) [4/14/2008 4:47:15 PM]
-------
Appendix E - 2004 Customer Satisfaction Survey Analysis
The 2004 Customer Satisfaction Survey was analyzed in 2004 by Weinreich Communications.
However, certain questions were not split into distinct business and TAP audiences at that time.
Abt Associates analyzed these questions using Weinreich Communications' methodology for
dividing the audiences for purposes of consistency; thus, respondents are separated into two
categories: those who identified themselves as "Business/Industry," and all other respondents.
"All other respondents" are noted in the following graphs as "TAPs," however it is not certain
based on the survey questions whether other respondents were functioning as assistance
providers. For certain questions, raw data provided to Abt Associates by Weinreich
Communications was also broken down by center.
Question 2: Please choose the category that best describes you: (select one)
Response Rates, by Center
R1/2 R3/4 R5 R6 R7 R8 R9 R10
Center
• Other
n Consultant
• Academia
DNGO
• Business
D Local Gov
n State Gov
• Fed Gov
DTAP
Question 5: How did you find out about P2Rx/Regional Center? (check all that apply)
Q5. Businesses
D Search Engine
D Link from other website
D Email message or list
serve
D Trade show or
conference
• Brochure or newsletter
D Vendor
• Colleague
Evaluating the Effectiveness of the EPA PPIN Grant Program
163
-------
Q5. Businesses
D Search Engine
• Link from other
website
D Email message or list
serve
D Trade show or
conference
• Brochure or
newsletter
D Vendor
• Colleague
D Government
assistance provider
• Other
Q5. Non-Business
D Search engine
• Link from other website
D Email message or list
serve
D Brochure or newsletter
• Colleague
n Other
70%
Q5. Non-Business
D Search engine
• Link from other
website
D Email message o
listserve
D Brochure or
newsletter
• Colleague
D Other
R10
Evaluating the Effectiveness of the EPA PPIN Grant Program
164
-------
Question 7: How often do you use the materials or information you receive from P2Rx/Regional
Center for the following actions?
e. Assist my own organization in its pollution prevention activities
Q7e. Businesses, by center
100% n
80% -
60% -
40% -
20% -
0% -
D never, w on't use
• never, might use
D daily
D w eekly
D monthly
D yearly or less
C\|3;l£)CDI^000'0
£ ce ce
Center
Q7e. Businesses, all centers
D yearly or less
• monthly
D weekly
D daily
• never, might use
D never, won't use
Q7e. Non-Business, by center
• n/a
D never, w on't use
• never, might use
D daily
D w eekly
D monthly
D yearly or less
Center
Q7e. Non-Business, all centers
D yearly or less
• monthly
D w eekly
D daily
• never, might use
D never, w on't use
• n/a
/ Assist a client with reducing or preventing pollution (nonbusiness only)
Q7j. Non-Business, by center
100% n
90% -
80% -
70% -
60% -
50% -
40% -
30% -
20% -
10% -
0% -
Center
• n/a
D never, won't use
• never, might use
D daily
D w eekly
• monthly
D yearly or less
Q7j. Non-Business, all centers
D yearly or less
• monthly
D weekly
D daily
• never, might use
D never, won't use
• n/a
Evaluating the Effectiveness of the EPA PPIN Grant Program
165
-------
k. Achieve (/help a client achieve) compliance through reduction of pollution generation
Q7k. Business, by center
D never, won't use
• never, might use
D daily
D w eekly
• monthly
D yearly or less
Center
Q7k. Business, all centers
D yearly or less
• monthly
D w eeMy
D daily
• never, might
use
Q7k. Non-Business, by center
100%
80% -
60% -
40% -
20% -
0%
• n/a
D never, w on't use
• never, might use
D daily
D w eekly
D monthly
D yearly or less
Q7k. Non-Business, all centers
D yearly or less
• monthly
D w eekly
D daily
• never, might use
D never, won't use
• n/a
Center
Evaluating the Effectiveness of the EPA PPIN Grant Program
166
-------
Question 9: How useful are our resources for your needs?
Q9. Businesses
• N/A
D Not useful
D Somewhat useful
• Useful
D Very useful
6
^d?x
^
Q9. Non-Business
• N/A
D Not useful
D Somewhat useful
• Useful
DVery useful
Evaluating the Effectiveness of the EPA PPIN Grant Program
167
-------
Question 11: How useful are our services for your needs?
100% -
90% -
80% -
70% -
60% -
50% -
40% -
30% -
20% -
10% -
0% -
Q
-
=
m.Bi
isinesses
Listservs Regional Rapid Response
Roundtables
• N/A
D Not useful
D Somewhat useful
• Useful
D Very useful
100%-
Q0% -
pnoA
/ UYO
fino/.
cno/
AC\0/n
?0%
onoi
mo/, -
0%
n
Q1
B
I.Nor
i-Business
Listserves Regional Roundtables Rapid Response
• N/A
D Not useful
D Somewhat useful
D Useful
D Very useful
Evaluating the Effectiveness of the EPA PPIN Grant Program
168
-------
Question 12: How useful to you are the following types ofP2 information?
Q12. Businesses
o°
• N/A
D Not useful
D Somewhat useful
• Useful
D Very useful
Q12. Non-Business
• N/A
D Not useful
D Somewhat useful
• Useful
D Very useful
Evaluating the Effectiveness of the EPA PPIN Grant Program
169
-------
Appendix F - Interview Guide
Introduction
Good morning. I am (introduce self). I work for Abt Associates, a research
consulting firm in Cambridge, MA. We are working with the U.S. Environmental Protection
Agency to conduct an evaluation of the Pollution Prevention Resource Exchange, orP2Rx.
We're conducting interviews with P2Rx Center Directors, Technical Assistance Providers, and
Business Representatives to get their input for our evaluation.
This interview is being conducted to get your input about the [Pollution Prevention Resource
Exchange/specific center name]. All of your comments will remain confidential. We will be
compiling a report that will contain a synthesis of all comments without any reference to
individuals.
Questions for P2Rx Center Directors
[Send logic model(s) out ahead of time, ask them to review and have ready]
Questions about the Logic Model: .;'.•.., •.;.;•. .;.,••!•.•«
Did you receive the logic models we sent you via e-mail? We sent the original one you/your
staff developed and sent to EPA, plus a generic one that should roughly describe all of the
centers. However, we know that the centers differ from one another in how they promote P2.
Part of our evaluation is to identify the similarities and differences so we can group the centers
together based on their common modes of information dissemination and other
characteristics. As we go along with our questions, feel free to point out connections to other
centers you may be aware of.
A. Resources:
Can you identify from the list of the resources in the generic logic model, the ones your center
primarily uses and the partners you primarily work with?
Which resources listed, if any, does your center not utilize?
Are there any resources your center uses that are not listed?
B. Activities:
Can you identify from the list of activities in the generic logic model, the ones that your center
spends the most resources on?
What do you consider your most important activities in terms of fulfilling your mission?
Which activities do you do, but are not a major focus?
Which activities listed, if any, does your center not do at all?
Are there any activities your center is involved in that are not listed?
C. Outputs:
Evaluating the Effectiveness of the EPA PPIN Grant Program 170
-------
Can you identify from the list of outputs in the generic logic model, the ones that your center
spends the most resources on?
Which outputs do you produce, but are not a major focus?
Which outputs, if any, does your center not produce at all?
Are there any outputs your center produces that are not listed?
D. Customers:
Can you identify your primary customers on the list in the logic model?
Why, or in what way, do you consider them primary customers?
To what extent do you focus your efforts on providing information to TAPs as opposed to
businesses directly?
Who would you consider your secondary customer(s) on the list provided?
Are there any customers listed with whom your center does not work?
Do you work with any customers that are not listed?
II. What do you consider particularly effective or efficient strategies in how you operate?
A. What do you consider particularly ineffective or inefficient?
III. Are there ways in which things can be shared among centers to make you more efficient in
delivering information?
A. For which tasks, if any, would this work well?
OK, now moving away from the logic models...
IV. What do you do to reach businesses through TAPs?
A. Why do you use this approach?
B. Is it effective at ultimately reaching businesses?
C. (Whether answer is "yes" or "no" ask) How do you know?
V. What do you do to reach businesses directly?
A. How does this differ from how you reach them through TAPs?
B. Why do you use this approach?
C. Do you know if it is effective? How do you know?
VI. How do you determine the needs of your users?
Evaluating the Effectiveness of the EPA PPIN Grant Program 171
-------
VII. What kind of outcomes could your center potentially affect...
(outcomes are changes in conditions on the part of your audience (e.g., increased knowledge,
made changes in behavior), or changes in environmental and human health.
A. for TAPs?
B. Through TAPs for businesses?
C. for business directly?
D. How do you know?
VIII. Do you think your center has had an effect on any of these outcomes?
A. If so, how do you know?
IX. What do you know about the effect your center has had on the implementation of P2 by
business? (Zero Waste - probe for case studies)
A. Do you collect information to help you answer this question?
B. If so, what type, how, and how frequently?
X. What information would be useful to collect to measure the influence of your center's activity
on the implementation of P2?
A. What would be your recommendations for meaningful performance measurement
information?
B. Can you explain the advantages of that approach?
XI. What ability does your center currently have to gather outcome measurements?
[e.g., Expertise, data systems, infrastructure]
A. Is it sufficient? If not, in what ways?
XII. How could you build your capacity to improve collection of desired outcome or performance
measurements/information?
A. What kinds of information are you missing?
B. What do you need in order to collect it?
XIII. What ideas or recommendations do you have for how such information can best be collected
based on available technology/databases, funds, and survey approval requirements?
Evaluating the Effectiveness of the EPA PPIN Grant Program 172
-------
For TAPs and industry:
I'd like to start by having you briefly describe your interaction with the [P2Rx center]. (Note to
interviewer: You may need to probe to gather information on the extent of their interaction,
frequency, etc.).
Questions for Technical Assistance Providers
[Replace "P2Rx" with the name of the specific center.]
I. What do you need from P2RX centers to promote businesses' adoption of pollution prevention
practices?
A. In what way(s) will that help you do your job?
II. In your view, are the P2Rx centers providing the right products, services, and content to you
to promote the adoption of P2 practices by businesses?
A. In what way(s) are they/are they not right?
B. How do you know?
III. What additional products, services, and content do you think businesses need in order to
consider the adoption of P2 practices?
A. Why?
B. How do you know?
IV. Based on your knowledge or your opinion (specify which), what effect has P2Rx had on the
implementation of P2 by business?
A. How do you know or why do you think this?
V. What type of outcomes do you think can be attributed to the following information received
from P2Rx centers?
A. Information received by TAPs.
B. Information received by businesses through TAPs?
C. Information received by business directly?
D. Why do you think these outcomes can be attributed all or in part to information from P2RX
centers, as opposed to other influences or causes? What other sources could these
outcomes have?
Evaluating the Effectiveness of the EPA PPIN Grant Program 173
-------
Questions for Business Representatives
I. Please describe what type of business you are in.
II. What information and services do you need to make decisions about whether to adopt
pollution prevention (P2) practices? By P2 we mean specifically waste reduction activities, or
what you might think of as sustainable production or source reduction. We're not talking
about environmental improvements in general.
[hold prompts]:
For example, topic-specific information, sector-specific information, news items,
vendor information, case studies/success stories, analytical tools, on-site visits,
personalized on-line research assistance, networking, or paper such as fact sheets,
checklists, guidance manuals or handbooks, articles or reports.
A. Why is this helpful to you?
B. How do you prefer receiving this information and services?
[hold prompts]:
For example, websites, phone assistance, live training, webinars, passive internet
resources, direct mail, teleconferencing, listserves
C. Why is this your preference?
III. In your view, are the P2Rx centers providing, either directly or indirectly, the right products,
services, and content to help you implement P2?
A. Are you referring to direct or indirect services?
B. In what way(s) are they/are they not right?
C. What do you need from the center to help you implement P2?
IV. In your view, are the TAPs providing the right products, services, and content to help you
implement P2?
A. In what way(s) are they/are they not right?
B. If not, what products, services or content do they need to provide in order to help
you implement P2?
V. Have you used the information provided by a P2Rx center to implement P2 practices?
A. If yes, what information did you use?
B. Can you give some examples of instances in which you were able to use
information to implement changes?
C. Were there documented changes in outcomes? What types?
Evaluating the Effectiveness of the EPA PPIN Grant Program 174
-------
VI. What leads you to seek information about P2? (evaluation question 5a)
VII. How successful do you think the P2Rx centers have been at reaching you and your peers in
business and industry? Use a scale of 1 to 5, with 5 being extremely successful, 1 being not
at all successful, (evaluation question 7)
A. How do you gauge that success?
B. What means did they use to reach you?
Evaluating the Effectiveness of the EPA PPIN Grant Program 175
-------
Appendix G - Focus Group Moderator's Guide
2007 Western U.S. Pollution Prevention Conference
Moderators' Guide
October 23-25, 2007
Focus Group with Business
Hello and welcome to our discussion group. Thanks so much for taking the time to join us. My
name is Jocelyn Siegel, and this is my colleague Lissa Lynch. We work for Abt Associates, a
research consulting firm in Cambridge MA. We are working with the U.S. Environmental
Protection Agency to conduct an evaluation of the Pollution Prevention Resource Exchange, or
P2Rx. We're conducting focus groups with the regulated community to learn your perspectives
about P2Rx.
P2Rx is a consortium of eight regional pollution prevention information centers, funded in part
through grants from EPA. These centers all provide pollution prevention information, networking
opportunities and other services.
The centers you might be familiar with are:
• Western Regional Pollution Prevention Network (WRPPN)
• Ed Gonzalez
or
• Pacific Northwest Pollution Prevention Resource Center (PPRC)
• Chris Wley
The overall goal of this focus group is to gather information about P2Rx to aid in our evaluation
of the program's effectiveness in reaching the regulated community (P2Rx's end users) with the
information and mechanisms provided. This evaluation is not intended as a critique of centers'
achievements or comparison of achievements across centers.
Please write your first name on the table tents. This is just to facilitate the session; your name
will not be used when reporting what we learn in focus groups. For those of you unfamiliar with
focus groups, a focus group is a planned discussion designed to get opinions and hear
experiences about a specific area of interest, in this case the P2Rx centers. The purpose of a
focus group is to hear views, not to get "answers."
Before we begin, I'd like to go over some ground rules. First, it's very important to respect one
another both in terms of letting everyone voice their opinions if they wish, and by not sharing
what you hear today with anyone outside the group. We will keep your comments confidential
and will not use your names in any reports. These rules aim to help make the focus group an
environment where people can be open and honest, which will provide the study team with the
most accurate picture of P2Rx.
Our conversation will last about an hour and a half, but if at any time you want to leave, please
feel free to do so. If any part of the conversation makes you uncomfortable, we would very
much appreciate you letting us know. You can do so either by telling me or Lissa after the
session ends.
Evaluating the Effectiveness of the EPA PPIN Grant Program 176
-------
We would like to tape record the sessions to be sure our notes are accurate. Does anyone
have any objections?
Before we start our discussion, does anyone have any questions?
OK, let's begin.
1. What type of businesses are people in, and what is your role? (go around room)
Questions about direct contact with P2Rx centers:
2. How many of you have had contact with a P2Rx center?
a. If so, can you describe briefly what the circumstances of the contact you
had/have.
(go around room)
b. What kinds of information have you received from a P2Rx Center?
3. Have you used the information provided by a P2Rx Center to implement pollution
prevention, or P2 practices? By P2 we mean specifically waste reduction activities, what you
might think of as sustainable production or source reduction. We're not talking about
environmental improvements in general.
If so, please elaborate.
c. If so, is information about this implementation publicly available?
Questions about contact with TAPs:
4. How many of you have had contact with a state or local Technical Assistance Providers, or
TAPs?
(You may be familiar with TAPs as local or regional programs that provide businesses
with environmental management assistance and help identify and implement P2
measures through such things as voluntary onsite audits, planning assistance, training
or other services.)
d. If so, can you describe briefly what the circumstances of the contact you
had/have, and how often you interact with them?
(go around room)
e. If you've interacted with TAPs, have they mentioned the P2Rx center to you?
f. Specifically what kinds of information have you received from a TAP?
5. Have you used the information provided by a TAP or center to implement P2 practices?
Please elaborate.
g. If so, is information about this implementation publicly available?
6. Are there suggestions you would make, or recommendations about getting P2Rx
information directly to you more effectively?
General questions:
7. When you do need pollution prevention information, where do you typically look? Why?
What type of information or assistance are you trying to find?
Evaluating the Effectiveness of the EPA PPIN Grant Program 177
-------
8. What information and services do you need to make a decision or consider adopting a P2
practice?
[hold prompts]:
h. What do you need?
Topic-specific information, sector-specific information, news items, vendor
information, case studies/success stories, analytical tools, on-site visits,
personalized on-line research assistance, networking, or paper such as fact
sheets, checklists, guidance manuals or handbooks, articles or reports.
\. How do you want to get it?
Websites, phone assistance, live training, webinars, passive internet resources,
direct mail, teleconferencing, listserves.
This is our final question:
9. We've conducted this meeting to help us evaluate the effectiveness of P2Rx. In addition to
finding out how well P2Rx and its centers are working, we want to learn how to improve the
program for the future. Is there anything we've missed? Is there anything you came to this
meeting wishing to say that you haven't had a chance to say?
Thank you for participating in this focus group. Your comments and viewpoints are extremely
valuable, and we appreciate you taking the time to share them with us. We'll be around for a
few minutes if you have any questions about the study, and feel free to contact Beth Anderson
at EPA Headquarters with any questions in the future. The report is due to be completed next
summer and you can see a copy by contacting Beth Anderson.
Evaluating the Effectiveness of the EPA PPIN Grant Program 178
-------
2007 Western U.S. Pollution Prevention Conference
Moderators' Guide
October 23-25, 2007
Focus Group with Technical Assistance Providers
Introduction:
Hello and welcome to our discussion group. Thanks so much for taking the time to join us. My
name is Jocelyn Siegel, and this is my colleague Lissa Lynch. We work for Abt Associates, a
research consulting firm in Cambridge MA. We are working with the U.S. Environmental
Protection Agency to conduct an evaluation of the Pollution Prevention Resource Exchange, or
P2Rx. We're conducting focus groups with Technical Assistance Providers to learn your
perspectives about P2Rx. We're also holding focus groups with the regulated community for
the same.
P2Rx is a consortium of eight regional pollution prevention information centers, funded in part
through grants from EPA. These centers all provide pollution prevention information, networking
opportunities and other services. (Be prepared to define pollution prevention.)
The centers you might be familiar with are:
• Western Regional Pollution Prevention Network (WRPPN)
• Ed Gonzalez
or
• Pacific Northwest Pollution Prevention Resource Center (PPRC)
• Chris Wley
The overall goal of this focus group is to gather information about P2Rx to aid in our evaluation
of the program's effectiveness in reaching the regulated community (P2Rx's end users) with the
information and mechanisms provided. This evaluation is not intended as a critique of centers'
achievements or comparison of achievements across centers.
Please write your first name on the table tents. This is just to facilitate the session; your name
will not be used when reporting what we learn in focus groups. For those of you unfamiliar with
focus groups, a focus group is a planned discussion designed to get opinions and hear
experiences about a specific area of interest, in this case the P2Rx centers. The purpose of a
focus group is to hear views, not to get "answers."
Before we begin, I'd like to go over some ground rules. First, it's very important to respect one
another both in terms of letting everyone voice their opinions if they wish, and by not sharing
what you hear today with anyone outside the group. We will keep your comments confidential
and will not use your names in any reports. These rules aim to help make the focus group an
environment where people can be open and honest, which will provide the study team with the
most accurate picture of P2Rx.
Our conversation will last about an hour and a half, but if at any time you want to leave, please
feel free to do so. If any part of the conversation makes you uncomfortable, we would very
much appreciate you letting us know. You can do so either by telling me or Lissa after the
session ends.
We would like to tape record the sessions to be sure our notes are accurate. Does anyone
have any objections to this?
Evaluating the Effectiveness of the EPA PPIN Grant Program 179
-------
Before we start our discussion, does anyone have any questions?
OK, let's begin.
1. I'd like to get a sense of whether you are from a state or a local TAP, or a related
organization, and what your role is. (go around room.)
2. What do you feel you need to promote businesses' adoption of pollution prevention, or
P2 practices?
Note: when we say "businesses," we mean broadly your end users, or the regulated
community. Is there a term that would be more suitable?
3. In your view, are the P2Rx centers providing the right products, services, and content to
you to promote the adoption of P2 practices by businesses?
a. What value do you see in P2Rx for you and your clients?
4. How would you describe your role in terms of promoting P2Rx and the centers? Do you
pass on information about P2Rx when you're helping businesses? If so, what types of
information and how?
a. To what extent do you actively promote P2Rx?
(scale of 1-5, 1 being not at all, and 5 being to a great extent)
b. How useful do you feel that has been at helping businesses make changes?
(scale of 1-5 again)
c. Which resources provided by P2Rx have been most helpful? Which have not
been particularly helpful?
5. What do you know about the effect P2Rx has had on the implementation of P2 by
business?
a. Are you aware of any businesses that have used P2Rx information to make
changes?
b. If so, is there any information publicly available about that?
6. What suggestions can you make about how to measure the effectiveness of the P2Rx
centers on the adoption of P2 by businesses and other end users?
7. Are you aware of EPA's interest in having the P2Rx centers deal more directly with end
users? What do you think about this?
8. Are there suggestions you would make, or recommendations, about getting P2Rx
information to the end users more effectively?
This is our final question:
9. We've conducted this meeting to help us evaluate the effectiveness of P2Rx. In addition
to finding out how well P2Rx and its centers are working, we want to learn how to
improve the program for the future. Is there anything we've missed? Is there anything
you came to this meeting wishing to say that you haven't had a chance to say?
Thank you for participating in this focus group. Your comments and viewpoints are extremely
valuable, and we appreciate you taking the time to share them with us. We'll be around for a
Evaluating the Effectiveness of the EPA PPIN Grant Program 180
-------
few minutes if you have any questions about the study, and feel free to contact Beth Anderson
at EPA Headquarters with any questions in the future. The report is due to be completed next
summer and you can see a copy by contacting Beth Anderson.
Evaluating the Effectiveness of the EPA PPIN Grant Program 181
-------
Appendix H - Center and National P2Rx Documents
Reviewed for Analysis
• Grant reports submitted periodically to EPA
• PPRC case studies: Canyon Creek Cabinets, Columbia Paint and Coatings, Lasco
Bathware, and Woodfold Manufacturing
• Case studies of a fiberglass molding company, a chemical company, and a
thermoplastics company from Zero Waste (e-mail)
• Case examples of successes from GLRPPR (e-mail)
• Report for the EPA Region 6 PPIN Grant, FY 2007, Zero Waste Network. Final Report
for three year P2Rx Grant Awarded to the University of Texas, Austin
• Conference Evaluation Summaries from WRPPN/PPRC 2007 conference
• Survey Results from State P2 Coordinators about FY 2006 Priorities (Region 8/Peaks to
Prairies)
• Tribal Needs Assessment Survey (Peaks to Prairies)
• NEWMOA FY 2006, 2007, 2008 Survey Results
• Summary of Feedback on P2 Results Data System (GLRPPR)
• Great Lakes/Great Plains P2 Conference March 13 & 14, 2007 Chicago, Illinois
feedback form
• December 2006 "Ask your states..." survey conducted by EPA
• Library Evaluation and Performance Measures: A Literature Review and Research
Summary prepared by Fred MacVaugh for PPRIC, February/March 2007
• Various e-mails from center directors and staff
• Center and P3Rx websites
• 2007-2008 P2Rx Strategic Plan External Draft, Revised 1/18/07
• "P2Rx Centers' Processes for Identifying Regional Needs" November 8, 2006
• "Why P2Rx?"
Evaluating the Effectiveness of the EPA PPIN Grant Program 182
-------
Appendix I - Abt Associates Data Analysis Plan
Evaluating the Effectiveness of EPA's Pollution Prevention Information Network (PPIN)
Grant Program
Data Analysis Plan
Data analysis for the P2Rx evaluation will focus on primary data gathered through interviews
and focus groups conducted by Abt Associates in 2007 and 2008, as well as logic models
developed for this evaluation. This information will be supplemented by information obtained
from other sources, specifically:
• 2007/8 Behavior Survey
• Web Activity Measures
• 2004 Customer Satisfaction Survey
• 2004 Interview summaries
• Center-Specific documents
Where findings for a particular research question are contradictory, primacy will be given to the
primary data. This data analysis plan provides more extensive detail on the analysis of the
2007/8 Behavior Survey than on the other data sources, as requested by the EPA Work
Assignment Manager.
2008 Interviews
Abt Associates has conducted telephone interviews with three key audiences for the evaluation:
• P2Rx center directors or key staff
• Business representatives, and
• Technical assistance providers.
The interview questions were written specifically to address the evaluation questions for which
there is no other source of information. Because these interviews are a primary data source,
and provide information rich in context and insight, these results will form the foundation of the
overall evaluation. Further, the ability to address the three key audiences provides information
not obtained elsewhere.
Findings will be synthesized and summarized by theme, generally tied to the evaluation
questions, for each of the three audience types. We will identify similar answers within each
audience type and where consistent answers are identified, a descriptive characterization will be
offered. The unique perspectives of each audience type, as well as of each individual center,
will be portrayed. For the centers, we will note whether or not we heard similar answers across
centers and what the similar and dissimilar responses were. On issues for which a set of
centers presented a consensus, we will synthesize those responses and contrast it with those
that departed from the consensus. In many cases there will be no clear consensus on a topic,
and these responses will be given less credibility in the report than those that are more
consistent.
Focus Groups:
Focus groups were conducted at the Western Regional Pollution Prevention Conference in San
Diego, a conference that was hosted by the PPRC and WRPPN centers. Focus groups were
conducted with technical assistance providers (TAPs) (2 groups) and with business
representatives (1 group). Representation at the focus groups was from these two regions.
The businesses represented were extremely varied. Taken in this context, results will be used
Evaluating the Effectiveness of the EPA PPIN Grant Program 183
-------
together with interview results as a foundation for the analysis. The focus groups provide in-
depth information and opinions from a small number of conference participants.
Similar to interviews, findings from the focus groups will be synthesized and summarized by
theme, generally tied to the evaluation questions, for each of the two audience types. We will
identify similar answers within each audience type and where consistent answers are identified,
a descriptive characterization will be offered. The unique perspectives of each audience type
will be portrayed. Information obtained from the TAPs will be much easier to summarize than
from the business group. For this audience, we will note whether or not we heard similar
answers across TAPs and what the similar and dissimilar responses were. On issues for which
a number of TAPs suggested a consensus, we will synthesize those responses and contrast it
with those that departed from the consensus. In many cases there will be no clear consensus
on a topic, and these responses will be given less credibility in the report than those that are
more consistent.
Logic Models
Logic models had been developed by each center individually, which were then combined by
Abt Associates into a single "generic" logic model. This generic model was used as a launching
point for drawing comparisons and relationships among different aspects of center operations.
Feedback was solicited on the generic logic model during interviews with P2Rx center staff.
This information will be analyzed and interpreted to identify similarities and differences between
centers, and possibly groupings among them. Logic models will be used to answer Evaluation
Question 1 only.
2007/8 Behavior Survey:
Behavior Survey administered using Survey Monkey
Behavior Survey administered in hard copy
The Behavior Survey was administered electronically through a link on each center's website
(using Survey Monkey), as well as being administered to various live audiences in a hard-copy
format. Both sets of results were provided to Abt Associates by individual center. Abt will
merge the center data together into a single data set and conduct our analysis on that set. A
detailed center-by-center comparison of these survey results is not within the scope of this
evaluation and can be obtained by examining the Survey Monkey results directly.
Hard copy surveys differ from electronically administered surveys in two ways: some response
options varied (to a small extent); and the hard copies did not limit the number of possible
responses on certain questions for which Survey Monkey restricted to one. For this reason we
may have to analyze the hard copy surveys slightly differently than the electronic surveys. Any
differences will be documented in the final report.
Logs were kept by some centers who reached out to specific audiences for responses, either in
administering hard-copy surveys or by sending directed e-mails asking people to respond to the
on-line survey. The log information will be used as available to identify particular sources of
bias in respondent types.
The following analysis will be conducted on survey results with numeric results presented in
tabular form and summary findings included in narrative with appropriate caveats regarding the
statistical limitations of the data. The analysis will be bivariate for all questions, with cross tab
descriptions for each of the two respondent groups.
Respondent Type:
Evaluating the Effectiveness of the EPA PPIN Grant Program 184
-------
The following question will be used to separate the two types of respondents.
1. Please select one option from the following list:
• I primarily use [P2Rx] to find environmental assistance information to apply to my
own organization or facility. ("Business")
• I primarily use [P2Rx] in my capacity as a provider of environmental assistance to
others outside my organization. ("TAPs")
The two groups will be analyzed together and separately.
Representation Type:
The following question will be used to determine what types of organizations responded
to the survey.
2. What type of organization do you represent? (Check all that apply.)
Frequency (%) for each possible response for each of two groups.
Frequency of Use:
This question will be used to determine how frequently each respondent used the P2Rx
or its center's resources.
3. How often do you use [P2Rx]? (Check only one answer.)
Frequency (%) of each possible response for each of two groups to compare use, and
total combined for summary of responses.
For the hard-copy responses, omit those who said they never used it from the analysis,
but capture the number.
How Learned About:
This question will be used to get an indication of how respondents became aware of
P2Rx and its center's resources.
4. How did you hear about [P2Rx]? (Check only one answer.)
Frequency (%) of each possible response for each of two groups to compare, and total
combined for summary of responses.
Awareness (Outcomes):
This question will be used to learn the extent to which short-term outcomes may have
been impacted by P2Rx for both business and TAPs.
5. (Business) [P2Rx] helps me to improve my awareness of environmental practices to
reduce pollution. (Check only one answer.)
5. (TAPs) Access to [P2Rx] information has improved my ability to provide technical
assistance to my clients. (Check only one answer.)
Frequency (%) of each possible response.
Usefulness:
This question will provide information about how useful each audience type found the
P2Rx center resources.
6. How useful were [P2Rx] resources in meeting your needs? (Please make one
selection for each resource listed.)
Frequency (%) of each possible response for each of two groups to compare, and total
combined for summary of responses.
Note: available responses to this question will vary by center outside of a core group of
resources. We may choose to analyze the core group only, as responses regarding
center-specific resources are limited by the number of respondents for that center.
Type of Use:
Evaluating the Effectiveness of the EPA PPIN Grant Program 185
-------
This question will provide information about what information was used and in what way
by each audience.
7. (Business) How did you utilize the [P2Rx] website(s)? (Check all that apply.)
7. (TAPs) What information did you use to improve or influence your clients'
understanding of environmental practices to reduce pollution? (Check all that apply.)
Frequency (%) of each possible response.
Note: Available responses to this question differ between the two audience types.
Results are not comparable.
Behaviors (Outcomes):
This question will be used to learn the extent to which intermediate-term outcomes may
have been impacted by P2Rx for both business and TAPs.
8. (Business) What action(s) have you taken to improve environmental practices, in
whole or in part, due to information you have found through [P2Rx]? (Check all that
apply.)
8. (TAPs) What action(s) have your clients taken to improve environmental practices, in
whole or in part, due to information you have found through NEWMOA and/or P2Rx?
(Check all that apply.)
Frequency (%) of each possible response.
Condition (Outcomes):
This question will be used to learn the extent to which long-term outcomes may have
been impacted by P2Rx for both business and TAPs.
9. (Business) Please identify whether you reduced or eliminated pollutant(s) as a result
of [P2Rx] use. (Check all that apply.)
9. (TAPs) Please identify whether your client(s) reduced or eliminated pollutant(s) as a
result of [P2Rx] use. (Check all that apply.)
Frequency (%) of each possible response.
Cost Savings (Outcomes):
This question will be used to learn the extent to which long-term outcomes may have
been impacted by P2Rx for both business and TAPs.
10. (Business) Has using [P2Rx] or the action(s) taken above resulted in cost savings to
you? (Cost includes time as well as dollars.) (Check only one answer.)
10. (TAPs) Has using [P2Rx] or the action(s) taken above resulted in cost savings to
your client? (Cost includes time as well as dollars.) (Check only one answer.)
11. (TAPs) Has using [P2Rx] saved you time or money in serving your client(s)? (Check
only one answer.)
Frequency (%) of each possible response.
Web Measures
Use of the Web measures data will be used to answer Evaluation Question 2 only.14 Analysis
will summarize key points about the number of hits to each centers' site, the users' organization
14 Question 2: What groups/types of customers access P2Rx information?
How do the customers vary according to method of outreach by the centers?
Via the website
Via TAPs
Via direct contact (e.g., telephone, correspondence, or other communication)
Via P2Rx workshops, trainings, and regional meetings
Evaluating the Effectiveness of the EPA PPIN Grant Program 186
-------
type, and what products are accessed. Combined center data will be used to illustrate the
relative use of the national network by different audiences or by different product/service types.
A detailed center-by-center analysis will not be conducted. However, center-specific data will
be used in context to illustrate findings in the evaluation (i.e., relative popularity of a particular
resource mentioned during interviews). It may also be used to illustrate overall differences in
how the Centers function and where they focus their attention. Only loose connections can be
drawn to address the question of "How do the customers vary according to method of outreach
by the centers?" This will be done by looking at the most prevalent customer type and the most
prevalent method of outreach used, but a link from customer to outreach type is not able to be
drawn with the available data.
2004 Customer Satisfaction Survey
This survey had nearly 500 responses, mostly from state and local government and TAPs, with
only 7% of respondents from industry. To address specific evaluation questions, the data will
be broken out by customer group (i.e., business versus government/TAP respondents, where
possible). Data from select questions on this survey will be used to add detail to the analysis.
Additional insight will be gathered from the summary and PowerPoint presentations prepared by
Weinreich and Associates.
2004 Interview Summaries
Interviews were conducted by lEc Inc. for an earlier evaluation. While this evaluation did not
directly address the questions we are currently addressing, interviews were conducted with EPA
managers and project officers, P2Rx coordinators, P2Rx center contacts, and TAPs, and some
general points were captured that will help inform this evaluation. The summaries of these
interviews will used on an ad hoc basis to glean additional information that will help us respond
to the current set of evaluation questions.
Center-Specific documents
An array of documents has been obtained from individual centers, including surveys, case
studies and other materials. Surveys and case studies may be used to informally connect P2Rx
center activities to outcomes. Other documents will be used on an ad hoc basis to add detail to
the evaluation.
Evaluating the Effectiveness of the EPA PPIN Grant Program 187
------- |