United States
Environmental Protection
Agency
Prevention, Pesticides,
and Toxic Substances
(7409)
EPA742-R-01 -001
February 2001
www.epa.gov/oppt/epp
vvEPA
Qualitative Measurement of
Environmentally Preferable
Purchasing (EPP) Among
Federal Employees in 2000
Recycled/Recyclable-Printed with Vegetable Oil Based Inks on Recycled Paper (Minimum 30% postconsumer content).
-------
Qualitative Measurement of
Environmentally Preferable
Purchasing (EPP) Among
Federal Employees in 2000
Prepared for:
Julie Winters
Pollution Prevention Division
Office of Pollution Prevention and Toxics
U.S. Environmental Protection Agency
Washington, DC 20460
Contract Number: OW-0083-YTSA
February 2001
-------
Contents
Preface iii
Executive Summary v
Introduction: Project Overview 1
Research Objectives 1
Research Learning Questions 2
Research Design Overview 3
Secondary Research 5
Existing EPA Materials and Reports 5
Environmental Scan of EPP Penetration 5
Other Government Resources 5
Qualitative Research 7
Research Overview 7
Key Findings 7
Key Findings Organized by Learning Questions 12
Peer Review 23
Appendix A: Research Design 25
Appendix B: Screening Protocols 33
Appendix C: Interview Protocols 35
Appendix D: Secondary Research Bibliography 51
Appendix E: Interview Tracking 53
TABLES AND FIGURES
Table 1: Qualitative Study Learning Questions (LQs) 2
Table 2: Overall Cumulative Interviews 7
Figure 1: Participant Purchasing Roles 7
Table 3: Findings in Context of Learning Questions 13
Table 4: Study Population 27
Contents
-------
-------
Preface
The U.S. Environmental Protection Agency's (EPA's)
Environmentally Preferable Purchasing (EPP)
program is mandated by Executive Order 13101.
This Executive Order, entitled "Greening of
Government Through Waste Prevention, Recycling,
and Federal Acquisition," directs Executive agencies
to implement a variety of waste reduction and other
activities, and directs EPA to assist Federal agencies
in making purchasing decisions that are less
damaging to the environment.
To achieve the goals of environmentally preferable
purchasing set forth in the Executive Order, EPA's
EPP program:
issued guidance articulating five principles for
applying EPP in the Federal government setting;
initiated demonstration pilot projects in
conjunction with Federal agencies; and
developed outreach materials and tools to train
and provide assistance to Federal purchasers.
To strategically guide the EPP program toward the
most effective methods and program activities, and
for the purposes of the Government Performance
and Results Act (GPRA), EPA initiated a multi-
phased market research or measurement project.
The project is designed to measure the awareness
and success of current EPP efforts and identify what
motivates people to connect environmental
considerations to their purchases of products and
services.
The measurement research activities were designed
to:
obtain immediate qualitative feedback on the
effectiveness of the EPP program;
identify qualitative inputs to the development of
quantitative research, future tools and resources
for purchasers, outreach activities, and other EPP
methods or policies; and
develop a quantitative baseline to measure the
effectiveness of the EPP program, specific
projects, and, to the extent possible, gauge
environmental improvement over time.
This Qualitative Measurement of Environmentally
Preferable Purchasing (EPP) Among Federal
Employees in 2000 is the first phase of the
measurement research. Subsequent quantitative
phases are planned.
The findings from this qualitative market study
served the EPP program well as "grist for the mill."
Generally, the results of this study have helped:
identify the need to create "drivers"for example,
a driver could be to help establish mandates from
top management at more departments and
agencies;
identify how purchasers and requesters can fit
EPP into their acquisition process;
underscore the need to target marketing to bring
about changes in perceptions and behavior
regarding environmental purchasing practices;
and
highlight the need to engage the supply side, or
vendors, in strategies and activities to advance
EPP across the Federal government.
Specifically, study findings, combined with the
learnings from the pilot projects, are informing EPP
program plans for:
2001-2005 strategic plans;
product- or service-specific marketing plans;
general outreach and EPP program awareness
plans; and
inputs to the development of the quantitative
research.
Julie A. Winters, Project Officer
Pollution Prevention Division
U.S. Environmental Protection Agency
Preface iii
-------
-------
Executive Summary
Background
The U.S. Environmental Protection Agency's (EPA's)
Environmentally Preferable Purchasing (EPP)
program is mandated by an Executive Order.
Signed on September 16, 1998, Executive Order
13101, Greening the Government Through Waste
Prevention, Recycling, and Federal Acquisition
supersedes Executive Order 12873, Federal
Acquisition, Recycling, and Waste Prevention, issued
on October 20, 1993. Executive Order 13101 retains
a similar requirement for EPA to develop guidance
to "address environmentally preferable purchasing."
In essence, this Executive Order tasks EPA with
assisting Federal government agencies in
considering the environment in all government
purchases. It is one of several programs that fall
under the "Greening of Government" initiatives.
Over the past several years, EPA has worked with
various agencies to promote the program. The
Agency also developed EPP outreach materials,
including print and electronic media, for distribution
within the Federal government. It was unclear to
EPA the extent to which Federal employees actually
practice EPPi.e., comply with the mandate. To
measure the awareness of EPP among Federal
employees and the extent to which they practice
EPP, and to assess their information needs about
the program, EPA contracted with NuStats of
Austin, Texas, for professional market research
services.
Research Objectives
The goals of the study were to:
define and understand Federal procurement
practices and behaviors,
explore what motivates the Federal workforce to
link the environment to purchasing-related
decision making,
test the recognition and usefulness of the EPP
outreach materials, and
assess the viability of methods for the subsequent
quantitative research phase.
The knowledge gained from this study, along with
future quantitative research findings and
experiences from pilots and interactions with other
Federal agencies, will provide input to the strategic
planning for the EPP program.
Research Design
The chosen methodology was a two-phase study,
beginning with qualitative research. The decision to
use qualitative methods for Phase I was deliberate.
From a research perspective, capturing qualitative
data first made good sense, given the objective of
this part of the researchi.e., to understand
underlying attitudes and behavior related to EPP.
Phase I, the focus of this report, consisted of three
stages: Discovery, Exploratory, and Confirmatory.
Each stage added another layer of insight from the
primary target audience: Federal employees who are
part of the purchasing process within the
government. Study participants included employees
in procurement, contracting, purchasing, and
acquisitions, as well as those who requested
products and services. Interviews with a limited
number of private contractors (a secondary target
audience) added another dimension to the research.
NuStats conducted qualitative research with more
than 130 individuals over the course of several
months, using a combination of one-on-one in-person
and telephone interviews, focus and mini groups,
dyads, and triads. The result was a rich and in-
depth look into the motivations and underlying
factors that drive purchase decisions in the Federal
government.
Learning Questions
The following series of "Learning Questions" guided
the research.
What are the general attitudes, behaviors, and
perceived information needs regarding
purchasing?
What are the attitudes, behaviors, and perceived
information needs in defining environmental
preferability?
Executive Summary v I
-------
What is the general awareness of the "Greening
of Government" initiative?
What are the awareness levels and opinions
about EPP outreach materials?
Are there opportunities to fit EPP into day-to-day
procurement activities?
To answer these questions, NuStats tapped into the
tremendous knowledge base and extensive
background and experience of Federal employees
involved in purchasing.
Key Findings
The study's eight key findings are:
Executive Order provisions on the "Greening of
Government" are not perceived by study
participants to be mandates.
Agency- or department-specific mandates
motivate Federal purchasers more than
government-wide Executive Orders.
Study participants consider different factors
when evaluating products versus services.
Few study participants recognized the term
"environmentally preferable purchasing."
Some Federal purchasers and requesters consider
the environment in a purchase decision.
Study participants believe the responsibility for
doing EPP lies elsewhere.
Federal purchasers and requesters rarely
mention environmental factors as a primary
purchasing consideration.
Awareness of EPA's EPP tools and resources is
low among study participants.
Ivi Executive Summary
-------
Introduction: Project Overview
Executive Order 13101 mandates that the U.S.
Environmental Protection Agency (EPA) assist
Executive agencies in making purchasing decisions
that are less damaging to the environment.1 The
Environmentally Preferable Purchasing (EPP)
program was established to provide guidance and
carry out a variety of initiatives and outreach
activities targeting a wide constituency, including
Federal agencies.
Committed to effectively carrying out its mandate,
EPA set out to measure the degree to which EPP
principles have penetrated Federal government
purchasing practices. The Agency's goals were to
explore the underlying motivators that impede
and/or drive EPP within the Federal government. In
general, EPAs questions were:
Are Federal employees aware of the EPP
program?
Are Federal employees practicing EPP and, if so,
to what extent? If not, why not?
What attitudes exist about EPP?
What motivates people to make EPP decisions?
What type of communication needs do Federal
employees have regarding EPP?
To answer these questions, EPA contracted with a
consultant with expertise in capturing attitudes,
motivations, and behaviors from specialized
populations. In the spring and summer of 2000,
NuStats, an Austin-based market research firm,
initiated Phase I of a two-phase research design that
uses qualitative research (Phase I) and quantitative
research (Phase II) as complementary strategies.
From a research perspective, capturing qualitative
data first made good sense, given the objective of
this part of the researchi.e., to understand
underlying attitudes and behavior related to EPP.
Qualitative analysis draws on an inductive process
in which themes and categories emerge through the
synthesis of data gathered by such techniques as
focus groups, case studies, and one-on-one telephone
or in-person interviews. Samples are usually small,
and often purposively selected.
Qualitative research differs from quantitative
research in that the latter is characterized by the
use of large samples, standardized measures, a
deductive approach, and highly structured interview
instruments. Qualitative research does not
statistically sample representative groups of the
study population. Rather, it draws on the insight of
carefully screened participants who fit a pre-defined
profile. Their answers are valid and extremely
useful in the research process, but are not
statistically representative and cannot be
generalized.
This report summarizes the findings of Phase I, the
qualitative research. It is specifically organized to:
provide an overview of the study objectives,
explain the study methodology, and
present a summary of primary and secondary
research results.
Research Objectives
Exploring the context in which Federal government
employees make environmental considerations in
purchasing decisions requires "forecasting" or
explaining behavior. This research extends beyond
an exploration into the motivations that contribute
to "doing EPP," and also studies motivations that
relate to "wanting to practice it," "knowing how to
practice it," and "knowing why to practice it." By
first studying the awareness levels and attitudes
regarding purchasing and the EPP program, the
research uncovers the driving forces behind and
barriers to acceptance and behavioral change.
The knowledge gained through this research, along
with future quantitative research experiences from
pilots and interactions with Federal agencies, will
provide input into the EPP program's long-term
strategic planning.
9 Signed on September 16, 1998, Executive Order 13101, Greening the Government Through Waste Prevention, Recycling, and
Federal Acquisition supersedes Executive Order 12873, Federal Acquisition, Recycling, and Waste Prevention, issued on October 20,
1993. It retains a similar requirement for EPA to develop guidance to "address environmentally preferable purchasing."
Introduction: Project Overview 1
-------
Specific research objectives include the following:
to define and understand Federal procurement
practices and behaviors,
to explore what would motivate the Federal
workforce to link the environment to purchasing-
related decision making,
to test the recognition and usefulness of the EPP
outreach materials, and
to assess the viability of methods for the
subsequent quantitative research phase.
Research Learning Questions
At the onset of the study, a set of "Learning
Questions" related to purchasing and environmental
decision making were developed to guide the study
(see Table 1). Using a funneling approach, this
particular sequence moves from the general to the
very specific and ensures deep and careful probing to
understand the purchasing beliefs and behaviors of
study participants.
TABLE 1
QUALITATIVE STUDY LEARNING QUESTIONS (LQs)
LQ. A. What are the general attitudes, behaviors, and perceived information needs regarding purchasing?
Identify motivations for making a purchase decision and/or product choice (PD/PC).
Identify the least important factors used in a PD/PC, citing examples.
Identify the most important factors used in a PD/PC, citing examples.
Identify sources that provide the most information for considering the most important factors used in a PD/PC and other necessary
information, citing examples.
LQ. B. What are the attitudes, behaviors, and perceived information needs in defining environmental preferability?
Define "environment" and what is "green."
Identify who should be responsible for defining "environment" and "green."
Identify whether and how environmental and/or green connections are made.
In a PD/PC, identify what environmental factors or attributes were considered, if any, in the last purchase, citing examples.
In a PD/PC, identify if environmental factors or attributes are considered routinely, occasionally, or as a one-time effort only.
If the stated barriers are removed, identify how environmental factors or attributes would be used.
Among environmental factors or attributes that are routinely or occasionally considered, identify the least and most important.
Identify information sources of environmental factors or attributes, citing examples.
LQ. C. What is the general awareness of "Greening of Government?"
Assess the level of awareness of the government movement to incorporate environmental considerations into PD/PC, citing examples.
LQ. D. What are the awareness levels and opinions about EPP outreach materials?
Test the level of awareness of EPP outreach materials.
Determine which outreach materials have been received, and explore the delivery method of the tools.
Explore if the outreach materials are being used/read, and test their utility in supporting an environmental PD/PC.
Identify specific actions taken as a result of using/reading the outreach materials, citing examples.
Identify other existing or needed materials that help make an environmental PD/PC.
Test prototypes and messages.
Determine whether current/planned tools address motivating factors that influence environmental PD/PC.
LQ. E. Are there opportunities to fit EPP into day-to-day procurement activities?
Test the awareness and perception of who is responsible for environmental PD/PC.
Review EPP pilot experiences for the pros and cons related to facilitative EPP efforts.
Review ad hoc EPP experiences for pros and cons.
12 Introduction: Project Overview
-------
Research Design Overview
The study design process involved planning the
overall approach for interviewing participants,
defining the study population, and devising a
systematic protocol for selecting and interviewing
participants. A detailed discussion of the research
design is contained in Appendix A. This section
briefly reviews key components of the research
design: the research approach, the study population,
and the screening and interview protocols.
The Research Approach
In-depth interviews and focus groups are two of the
most common forms of qualitative research. These
interview formats, along with dyad or triad mini
groups, were integrated into a three-stage data
collection approach. The activities of the three-
staged approach (collecting and analyzing data,
developing and modifying theory, and elaborating or
refocusing research questions) occurred
simultaneously during qualitative research, each
influencing all of the others. This qualitative study
provided directional pointers about the attitudes,
behaviors, and perceptions of the study population.
Selecting a multi-stage research approach facilitated
mid-course modifications based on the strengths of
the directional pointers.
First, the Discovery Stage allowed for the
exploration and analysis on a general level at the
beginning of the study and led to a fine-tuning of the
research objectives and understanding of the study
population. Next, research hypotheses emerged
from detailed, systematic observations and
interpretation of behavior during the Exploratory
Stage. Finally, during the Confirmatory Stage, the
research hypotheses were examined and confirmed.
Appendix A contains detailed research plans for each
of these stages.
The Study Population
Properly defining the target population is a crucial
step in research design. If not properly defined, the
information collected may be misleading or may not
be relevant for the research question. From the
onset of the study, it was clear that the study
population would comprise Federal employees
involved in purchasing products or services.
However, this definition is so broad that it could
have included nearly every Federal employee.
NuStats followed a systematic approach to view this
study population in more narrow segments, based
upon their inherent job functions as they relate to
purchasing. Working with EPA, NuStats devised
the following list of purchasing-related tasks:
identify a need for or request products or
services,
write specifications,
influence the selection of products and services,
make purchasing decisions,
conduct life-cycle costing analysis,
purchase products and services, and
conduct post-purchasing follow-up.
Based upon this, NuStats defined two research
populations: "Doers" and "Facilitators."
"Doers" represent Federal employees who originate
or make purchasing decisions and/or coordinate the
purchase of products and services. This audience
includes:
government bankcard holders;
contracting, purchasing, and supply specialists
(primarily the GS-1102 series); and
service and product "requesters" or "specifiers."
These individuals can hold one or more of the
functions included in the list above.
"Facilitators" represent Federal employees who set
policy, share expertise, conduct training, or provide
product and service information. This audience
includes management, training providers, EPP
Pioneers (early initiators of EPP practices), people
with specialized training or expertise (e.g., pollution
prevention, environment, safety, and health, and
life-cycle costing) who provide decision-making
assistance, and vendors. These individuals normally
influence the selection of products and services.
The Screening and Interview Protocols
Two final steps in designing the study were to
design the recruitment, or screening, and interview
protocols. Screening protocols consist of carefully
designed questions to qualify people who represent
the target population to participate in the study.
The screening protocols that were used for this study
are contained in Appendix B.
In this study, all participants met the following
criteria:
Introduction: Project Overview 31
-------
Their job encompassed one or more of the
following purchasing-related functions:
identify a need for or request products or
services,
write specifications,
influence the selection of products and
services,
make purchasing decisions,
conduct life-cycle costing analysis,
purchase products and services, and
conduct post-purchasing follow-up.
They had experience performing that function for
at least six months.
In addition, NuStats monitored the recruitment
process to ensure that a mix of "Doers" and
"Facilitators" and civilian and military agencies was
included in the study.
Interviews were conducted using an interview
protocol, or discussion guide. The study's Learning
Questions, identified in Table 1, led the development
of this guide. Copies of the discussion guides used
throughout this study are contained in Appendix C.
4 Introduction: Project Overview
-------
Secondary Research
In this study, secondary research helped define the
research problem, identify the study population,
finalize the research design, and answer some
research questions. Secondary research included the
following:
A review of existing materials and reports
prepared by EPA in carrying out Executive Order
13101 provided (1) insight on the number and
types of materials produced by EPA and (2)
awareness about previous efforts to measure
compliance with the Executive Order.
A cursory scan of the Internet and on-line
bibliographic databases provided insight on the
penetration level of EPP in Federal agencies and
in readily available non-EPA information.
An exploration of Federal government reports
and materials uncovered insight regarding
potential research approaches (e.g., Federal-
sector purchasing models, target research
audience, focus on purchasing processes or
specific product/service purchases).
For each of these areas, the data collection
methodology used and key observations from the
data are reviewed in the following section. A
detailed bibliography is presented in Appendix D.
Existing EPA Materials and Reports
Methodology: Review of information materials and
reports provided by EPA.
Finding: EPA uses a shotgun approach to outreach.
A review of the EPP program communications plan
(U.S. EPA, no date), indicated a "shotgun" approach
to outreachan approach practiced by many
government programs. Because this approach offers
"something for everyone," it stretches limited
budgets and fails to match appropriate outreach
mechanisms with target audiences.
This study provided insights on the success of this
approach versus a focused outreach effort on a single
audience. It also explored:
the relationships within and among audience
members;
the applicability of multi-audience materials;
the existence of high-impact audience sectors
that should receive intense, focused outreach;
and
whether the materials and outreach meet priority
and immediate needs.
Environmental Scan of
EPP Penetration
Methodology: Internet-based searches and on-line
bibliographic databases.
Finding: There is limited penetration of EPP
materials.
In general, Internet and other on-line searches
reflected a very limited penetration of EPP into
existing literature and articles other than on the
Web sites of EPA, the U.S. General Services
Administration (GSA), the Department of
Defense (DoD), the Office of the Federal
Environmental Executive, and others charged
with carrying out "Greening of Government"
Executive Orders.
Detailed and more complex searches reveal that a
few Federal agencies have developed their own
internal policies and programs related directly to
EPP (e.g., the National Aeronautics and Space
Administration (NASA), the Department of the
Interior (DOT), and DoD).
Often, references to EPP were made in the
context of policies or reports relating to
affirmative procurement.
This research suggests that EPP has not penetrated
deeply into the information network that facilitates
purchasing practices at Federal agencies.
Other Government Resources
Methodology: Collection and review of Federal
government reports and publications.
Finding: Washington, DC, is a primary geographic
area for this study.
Much of NuStats' research focused on the
Washington, DC, metropolitan area, including
Northern Virginia and Maryland, for two primary
Secondary Research 5
-------
reasons. First, the majority of Federal Executive
agencies are headquartered in this geographic area,
and could provide a large pool of respondents from
which to recruit.
Second, a recent GSA Federal Procurement Data
System report (GSA, 1999), revealed that
Washington, DC, has been one of the top 10 areas by
procurement for the last five years (others were
California, Virginia, Texas, Maryland, Florida,
Missouri, Massachusetts, New York, and
Tennessee). The report further identified the top
civilian agencies, DoD facilities, and contractors
within each state. This information supports
NuStats' decision to conduct research with DC-based
Federal employees.
Finding: A variety of resources facilitates target
audience identification and selection.
Several resources provided insight into the
procurement and acquisition workforce and
facilitated identifying, defining, and contacting the
research audience.
Two reports identified the major classifications of
personnel in the procurement and acquisition
workforce (FAI, 1998; U.S. 0PM, 1999). These
resources helped identify "buyer" study
participants.
The ARNET database of Government Training
and Development Resources (OFPP, 2000)
identified key education and training sources for
this workforce. Some of the following resources
were useful in recruiting study participants.
Defense Acquisition University,
Air Force Acquisition Training Office,
Federal Acquisition Institute,
GSAs Trail Boss Professional Development
Program,
Defense Systems Management College,
Navy Acquisition Center of Excellence,
Federal Mediation and Conciliation Service,
National Contract Management Association,
National Association of Purchasing Managers,
Project Management Institute,
Council of Logistics Management,
International Association of Contracts and
Commercial Managers, and
National Institute of Government Purchasing,
Inc.
Nearly all Federal agencies are required to
produce an annual forecast of contracting and
subcontracting opportunities that identifies the
type of procurement or acquisition activities
projected for the fiscal year, along with a contact
person, including phone and/or e-mail address.
These reports were used as a resource for the
study's participant recruitment.2
GSAs Federal Procurement Report (1999)
identifies amounts (volume and expenditures) of
products and services categorized by agency and
contractor. This resource was useful in designing
the research recruitment approach.
2 Forecasts from the following agencies were used in this study: the U.S. Department of Energy, DOI, the U.S. Department of Justice,
the U.S. Department of Labor, the U.S. Department of Education, EPA, GSA, the U.S. Department of Health and Human Services,
and the Veterans Administration (VA). On-line copies of these forecasts can be found at each agency's respective Web site.
6 Secondary Research
-------
Qualitative Research
This section reviews the research conducted by
summarizing the interviews and identifying the
participant mix by role and agency. It also reviews
the study's findings by theme and then by Learning
Question.
Research Overview
NuStats conducted a total of 133 qualitative
interviews using a combination of one-on-one, dyad,
or triad in-person and telephone in-depth interviews,
and mini/focus groups.3 In general, two types of
interviews were conducted: "Full" and "Tools."
"Full" interviews covered all five Learning
Questions. "Tools" tests addressed Learning
Question D and included in-depth focus on the
utility of specific EPP tools. Table 2 summarizes the
types of interviews and audience mix.
Of the people interviewed, most were civilian (63%)
or military (31%) personnel, with the remaining 6%
being non-government entities (e.g., vendors).4 By
purchasing role, most people were buyers (49%),
and of those, 26% were bankcard holders (see
Figure 1).
FIGURE 1
PARTICIPANT PURCHASING ROLES
Vendors
Facilitators
22%
Requestors
23%
Appendix E provides a breakdown of participants by
agency and by interview type.
Key Findings
This research led to the development of eight broad
findings:
Executive Order provisions on the "Greening of
Government" are not perceived by study
participants to be mandates.
Agency- or department-specific mandates
motivate Federal purchasers more than
government-wide Executive Orders.
TABLE 2
OVERALL CUMULATIVE INTERVIEWS
Audience Mix
Buyers
Requesters
Facilitators
Vendors
Total
One-on-One
Tools Full
1 11
0 4
0 3
0 0
1 18
Phone
Tools Full
10 17
1 18
0 12
0 8
11 55
Mini/Focus Group
Tools Full
8 19
3 4
0 14
0 0
11 37
Total
66
30
29
8
133
3 A mix of interview methods contributed to the balance of the depth and breadth in scope of the research findings. One-on-one, dyad
(2 people) or triad (3 people) interviews were typically conducted in the office or on the telephone. Mini focus groups typically
consisted of "in-office" discussions with 4-5 persons with similar study characteristics. Full focus groups represented group interviews
of 6-8 or more people.
4 Of the eight non-government entities interviewed, two represented National Industries for the Blind regional coordinator and Javits-
Wagner-O'Day, three DoD or DOE service contractors, and three industry product vendors.
Qualitative Research 7
-------
Study participants consider different factors
when evaluating products versus services.
Few study participants recognized the term
"environmentally preferable purchasing."
Some Federal purchasers and requesters consider
the environment in a purchase decision.
Study participants believe the responsibility for
doing EPP lies elsewhere.
Federal purchasers and requesters rarely
mention environmental factors as a primary
purchasing consideration.
Awareness of EPA's EPP tools and resources is
low among study participants.
This report presents these findings in two
perspectives: (1) in this section, each finding is
discussed as an independent finding in a broad
context as it relates to Federal purchasing, and (2) in
the following section, the findings are discussed in
the context of the five key Learning Questions that
guided this research.
The following is a discussion of specific study
findings that support each key finding.
Executive Order provisions on the "Greening of
Government" are not perceived by study
participants to be mandates.
A common theme that arose in this study was that
for Federal employees to comply with EPP, they
must perceive it to be mandatedi.e., "You must do
this." Participants agreed that if they are told they
must do something, they will. A contradictory
perception held by many study participants was that
even though the "Greening of Government"
initiatives are mandated by Executive Orders, they
were not perceived as a requirement.
The key to interpreting this finding is to understand
the use and definition of the term "mandate."
Participants applied this term to a wide variety of
items. The following examples of their perceptions
of "mandate" ranged from policy requirements to
regulatory requirements to purchasing routes:
internal policy related to purchasing;
requirement to purchase from GSA or the
Defense Logistics Agency (DLA);
requirement to purchase from Javits-Wagner-
O'Day (JWOD), Federal Prison Industries,
National Industries for the Blind (NIB), National
Industries for the Severely Handicapped (NISH),
the Federal Prison Industries (commonly referred
to as UNICORRN), etc.;
audited activities;
management directives;
direct or implied to be part of agency mission;
specifications for products or services; and
Federal Acquisition Regulations (FARs).
When asked to explain why they did not perceive
Executive Orders to be mandates, they often cited
the lack of enforcement. One participant cited the
recent Executive Order 13149 as being taken more
seriously because it contains an element of
enforcement:
"The most recent Executive Order has 'teeth,' requiring
agency reporting on fuel usage. DOE made sure it had
teeth. Upper management is paying attention to this one
and so are fleet managers who normally don't know about
Executive Orders."
Participants agreed that Federal employees would
practice EPP more frequently and would accept
environmental considerations if EPP were
incorporated into existing mandatesi.e., what they
perceive as being required.
"Most people need to have it mandated. They need
something concrete to follow, like a policy or stated
procedure."
"Mandates make people do it."
In addition, many participants agreed that
enforcement is a necessary component of a successful
EPP program, stating that mandates would be even
more effective if there were consequences of not
practicing EPP. One person cited:
"It would be practiced more if it were more forcefully
mandated. What are the repercussions for not doing it? If
there were a compliance mechanism, more people would
do it."
Some participants shared that they prefer being
"told what to buy," because it makes their job easier.
They believe that researching vendors and products
that comply with EPP would take too long and
would impede the purchase process. Related to this
is the belief by a significant number of buyers that
products purchased from GSA, DLA, or other
mandatory sources (e.g., JWOD, UNICOREN) comply
with Federal environmental purchasing
requirements. One participant put it this way:
8 Qualitative Research
-------
"We are so busy. We are told to buy from certain sources
like the Blind or Skill Kraft, so we assume that they are
complying."
Several participants suggested that a time-saving
approach would be to designate these products with
a logo or symbol "like the ENERGY STAR label" or
"like the seal of Good Housekeeping."
Agency- or department-specific mandates
motivate Federal purchasers more than
government-wide Executive Orders.
An issue imbedded in "mandating" EPP is that
where the mandate, or perceived requirement, comes
from makes all the difference regarding whether it is
practiced. As mentioned previously, some
participants believe that an EPA mandate or an
Executive Order holds little weight with Federal
employees. Many voiced the opinion that the
mandate should come directly from top management
within their own agency. Participants suggested
that the "trickle-down approach" was likely to be
more effective if top managers demonstrated their
support for EPP by encouraging their staffs to
practice it.
"The head of our agency should be the prime contact. We
look higher up in the agency for purchasing information
and policy."
"Contracting officials are aware of the 'green rules,' but
carrying them out falls through the cracks. It is not a
priority of upper management and has never had high-
level agency attention."
Several participants reported that they do not
practice EPP because they do not think that the
rules apply to them or their agency. One civilian
buyer responded:
"A lot of our purchasing is done overseas. The U.S. rules
don't apply to services conducted outside of our country."
A participant representing an agency with an
environmental mission added:
" I've practiced it because it's what we do. I don't think
other departments without this mission take it seriously."
Study participants considered different factors
when evaluating products versus services.
Many participants stated that when evaluating
products or services, they consider a variety of
evaluation factors according to the type of product or
service purchased. Most study participants agreed
that it was difficult to identify one set of factors that
they routinely used.
Even so, cost, along with quality, reputation, and
performance, were frequently cited. Best Value was
frequently mentioned, but more often by DoD
participants. Bankcard holders cited cost almost
exclusively, and purchasers of services placed less
emphasis on cost.
Many participants stated that when evaluating products or
services, they consider a variety of evaluation factors
according to the type of product or service purchased.
There is some indication that the decision-making
process for purchasing products and services is
somewhat different. A few participants mentioned
they consider slightly different factors for products
than for services. For products, these persons listed
a similar set of factors or decision criteria they use
when making a purchase decision, including cost,
reputation, ability of vendor/source to meet the
specifications, and availability/delivery. On the
service sidefor example, when contracting out for
janitorial services, landscaping, and operations
support servicesthat list was modified. Factors
heard repeatedly were past performance, quality of
vendor's work, ability to meet specifications, and
cost.
NuStats observed several trends that drive decision
making regarding the purchase of products and
services.
For products, observed trends include:
using a government-provided bankcard,
especially for office products;
purchasing on-line using GSA Advantage!;
researching products and vendors on-line using
the Internet; and
using government-wide agency contracts for
Information Technology (IT) and other large-
scale products.
For services, observed trends include:
some use of bankcards;
use of teams for development of specifications;
and
Qualitative Research 9
-------
use of government-wide agency contracts for IT
and other large-scale services.
Contrary to this, some participants shared that
whether they are purchasing a product or service
makes little difference. Another person went one
step further to say that selecting factors is more
case-by-case, which requires some judgement.
" It is a business decision, and more often than not it is just
based upon common sense. It is not always the same."
On the other hand, one person stated another
common theme:
" First you consider the cost, and then you apply factors."
Buyers frequently stated that requesters normally
ask for products with which they are familiar and
already have confidence in that product's
performance and/or brand name. One buyer put it
simply:
"The need drives the purchase."
A number of participants (primarily from DoD)
shared that for some support servicese.g.,
technical, contractual, custodialthe contractor is
responsible for making purchases of products and
services. They further shared that contracts rarely,
if ever, stipulated how the contractor should "get the
job done," and that applied to purchasing as well.
Environmental factors were rarely mentioned, if at
all, without probing. When asked whether they
consider environmental factors for service purchases,
several participants commented that they do, but
not as routinely as when purchasing products. One
person suggested that environment should be an
evaluation factor for Best Value. On the other hand,
some perceived that consideration of environmental
factors might be at the expense of Best Value.
Other participants explained that they do not
consider environmental factors because it is arduous
and would take too much time.
" Because of rush purchasing, there's no time to hunt around
for an environmental product."
" I just want someone to tell me which products are
environmentally friendly. I'll buy them."
Many buyers reported using environmental factors
in their contract clauses. Most stated that they
obtained these clauses from the FAR, and a few
provided more specific sources:
Section K: Certifications and Representations
(references to the Clean Water Act and Clean Air
Act),
FAR: Ozone-Depleting Substance clause, and
Agency FAR clauses on recycled paper.
Few study participants recognized the term
"environmentally preferable purchasing/'
There was very little recognition of the term
"environmentally preferable purchasing" among
most study participants, with the exception of some
facilitators (e.g., pioneers, policy developers,
"Greening of Government" workgroups). It was not
unusual for participants to comment that they "may
have read something about it" or report that "a
woman came and talked to us about it once." But
very few interviewees could support these
statements with a reference point regarding the
source from which they heard or read about EPP.
Once the term was defined, buyers easily grasped
the concept, but most stated they knew very little
about the specifics of EPP. Concepts that resonated
strongly with these individuals were those that were
already familiar to them as purchasing vernacular,
including the concept of multiple attributes, "making
tradeoffs" between environmental attributes, and
conducting life-cycle costing. For instance:
"I can relate to the concept of life cycling. This means
taking into account how much it will cost initially and
comparing savings over the long term. I do that anyway."
"The multiple attributes means a lot because of Best Value.
Making evaluation tradeoffs is what we do, anyway."
Several participants thought that the terminology
"environmentally preferable purchasing" was
meaningful, but a little wordy.
"It says a lot, but it's a mouthful."
" Environmentally sound products is easier to say."
Some people preferred a shorter, catchier, easier to
use term:
"Green purchasing is easier."
"Buying green. Something catchy. It has to mean
something to people."
Only a very few participants could not grasp the
term at all. As the following comments indicate,
110 Qualitative Research
-------
these individuals think they need more information
to understand it:
"The term is not okay. I need examples of what they mean,
not just a definition."
" It is too much trouble to understand. It will take a lot of
education to make sure everyone understands it."
The participants who recognized the term cited
the following as the sources from which they had
heard it:
Fed Center, (www.fedcenter.com);
another person (facility manager, pollution
prevention coordinator);
GSA and DLA catalogs;
Material Safety Data Sheets (MSDS);
procurement staff ("They train the program
offices"); and
internal memoranda.
Several participants shared the opinion that EPP
had not really caught on yet, but were convinced
that the EPP awareness level was growing:
" In the Department of Defense, there is more awareness
now than ever before, but there is still a long way to go."
"We are just starting to hear about it... especially in terms
of office supplies."
"With time and training, people will eventually learn it and
they will do it."
Some Federal purchasers and requesters
consider the environment in a purchase decision.
A major benefit to qualitative research is the ability
to probe deeper into thoughts, ideas, or perceptions.
As mentioned in the previous finding, most
participants cited little recognition of the term EPP.
During the interviews, however, NuStats'
researchers were able to explore the types of factors
and decisions that contribute to purchasing
decisions. Two important findings surfaced:
1. In some cases, purchasers were very used to
evaluating products based on multi-attribute
considerations. Although they were not
specifically considering environmental attributes,
a number of interviewees recognized that the
process they use is similar. They noted that they
often have to consider and balance a variety of
(non-environmental) factors and make tradeoffs
among these factors. For example, the purchase
of laptop computers with large memory capacity
equates to increased costs. The only difference
with EPP is that a number of different
environmental factors are considered and
balanced at the same time.
2. In other cases, purchasers were already
considering at least one environmental attribute
in the process of evaluating products but did not
make the connection with the environmental
nature of the attribute (s).
A few participants identified purchases in which
multiple attributes were considered. These
generally included large-system weapons and IT
acquisitions, building construction or renovation,
landscaping, and facilities management.
Most participants reported making more single-
attribute than multiple-attribute considerations. The
types of considerations varied according to the types
of products or services purchased. For instance,
after some probing, respondents often cited
environmental considerations for energy-efficient
office and IT products, such as green lighting or
ENERGY STAR computers; chlorofluorocarbon-free
(CFC-free) products, products that contain recycled
content, low volatile organic compounds (VOCs), or
low toxicity; or fuel-efficient products. For services,
participants primarily mentioned that they included
environmental FAR clauses in contracts or,
depending upon the service, focused on a single
environmental consideration, such as non-toxic
cleaning compounds for cleaning services or
extended maintenance warranties for IT equipment
service contracts.
Most participants reported making more single-attribute
than multiple-attribute considerations. The types of
considerations varied according to the types of products or
services purchased.
Many participants stated that it sometimes does not
make much sense to consider more than one
attribute, while other times, it does. One participant
illustrated this in the following example:
"For a simple, inexpensive purchasefor a single item, for
instanceit might only make sense to consider one
environmental factor. But for large purchasessuch as for
IT systemsmore than one environmental factor might be
considered as part of the overall mix."
Qualitative Research 11
-------
Study participants believe the responsibility for
doing EPP lies elsewhere.
There was a perceived, shared responsibility for
making environmental considerations among
buyers, requesters, specifiers, and vendors. At the
same time, there was also some finger pointing
regarding when the decision making should occur.
Some participants think these considerations
would be best handled at the beginning of the
purchase process (i.e., when the need is identified
or the specifications are written), before it gets to
procurement. One participant (buyer) put it this
way:
" I assume someone else has the responsibility to make
environmental considerations. I don't know enough
about it."
Yet, even those who think the primary responsibility
lies with the initiator, or specifier, support the idea
that people at all points in the purchasing process
should be aware of environmental preferability.
Many participants, regardless of their role in an
agency, voiced the need for a final checkpoint for
making environmental considerations in the
purchasing process. Also regardless of their job
series, most people think that checkpoints are
needed at the contract office level.
The study team noted that this perception was
contradictory to the perception held by a majority of
facilitators regarding who was responsible for
making environmental considerations. Facilitators,
who primarily set policy or promoted initiatives
related to "Greening of Government," shared the
belief that within their agencies, buyers did not hold
this responsibility. In contrast, the research team
observed that buyers shared one or more of the
following opinions: (1) they needed to play a role in
defining the environmental consideration with the
specifier or requester, and/or (2) they should be part
of the "checks and balances" to make sure
environmental considerations are made.
During the study, NuStats observed a growing trend
of forming multidisciplinary teams to develop
purchasing specifications for large-system
acquisitions, especially IT, building construction and
renovation, etc. This was also recently cited as a
trend in Government Executive's special annual
issue (Government Executive, 2000). These teams
include representatives from program offices,
procurement, and environmental health and safety.
Examples of such teams were observed in both
civilian and military agencies.
Federal purchasers and requesters rarely
mention environmental factors as a primary
purchasing consideration.
Very rarely did participants list a specific
environmental concern when building the laundry
list of factors or driving forces behind their purchase
decision making. When probed further, they
realized they consider environmental factors in some
cases, usually depending on the nature of the
product or service needed.
When asked to explain their decision-making
considerations, participants mentioned that
environmental factors come up at various stages in
the purchase process. Buyers, in particular,
incorporate the environmental FAR language into
service contracts when the contract is being
developed. People who write product specifications
were more likely to mention that they make
environmental considerations early in the purchase
process.
Awareness of EPA's EPP tools and resources is
low among study participants.
Very few participants were aware of EPP outreach
materials, regardless of format (i.e., print or
electronic). Most participants had never seen or
heard of EPA or the EPP program tools. NuStats
conducted interviews and focus groups with Federal
employees to gauge awareness levels and to gain
insight into how to (1) improve existing materials
and (2) effectively reach the target audience.
Key Findings Organized by
Learning Questions
Because the Learning Questions developed by EPA
at the outset of this research are an important tool,
it is critical to discuss the study's key findings in the
context of those questions. Doing so illustrates the
complex interrelationship that exists between the
findings and addresses the underlying factors,
attitudes, and behaviors that contributed to their
formulation.
As shown in the Table 3, each of the findings can be
interpreted in the context of at least one Learning
Question.
12 Qualitative Research
-------
CO
LLJ
0
2 8 1 8" g «S
jj 5 ;.= .= -o E $
d - i §: s 1 1
"""§ Is -1 1 =
o
o
0> g _Q _=
-= £ o «> c-
0 £~ = £-5
o c
"i-=
.. O*O g> 0) O
<-» a) a -1 E £
2 - i 1 1 =
'£ 2 ,5 g ;=
o g a o
sf *
* > in
^°3-o _
3 « a) ^ c-
c g_ = f = .-="
cja 0 -o = .= g ^
_B o ^= *
-a
"5 o o
g cl i9,,
^ -3
i ° 1
4 £ §-
CO
Qualitative Research 13
-------
The key findings are presented in the context of the
Learning Questions below.
10. A.: What are the general attitudes,
behaviors, and perceived information needs
regarding purchasing?
The Federal government purchasing system has
become increasingly customer driven in recent
years. Procurement officials and requesters of
products and services alike overwhelmingly report
that the greatest motivator in making a purchase
decision or product choice is customer need. Buyers
indicated that customers identify the need for a
product or service and then make the initial
decisions regarding the attributes or qualities that
meets their needs. They offered that a requester's
"need" is typically driven by the underlying factors,
including the quality, technical capabilities and
performance, cost, and immediate availability of a
product or service. Representative comments
include:
"I need it to last 30 years."
" I need to know that the maintenance crew is always
'on call' and will respond to my calls for service within
an hour."
" I need it now."
Procurement officials and requesters of products and
services alike overwhelmingly report that the greatest
motivator in making a purchase decision or product choice
is customer need.
Buyers reported that, more often than not, they are
not involved in the initial decision making about the
product or service specifications. However,
participants suggested that a team approach to
purchase/service decision making is a growing trend.
Several civilian and military agencies reported the
use of multi-organizational teams to revise existing
or create new specifications for large-scale product
and service purchases. These teams consist of
representatives from the primary program office
(the technical staff), procurement staff, and
depending upon the product or service, other agency
experts who can contribute to the development of
technical specifications or the development of
evaluation factors. Examples of such experts are
cost/benefit analysts or people with life-cycle costing
capabilities, facility managers, engineers, or people
with ancillary technical expertises, such as pollution
prevention or environmental protection.
Participants report that the team approach provides
a sense of "shared responsibility" in the decision-
making process.
Some relevant comments made by participants are:
"For purchasing ADP equipment, computers, and other IT
systems equipment, we follow guidelines that were
developed from our agency's computer group, and a team
evaluates the bids."
"On our base, we established a 'virtual team' that works as
a network to design new facility specifications. Developing
criteria collaboratively builds more buy-in from the field
offices."
"A Product Standardization Group consisting of people
from facility operations, the end users, and procurement
is centralizing specifications for certain products and
services."
Despite the growing trend of a team approach in
purchasing, respondents generally struggled with
identifying who has ultimate responsibility. Buyers
often said they normally have little input on the
development of specifications, but they agreed that
they have input on determining or guiding the
appropriate purchasing route.
Decision-making Factors
Overwhelmingly, buyers commented that their
priority decision-making factor is "mandatory
sources." They perceived that these types of
purchases were "being watched" or monitored.
Examples included:
JWOD,
NIB,
NISH,
UNICORRN,
GSA, and
DLA.
When buying in greater quantity, participants also
stressed that cost, convenience, and the type of
product or service also drive the purchase source
selection. For instance, JWOD, GSA/DLA,
UNICORRN, and NIB/NISH were often mentioned
for products and some services. When the costs for
these products and services are higher, other
purchasing sources were identified, such as
government-wide agency contracts with
114 Qualitative Research
-------
GSA/Federal Supply Service or other agencies for IT
or competitive and open procurements for services.
Another motivator in making product or purchasing
decisions is convenience. Sources perceived to be
"convenient" and "easy" also drive purchase
decisions. Some indicated that procurement officials
are reluctant to change the way they do business,
especially if they have been following a certain
procedure or process for many years. More often,
participants cited that there was "too much on my
plate," and taking the easiest route helped lighten
their workload. For example:
"Staples is convenient and nearby. I go there on my way
home from work or at lunch to pick up supplies."
" People go with whatever purchasing route is easiestthis
means what takes less time, not learning something
newand using commercial vendors they know."
Convenience and ease of use also relate to the use of
bankcards, a purchasing method that, according to
respondents, is growing in popularity among many
Federal agencies.
Whether a participant was a bankcard holder also
made a difference in purchase decision making. The
credit cards offer the option of purchasing products
and services up to a fixed dollar limit, and
purchasing directly through a commercial source
(e.g., Office Depot, Staples, or a service provider).
It is noteworthy that buyers with these cards tended
to report that they would continue to use a
mandatory source prior to using a commercial
source. They cited that these mechanisms were well
known to them. However, cardholders who are not
procurement officials rarely mentioned any
knowledge of this option.
Other factors commonly mentioned include:
Many participants frequently mentioned Best
Value. Within DoD, it was referenced as
essential in "protecting the government's best
interest." In using this factor, one person stated
that "tradeoffs are the key," citing cost, technical
specifications, and performance among the
factors considered. Another participant
suggested that "the best way to get EPP
practiced more widely by procurement officials is
to introduce it as a factor for the Best Value
mix." However, respondents, mostly buyers,
expressed concern over how to make the
environmental tradeoffs and who should make
them.
Performance-based factors were also mentioned.
In using these factors, participants reported
specifying "what we want the contractor to do,
not how they need to do it." DoD contract
officers, who are responsible for major
acquisition or general service contracts,
mentioned this very often.
Local concerns enter service contracts, such as
fleets, landscaping, and new construction.
Related to this were state regulations or
requirements that were associated with
purchasing. One participant shared that during
a landscaping and erosion-control project, the
project team needed to consider local weather
concerns in the selection of its work materials.
The team selected an alternative to plastics for
erosion control that included the use of coconut
fiber mattress and biodegradable staples. For
stabilization of riverside banks, willows and
reeds were used.
Information Sources
Though not mentioned frequently, some respondents
felt electronic commerce (e-commerce) offered an
easy and convenient method for purchasing and for
making purchasing decisions. NuStats observed
little first-hand experience in e-commerce among
respondents. GSA Advantage! was referred to as the
e-commerce site most used by Federal purchasers.
When asked why they like this site, most said it is
easy to understand and use and it allows quick
purchasing. The most frequently cited non-
government e-commerce Web site was
FedCenter.com. On further probing, NuStats found
that none of the participants actually made a
purchase from this site. Reasons included not
having a government bankcard and mistrust, e.g., "I
am not so sure about this e-stuff."
Some respondents felt electronic commerce (e-commerce)
offered an easy and convenient method for purchasing and
for making purchasing decisions.
However, most people were receptive to the Internet
and recognized it as a useful source of information
pertaining to government purchasing. Web sites
Qualitative Research 15
-------
that were most frequently named were government-
sponsored, and include:
ARNET,
Planet GSA,
Small Business Administration's Pro-Net, and
Defense Acquisition Deskbook.
Nearly all the participants also cited their own
agency's Web site pertaining to acquisition.
Participants indicated that they use Internet
resources to obtain or research procurement-related
policy and regulations, to locate vendors, to access
product catalogs, and to obtain pricing/product
information. There is untapped potential for e-
commerce in Federal purchasing.
10.8.: What are the general attitudes,
behaviors, and perceived information needs in
defining environmental preferability?
Participants were asked to define two sets of
concepts:
environment and green, and
environmental purchasing and green products.
Their responses provided insight on how each
concept contributes to their attitudes regarding
environmentally preferable purchasing.
Regardless of their purchasing role, participants
viewed "environment" as a broad concept: "It's
where we live," "clean air," and "clean water."
"Green" was perceived as a concept they applied to
an individual's behavior or an action: "It has to do
with buying or doing something to protect our
environment or health, such as recycling or
purchasing natural products. It's getting back to
basics in how we live."
When probed further, buyers eventually made the
connection with these concepts to the purchasing
environment, which was not surprising, given that
the topic of discussion during the interviews was
Federal purchasing.
Practice of Environmental Purchasing
When asked about "environmental purchasing" and
"green products," buyers viewed these terms
differently. Many interpreted "environmental
purchasing" as incorporating FAR language into
contracts. This type of participant often referenced
FAR clauses related to adherence of the Clean Water
and Clean Air Acts and the requirement to use
recycled-content paper. "Green products" were
reviewed as specific types of products that comply
with social, or culturally sensitive, issues.
When participants were asked whether they practice
environmental purchasing or purchase green
products, their responses varied. Several responded
that they or others in their agency do not make
environmental purchasing considerations because it
has little or no relevance to their agency's mission or
what they purchase.
"It [environment] is not within my agency's mission, so
there is no directive or policy requiring us to practice it."
"I purchase program support services, such as research
studies and technical reports. There's no opportunity for
environmental purchasing."
"I only purchase support and operations contracts for large
defense weapons systems."
"Most agencies probably wouldn't make those
considerations unless they dealt directly with
environmental issues."
Other participants suggested the reason why they,
or their peers, do not make environmental
purchasing decisions is because they don't have the
time or the personal interest to do so.
"Just like with everything else, unless someone has
awareness of an issue, they don't understand how it
impacts them, and they don't do it. It's the same with
making environmental considerations. Unless someone is
aware that it has a personal impact, they probably won't
take an interest in it. They need to know how it will affect
them personally to really care."
"Time is an issue. My time is valuable. I do what I have to
do and I don't go beyond what is required of me."
Finally, a smaller portion of participants readily
agreed that they conduct environmental purchasing.
The underlying motivator to doing so is a personal
interest or a sense of civic duty as a Federal
employee. Several people noted that in their agency
there is a strong advocate of environmental
purchasing and that this person motivates others to
do so.
"Persons who are environmentally conscious are more likely
to do it or want to do it or take responsibility for doing it."
"The environment is my soapbox. I do it at home and at
work. People who care similarly do it. Otherwise, they
don't, and often it isn't because they are not awareit is
because they prefer to stick to their routines."
116 Qualitative Research
-------
NuStats observed that one reason for not readily
mentioning any consideration of environmental
factors during their purchasing decision making
could be because the term "environmental" may not
be defined or understood the same way by every
Federal employee. Participants frequently
interpreted this term according to the
environmental issues their agency considered
important or were concerned about. Environmental
issues that were cited included:
Health and Human Services (HHS) laboratory
and VA facility participants shared interest in
health care-related environmental issues,
including chemicals used in laundry and cleaning
facilities, pest control, reduction of non-regulated
waste, and chemical substitutions, such as
mercury.
Treasury (Bureau of Printing and Engraving)
participants clearly stated concerns about the use
and management of process wastes.
DOE and HHS laboratories shared interest in
solvent substitutes.
GSA fleet managers were interested in the use of
alternative fuels, vehicle mileage and gas
consumption, and re-refined oil.
DOI and nearly all facility managers of other
agencies expressed interest in environmental
health and industrial hygiene issues, including
the presence of lead and asbestos in their
buildings.
The U.S. Air Force and the U.S. Department of
Agriculture's Natural Resource Conservation
Service were interested in bio-based products.
This careful probing revealed that some Federal
employees are practicing EPP, even though they are
not aware of what EPP is.
Obstacles to Environmental Purchasing
Participants shared their opinions about the
obstacles to practicing environmental purchasing.
One perception-based hurdle is associated with the
"myth" that environmental products are "no good."
When asked to provide an explanation, they cited
experiences with recycled paper in printers or
photocopy machines, or refurbished toner cartridges
that do not perform as well as new toner cartridges.
When asked to cite a recent experience, nearly
everyone said their "experience" was based on a
product that was used several years ago (some 10 or
more years ago), or they explained that they did not
have personal experience but rather their knowledge
was based upon "hearsay" from other colleagues.
Early in this study, it was evident that negative
perceptions of environmental products or purchases
were based on rumors or a bad experience from
years ago. Several participants indicated that this
barrier is slowly "going away." One person
explained,
"People who work on my base never ask for environmental
products. This is because they remember the days when
the copier paper jammed the photocopiers or the toner
cartridges stopped working after a day or two. However, if
I buy these products now they will use them without
complaining and really don't say anything."
"Years ago, these products were 'junk.' But now it's
changing."
Lack of technical information and skill was
frequently mentioned as an obstacle, including not
feeling comfortable with making tradeoffs between
environmental factors or not knowing how to craft
environmental specifications. On the other hand,
participants acknowledged that in their normal
purchasing decision making, they are used to
making tradeoffs on other factors. Others observed
that life-cycle costing could easily incorporate
environmental factors. Participants offered the
following suggestions:
"We need guidance that equates making environmental
tradeoffs with the kinds of tradeoffs we make on a daily
basis."
"EPA should designate a technical person or offer a hotline
as a resource we can use to help us make these kinds of
decisions."
"When I need help, I go to our environmental department
for technical advice."
Lack of technical information and skill was frequently
mentioned as an obstacle.
Finally, when asked how barriers to practicing EPP
could be removed, respondents often suggested
providing more information on practicing EPP.
Responses included:
Educate people in niche areas (e.g., specification
writers, facility managers, environmental health
and safety officers).
Qualitative Research 171
-------
Provide information on making tradeoffs between
environmental factors, or establish Best Value
environmental factors.
Assign an EPA contact for each agency to provide
technical assistance, training, and general EPP
promotion.
Create more case studies (e.g., IT, major
acquisitions).
Information Sources on
Environmental Purchasing
The study identified that participants' information
sources on environmental purchasing are not varied.
For instance, buyers and regulators often cited their
procurement office as a credible source for
information. Buyers of services most often cited the
FAR, while buyers and bankcard holders alike deal
directly with vendors. NuStats observed that
participants would frequently seek out information
within their own agencies. Several people
referenced their internal procurement Web sites,
e-mail messaging systems, bulletin boards, and word
of mouth as their primary means of information
transfer.
Participants frequently seek out information within their own
agencies.
10. C: What is the general awareness of the
"Greening of Government" initiative?
In general, "Greening of Government" has low
prominence and relevance in comparison to other
purchasing-related priorities. By far, participants
who acknowledged awareness of the initiative were
only vaguely aware of it, as the following comments
illustrate:
" It is the environmental way to purchase things."
"The public is watching what the government is doing for
the environmentwe are setting an example."
Only a very few could cite specific Executive Orders
by name without promptingthose who could were
primarily the facilitators. Others familiar with the
initiative were buyers who have participated on
inter-agency workgroups related to the EPP
initiatives or those who purchase one specific
product or service that is a focus of an Executive
Order (e.g., computersENERGY STAR; paper
recycled content; fleetsalternative-fueled vehicles).
Awareness was more likely to be cited for the
following:
FAR environmental amendments, and
Executive Orders 12873, 13123, and 13101.
Many participants thought that their peers were not
complying with these requirements. One person
guessed that only 25% of their purchases are in
compliance. When asked to explain why, the lack of
enforcement was the most commonly cited reason.
Participants offered:
"No one does [comply] routinelyand there isn't anyone
checking to see if we are in compliance, either."
"Someone needs to monitor progress, and we need
accountability mechanisms. Upper management needs to
support these checks and balances."
Several people felt that they needed stronger
leadership from their upper management in making
this more of a priority in their agency. They also
shared the opinion that there is too much
information to process, given the many Executive
Orders and policy letters related to the
environment.
10. D.: What are the awareness levels and
opinions about EPP outreach materials?
Both written and Internet-based EPP outreach
materials were tested during this study. Regardless
of the format of the material, awareness levels were
generally very low among study participants.
Moderators observed that participants from agencies
with management that supports "Greening of
Government" initiatives or with an agency mission
closely associated with environmental issues were
aware of some materials. Participants most
frequently cited EPP Update as a familiar tool.
Once they were made aware of the variety and
range of materials, participants were generally
surprised that EPA has so much information
available. Though the majority of participants are
not familiar with the materials, they questioned the
utility of the materials. Participants were asked to
comment on the format and types of materials they
preferred. There were strong preferences for
materials that save time, are easy to use, and
pertain directly to the types of products or services
that they buy. One person shared,
" I don't waste my time on information materials that guide
my product buying because it takes too much time. If I
have a question, I prefer to ask someone."
118 Qualitative Research
-------
The following list details the types of materials they
prefer:
step-by-step descriptions (e.g., the "At a Glance"
sections of EPA's Promising Practices Guide
were preferred over detailed descriptions;
checklists;
product lists that include cost and performance,
and identify vendors; and
vendor lists with contact information.
Comments on EPP Outreach Materials
Specific reactions to written materials include the
following comments:
EPA's EPP Update (newsletter):
" Include hole punches so that it can be filed in a three-ring
binder."
" It is eye-catching with nice graphics and easy-to-read text."
" It would be a useful resourceeasy to file and easy to
use."
Proposed guidance brochure:
" The length is good."
"The summary is better than a detailed guidance. Just tell
me what I need to know."
Case studies:
" I'll only use these if the case study corresponds to my
purchases."
"State case studies don't have much relevance to how I
purchase within the Federal government. Provide more
Federal government case studies."
" It's too hard to transfer others lessons to exactly the
circumstances I deal with."
"They are too long. Cut to the chase."
"The case studies helped explain the process, but I wouldn't
have read them or even opened them unless they dealt
with the types of services I purchase."
Catalogs (GSA and DLA Environmental Products
Guides, U.S. Postal Service (USPS) Guidance,
Green Seal documents):
"The USPS format works for me. I like the color codes [red,
yellow, and green] because they are easy to understand
and they just make sense."
" I like the DLA's vendor information. This is exactly what I
need."
"The GSA catalog looks like an Office Depot catalog."
"The Green Seal documents are too wordy."
When asked how they prefer to receive materials,
participants cited the following:
Networking via task groups (citing the National
Academy of Sciences Federal Facility Council, the
DoD Joint Committee on Environmental
Attributes, various internal agency roundtables,
the American Hospital Association EPP Task
Group, and various regional Federal Facility
Roundtables).
Conferences/workshops/training.
Internal agency information channels, including
electronic routing of materials and e-mail
announcements, and key procurement contacts.
Guest speakers on procurement issues at internal
agency meetings and workshops.
Trade publications/journals, including:
Federal Times,
Federal Employees News Digest,
Government Executive,
Federal Computer Weekly,
Public Purchaser,
Clean Cities newsletters,
National Contract Management Association
and other profession association magazines,
and
GSA newsletter.
Web sitesthe following were mentioned
multiple times:
GSA Advantage!,
Planet GSA,
ARNET,
internal sites (DOI's stood out),
National Institute of Standards and
Technology; Building for Environmental and
Economic Sustainability,
PRO ACT,
Fedcenter.com, and
vendor sites.
Comments on EPP Electronic Tools
A second objective of this Learning Question was to
collect feedback on EPP electronic tools.
Participants were asked to review a set of tools
before a pre-arranged one-on-one or group interview.
Qualitative Research 19
-------
Awareness of the Web site was very low, as most
interviewees had never heard of or been to the site
before we contacted them.
Only a handful of them actually went through the
site prior to the interview. In terms of the other
study goals outlined earlier (e.g., explore how
materials are used in the decision-making process,
identify other materials needed), we learned that
most Federal employees would not use the Web site
frequently to make purchase decisions, unless the
site was tailored to meet their needsfor example,
having product and vendor lists with compliance
information, showing price comparisons, and
allowing the opportunity to quickly find an answer.
Initial reactions to the Web site were positive and
helpful:
" This is great stuff."
"The links are very helpfula link to the GSA and DLA
schedules would be great."
"IncludetheJWOD.com link."
"This is good. Almost all the tools need up-front, visible
text on who the users of the tools are and how the tools
will help them."
"All contract officers should use consistent contract language.
A Web site like this, containing contract language, would
make this possible."
Many helpful comments were offered. One theme
heard over and over in both the information and the
materials interviews was to make it easy, keep it
simplethe procurement process is complicated
enough already. Several interviewees also suggested
that more training on EPP is needed.
Some comments were conflicting regarding the
value and benefit of these materials, reflecting that
they have utility for some Federal purchasing
decision makers:
"Specification writers would only use these tools if forced to."
"These materials are really useful for specification writers."
" Unless the tools deal directly with what I buy [research
Many people commented that the Web site should have lots of
links and references, and be a resource that Federal
employees could turn to for help with contract language,
writing specifications, and making purchase decisions.
services], I have no need for them and won't use them."
"The case studies are very helpful. I like learning from
others' experiences."
Additional comments included:
"These tools are not for the 'average Joe.' This is complex
stuffthe normal GS-7s and below won't be able to use
them and wouldn't know what to do with them. Purchase
card holders can't use these tools either."
"I like the 'AskJeeves.com' Web site. This one should mimic
a similar question-and-answer approach."
"Model solicitations with environmental-related contract
language embedded in them would help me make
decisions."
Participants expressed a need for specific
information that they could not readily find with the
tools. This includes the following: contract
language, instructions on how to write
specifications, or sample specifications that they can
"cut and paste." When shown these materials in
several of the Internet tools, several participants
recommended that they be highlighted as a separate
tool or button. Some thought that EPA should do a
better job of describing up front what the tools were
and how to use them. They wanted tools specific to
their needs. When asked how these Internet tools
could be improved, their recommendations included:
a more comprehensive database of products, along
with vendors who comply with EPP; more relevant
case studies; links to mandatory sources and to GSA;
and tools similar to the "cleaning wizard" that relate
to the products and services they commonly
purchase.
In general, electronic tools seem preferable to
printed ones, though that was not true for every
single person interviewed. Many people commented
that the Web site should have lots of links and
references, and should be a resource that Federal
employees could turn to for help with contract
language, writing specifications, and making
purchase decisions.
Post-interview Evaluation on
EPP Electronic Tools
A post-interview was conducted with these
participants one month following the initial
interview to ascertain whether they accessed any of
the Web tools once their awareness levels about
them were raised. Of the 13 whom were reached, all
120 Qualitative Research
-------
remembered participating in the initial interviews
and recalled the EPP Internet tools. Nine of them
had not revisited the Web site, four had, and five
referred the tools to other people, including an
environmental specialist, "people on my purchasing
team," and contracting staff.
The following summarizes their responses:
Nine people had not revisited the Web site.
Most said that they had just been too busy
dealing the end of the fiscal year since the
interview.
One person stated that he thought "the tools
were just too difficult to use."
Another observed that "the tools did not
provide me with relevant information."
One person said that his agency did not have
to comply with EPA standards, since it just
purchases research and studies.
Four people had revisited the site.
One person was curious as to whether the
EPP Web site could be accessed through
ARNET. This respondent did not use the site,
but was pleased to learn that the site was
accessible through ARNET, and
"bookmarked" the site for future use.
Another person met with her "systems
person" to check that all of the tools could be
accessed without "glitches." She reported
that the Web site worked fine on her
computer, and added that she has not used
any of the tools since the interview because
she has been too busy.
Another person had accessed all the tools,
except the Tips on Greening Conferences,
citing the success stories to be the most useful
to date.
A focus group participant tried accessing the
site to see if she could do it on her own. She
did not have time to spend on it, and stated
that she did not easily see the relevance and
utility of the tools. She suggested that, for
each tool, the Web site should identify the
target audience and describe why or how it
will assist that audience member.
Three persons thought they would visit the site
after the fiscal year rush ended. One person said
that he would use the tools to "develop
purchasing policy," and two thought they would
use them to identify or locate vendors.
10. £.: Are there opportunities to Fit EPP into
day-to-day procurement activities?
To fit EPP into daily procurement activities,
participants shared the belief that they must be
mandated to do so by their agency management.
They further agreed that EPP must be easy to do.
In general, participants prefer to be told what to buy
as it relates to environmental purchasing.
NuStats observed that understanding how the
purchasing process differs between products and
services will help EPA understand how best to
target the right employees. Decisions about
products may lead employees to consider a different
set of factors than for what they select for services.
How those factors vary can make a difference in how
EPP is communicated.
Understanding how the purchase process differs between
products and services will help EPA understand how best to
target the right employees.
Awareness of EPP and the idea that different people
share the responsibility for making environmental
considerations also need to be taken into account. If
employeesand more specifically top
managementat an agency are not aware of EPP
in terms of language and identitythey are not very
likely to practice it.
Also, the job function of the participant seems to
influence the perception of responsibility. Although
some participants (mostly buyers) stated "it's not my
job," most acknowledged that several people were
actually responsible for environmental
considerations during the purchase process.
Participants provided concrete opportunities for
incorporating EPP into their day-to-day activities:
pollution prevention plans;
waste reduction performance measures;
master specifications (e.g., for construction,
renovation, IT systems);
comprehensive opportunity assessments5; and
FAR boilerplate language (Sections 15 and 52).
Others shared the following recommendations:
"I see a role for EPP in the purchase of computers."
Qualitative Research 21
-------
" If EPA produces any lists for products or vendors, use the
GSA coding scheme. We are used to the GSA stock
numbers, and vendors use them, too."
"Just encourage a "Seal of Good Housekeeping" type of
program so that we won't have to change how we do
business too much. It would be easier to look for a symbol
or sign that designates EPP products or services."
"Integrate EPP into 'Best Values'Look at 52.212.2."
"See Part 15.101the commercial clause on evaluation
factorsthis is a good spot for environmental factors.
Provide guidance on how to make tradeoffs on
environmental factors."
5 Cited by a participant from DOE's Lawrence Livermore National Laboratory as a component of pollution prevention plans to reduce
toxicity and hazardous waste generation in the facility.
22 Qualitative Research
-------
Peer Review
Background
In past EPA studies, such as the 1999 Consumer
Labeling Initiative Phase II Report, peer reviewers'
comments have been informative and helpful. Given
the potential impact of the recommendations
deriving from the qualitative measurement report,
the EPA wanted to determine whether we had gone
about our research appropriately and whether
independent researchers believed the
recommendations were supported by the research.
With these goals in mind, a peer review of the
Qualitative Measurement of Environmentally
Preferable Purchasing (EPP) Among Federal
Employees in 2000 was undertaken. We hope that
these collective efforts will further compliance with
Executive Order 13101 that mandates
environmentally preferable purchasing in Federal
agencies.
Document Reviewed
The document reviewed was the Qualitative
Measurement of Environmentally Preferable
Purchasing (EPP) Among Federal Employees in
2000. The document contained the following major
sections: 1) Executive Summary; 2) Project
Overview; 3) Secondary Research; 4) Qualitative
Research; and several appendices supporting the
research. Appendices included: A) Research Design;
B) Screening Protocols; C) Interview Protocols; D)
Secondary Research Bibliography; and E) Interview
Tracking. The document was created to report on a
research study conducted by NuStats Research and
Consulting under contract with the Pollution
Prevention Division located within the Office of
Pollution Prevention and Toxics.
Peer Reviewers
The peer review was conducted by six independent
reviewers not employed by EPA and one employed
by EPAs Environmental Monitoring for Public
Access and Community Tracking (EMPACT)
Program. Reviewers were selected based on their
expertise or experience in environmentally
preferable purchasing, consumer research and
testing, or survey methods.
Reviewers included Dr. Charlotte A. Cottrill, Senior
Science Advisor, EMPACT Program, EPA, Office of
Environmental Information (OEI), Office of
Technology, Operations and Planning (OTOP); Eric
Friedman, Environmental Purchasing Coordinator,
Massachusetts Operational Services Division;
Pamela Gallion, Survey Manager, Cannon Center for
Survey Research, University of Nevada, Las Vegas;
Robert Guillemin, former Environmental Purchasing
Coordinator, Connecticut Department of
Administrative Services; Lara Sutherland, Senior
Research Associate, INFORM, Inc.; Dr. Robert
Thomas, Professor of Marketing, McDonough School
of Business, Georgetown University; and Shapard
Wolf, Director, Survey Research Laboratory,
Department of Sociology, Arizona State University.
Charge to Reviewers
Reviewers were asked to respond to questions in five
categories: Secondary Research, Study Design, Study
Implementation, Study Results and
Recommendations, and General Comments. For
each of these categories, discussed below, the specific
review questions precede the summary of reviewers'
responses. The questions addressed topics such as
information sources, survey methodology,
participant recruitment, and the congruence of the
findings with the research.
Summary of Reviewers' Comments
Secondary Research
Did the secondary research address the major sources
of information and cover the appropriate issues to
help define the research methodology and inform the
key learning questions?
Most reviewers agreed that the secondary research
covered relevant sources of information. A few
reviewers made specific comments on secondary
research. One reviewer suggested additionally
surveying commonly read purchasing publications
such as The Public Purchaser to see the general
exposure of government employees to
environmentally preferable purchasing in their
routine readings. Another commented on the
apparent focus on government information without
Peer Review 23
-------
adequate inclusion of academic studies on consumers
and organizational behavior. One reviewer pointed
out that such a report usually describes the
contractor process and role in the introductory
section. In this case, EPA's relationship with
NuStats should be defined.
Study Design
Was an appropriate methodology, in particular for
both the screening and the survey, chosen to conduct
this study so that the key learning objectives were
effectively addressed?
Most reviewers felt the methodology was appropriate
and one especially liked the multi-phase approach.
Several reviewers commented on the participant
pool and recruitment efforts. A couple of reviewers
noted that despite the small pool of participants, the
additional one-on-one phone interviews effectively
complemented the research. One reviewer, however,
felt that more creative incentives could have been
explored, such as offering to give money to a
designated charity. This reviewer also thought the
study as a whole did not consider the effect of the
"organization," instead disproportionately focusing
on the role of individuals. As for the "doers" and
"facilitators" distinction, one reviewer felt the
distinction should be more clearly defined, while
another reviewer felt "doers" were under represented
in the Discovery Stage and "facilitators" under
represented in the Exploratory Stage.
Study Implementation
Were the recruitment screeners appropriate to acquire
the type of participants needed to conduct this study?
Were the questions asked in the discussion guides
appropriate and/or sufficient to acquire the
necessary participant responses to address the key
learning objectives of the study?
All reviewers seemed satisfied with the study
implementation, except for concerns previously
raised in the design section, such as the lack of
attention to the "organization." One reviewer
commented that the selection of example responses
and quotations effectively illustrate findings, and
authors quantify responses appropriately.
Study Results and Recommendations
Does the research support the findings? Do the report
findings adequately and effectively address the key
learning objectives of the study?
Most felt the research supported the findings. A
couple felt the findings were somewhat obvious for
those exposed to environmentally preferable
purchasing on a daily basis. One reviewer pointed
out that this study helps to fulfill the need for
comprehensive and systematic documentation of
environmentally preferable purchasing trends. This
reviewer said a statement explaining that certain
findings are obvious to specialists, but are needed for
documentation, would make the report more
valuable to those exposed to environmentally
preferable purchasing on a daily basis.
A majority of reviewers commented on the scope of
the recommendation section, noting that they
expected the recommendations to be exploratory and
not conclusive. Based on the learning objectives,
they expected the recommendations of this
qualitative study to clearly direct research for the
quantitative phase of research, and felt the study fell
short of this objective. Some specific suggestions
were asking interviewees how willing they would be
to participate in quantitative research, and
presenting hypotheses for further qualitative and
large-scale quantitative study. One reviewer felt the
key findings should be written to be more
independently informative because some readers will
skip directly to this section.
General Comments
Please provide further remarks if there are any
additional areas you would like to address or
comments you would like to include.
In the general comments section, most felt the report
was informative and conducted well. One reviewer
commented that it was clearly-written and enjoyable
for a reader with little familiarity with
environmentally preferable purchasing. Another
reader noted that suggestions for follow-up research
should be clearly written so that readers do not
consider the information presented conclusive.
24 Peer Review
-------
Appendix A: Research Design
Research Approach
This study followed a three-stage research approach.
This allowed for exploration and analysis on a
general level at the beginning of the study, leading
to a fine-tuning of the research objectives and
understanding of the study population. Next,
research hypotheses emerged from detailed,
systematic observations and interpretation of
behavior, which were then explored and confirmed
in the final stage.
In a qualitative study, the activities of collecting and
analyzing data, developing and modifying theories,
and elaborating on or refocusing the research
questions are usually happening simultaneously,
each influencing all of the others. As was the case in
this study, the researcher often may need to
reconsider or modify any design decision during the
study in response to new developments or changes in
some other aspect of its design. The three-stage
research design allowed mid-course modifications,
contributing to seamless transitions from one
research stage to another.
The three research stages are briefly summarized
below. Details on each stage are provided at the end
of this appendix.
Discovery Stage (December 1999 - March 2000)
Objectives: Explore underlying motivations,
barriers, and behaviors relating to Federal
purchasing, in general, and identify study
participants for subsequent research stages.
Methodology1: Two telephone focus groups (11
people), 9 one-on-one telephone interviews, and 5
one-on-one and 1 dyad in-person interviews.
Outcomes: Led to the identification of the target
research audience and systematic documentation
of general purchasing practices.
levels and responses to EPP outreach tools (e.g.,
written materials, Internet-based); and develop
research hypotheses.
Methodology: Two focus groups (11 people), 21
one-on-one telephone interviews, and 3 dyad
interviews.
Outcomes: Led to the development of core
research hypotheses and facilitated the
development of a focused test of Internet-based
EPP tools.
Confirmatory Stage (June 2000 - Late July 2000)
Objectives: Hone-in on actionable steps, fine-
tune research learnings, and fill research gaps.
Methodology: Two focus groups (10 persons), 51
telephone or in-person interviews, 1 dyad and 1
triad interview.
Outcomes: Led to a series of action step
recommendations and clarification of a
quantitative research approach.
Research Issues
A number of complexities, inherent in conducting
research with Federal procurement officials and
employees, arose in the early stage of the research,
which influenced the research approach in several
ways. NuStats encountered challenges in the
following areas:
selecting research approaches to maximize
participation,
crafting recruitment strategies for securing
participation, and
conducting "comfort zone" screening and
interviewing to achieve full and open disclosure.
This section of the report discusses these challenges
and identifies the actions taken to overcome them in
the research design.
Exploratory Stage (March 2000 - Early May 2000) Selecting Research Approaches
Objectives: Observe and record EPP awareness
levels, attitudes, and behaviors and awareness
The original research design emphasized the use of
traditional focus groups in conjunction with one-on-
1 Telephone and in-person interviews were conducted for an average of 45 minutes. The average focus group length was an hour and
a half.
Appendix A: Research Design 25
-------
one, dyad, and mini-group interviews, all of which
typically involve a financial incentive for
participants. However, during the Discovery Stage,
the study team learned that giving monetary
incentives to participants was not allowed.2 In
qualitative research, such incentives are necessary
as compensation for the time and effort of the
participants. Without the capability to provide
participant incentives, it was difficult to recruit
participants for the focus groups.
The study team shifted from conducting evening
focus groups to seeking opportunities to conduct
focus groups during business hours. Additional,
unexpected hurdles arose. Purchasing personnel,
especially contract officers and purchasing
specialists, have little flexibility in their work
schedules, as they work under extremely tight
timelines and provide customer service throughout
the day. As a result, it was nearly impossible to
schedule focus group sessions at a time that was
convenient for participants. The study team
learned, however, that the study population could
easily participate in pre-scheduled or on-the-spot
interviews. Therefore, during the Exploratory
Stage, the research design successfully shifted to
one-on-one interviews during office hours.
Additionally, a small number of focus groups and
mini-groups were also held during the study at
conferences and in other venues.
Several one-on-one telephone interviews led to
arranging some focus group and mini-group
interviews during the Confirmatory Stage.
These were generally held at the participant's
place of employment.
NuStats arranged four focus groups through
"piggy-backing" onto meetings of existing
groups, such as the Office of Management and
Budget's (OMB's) Front-line Forum and the
Office of the Federal Environmental Executive's
Executive Order Interagency Group. NuStats
conducted two focus groups at OMB, and two via
a telephone conference call.
Crafting the Recruitment Strategies
Another set of challenges arose in designing a cost-
effective and practical recruitment strategy.
Identifying qualified participants with specific job
functions, such as contract officials and bankcard
holders, required the use of existing lists and
directories (e.g., acquisition forecasts, bankcard
holder lists), intercept recruitment at conferences,3
and "snowball" or referral recruitment.
The use of existing lists was a highly successful yet
time-consuming recruitment method. The lists
often contained out-of-date or inaccurate
information, resulting in additional attempts to
reach and identity qualified participants.
During the Exploratory and Confirmatory Stages,
some participants were recruited through referrals
or the snowball sample. One drawback of using
referrals is the possibility of creating a convenience
sample, where participants are selected because they
are easy to recruit. To prevent this, the screening
protocol ensured that only those people who met
certain criteria qualified to participate in the study.
Another recruitment challenge was contacting
participants by telephone during office hours at
their place of business. Recruiting and screening
during business hours typically take longer than
during evening hours due to voice mail, "cat and
mouse" callbacks, and people not being at their
desks due to meetings, vacations, training, etc. The
addition of two NuStats recruiters resolved this
issue.
Conducting "Comfort Zone" Recruitment and
Screening Interviews
A final challenge presented itself through the
inherent reticence and non-disclosing nature of one
portion of the target study population: the
procurement official. The nature of procurement
work leads to a protective "firewall," where
purchasing officials may be guarded and protective
of their opinions and views. Because of this, some
participants were initially reluctant to participate in
the study. To address this reluctance, NuStats
edited the screening protocol introduction to (1)
disclose the sponsor of the study, (2) emphasize the
independent nature of NuStats, and (3) ensure
participant anonymity in the study report.
Following this modification, recruitment improved
dramatically.
2 As is common with public organizations, policies and procedures prohibited the use of contract funding to provide a monetary or
other incentive to pay Federal employee participants in focus groups.
3 Intercept recruitment and screening were conducted during the National Association of Contract Manager's annual conference in
April 2000 and during the GSA EXPO in May 2000.
126 Appendix A: Research Design
-------
Sampling Plan
The study goal was to gain an understanding of the
EPP program's target audience: employees within
the Federal government who make product and
service purchasing decisions. This required a
sampling plan that:
identified the study population (the "sample
universe"),
outlined an approach for contacting participants
(the "sample"), and
set criteria for selecting and qualifying
participants (the "screener").
The Sample Universe
For the purposes of this research, NuStats defined
two research populations that comprise the sample
universe: "Doers" and "Facilitators," based on their
inherent job functions as they relate to purchasing
within the Federal government.
"Doers" represent Federal employees who make or
coordinate the purchase of products and services.
This audience includes:
government bankcard holders;
contracting, purchasing, and supply
specialists/managers (primarily the GS-1102
series); and
service and product "requesters" or "specifiers."
"Facilitators" represent Federal employees who set
policies, share expertise, conduct training, or
provide product and service information. This
audience includes management, training providers,
"EPP Pioneers,4" product and service suppliers, and
people with specialized training or expertise (e.g.,
pollution prevention, environment and health, life-
cycle costing) who provide decision-making
assistance.
Table 4 provides an overview of the study
population and the roles and responsibilities as
related to purchasing.
The Sample
Identifying and recruiting willing participants who
represent the relevant segments of a study
population are necessary parts of research. To
facilitate this process, the study team identified
several strategies for obtaining target population
contact information.
Buyers were recruited using "intercept (on-the-
spot) recruitment" at two procurement-related
conferences: the National Contract Management
Association's (NCMA's) annual conference in
Washington, DC, and the General Services
Administration's (GSA's) Expo 2000 conference
in San Diego, CA
Buyers, requesters, and vendors were identified
through Fiscal Year 1999-2000 acquisition
forecasts, which are commonly available on-line
from each agency's respective Web sites.
Bankcard holders were identified using
bankcard-holder lists, which are commonly made
available on-line on agency Web sites.
TABLE 4
STUDY POPULATION
Doers
Facilitators
Requesters/Specifiers (Program staff)
Buyers (Contracting officials)
Government bankcard holders
Roles in Purchasing:
Channel needs and preferences
Purchase products and services
Make purchasing decisions
Influence product and service selection
Write guide and product specifications
Managers
Pioneers
Training organizations
Environmental/Life-cycle cost analysis experts
Service and product suppliers or vendors
Roles in Purchasing:
Set policy
Set examples
Provide relevant information
Share product information
4 "EPP Pioneers" are individuals who were early initiators of the practice of EPP. Many of their efforts are documented in EPP pilots
and case study documents.
Appendix A: Research Design 27
-------
Facilitators were identified through the following
procurement-related workgroups and/or contact
lists:
Office of the Federal Environmental
Executive: Executive Order Interagency
Group (EOIAG);
Office of Federal Procurement: Contact Lists
for the Front Line Forum, Agency
Procurement Executives, and Agency Senior
Officials for Executive Order 13123; and
Department of Defense, Defense Logistics
Agency, Joint Group - Environmental
Attributes Steering Committee.
Intercept recruiting at procurement-related
conferences provided contact information for
some requesters and facilitators.
Screening Protocol
NuStats developed a screening protocol that asked
pertinent questions to qualify participants for the
study. Since measuring general awareness and
attitudinal levels of EPP among decision makers was
a key research objective, it was critical that
participants be responsible for a wide variety of
purchasing-related tasks. It was also necessary to
recruit people with a range of skills, experience, and
viewpoints within the context of Federal purchasing.
Therefore, recruitment sought out long-time and
newer employees; those with backgrounds in
procurement, contracting, and major acquisitions;
and those who worked at civilian and military
agencies.
All participants met the following criteria:
Their job encompassed one or more of the
following purchasing-related functions:
identify a need for or request product/services,
write specifications,
influence the selection of products or services,
make purchasing decisions,
conduct "pre-purchasing evaluations" or life-
cycle costing analysis,
identify vendors or sources,
purchase products and services, and/or
conduct post-purchasing follow-up.
They had worked in that position for at least six
months.
They represented a mix of "Doers" and
"Facilitators."
They represented a mix of civilian and military
agencies.
In general, the recruitment call process used the
following steps:
identify willingness to participate in the study,
screen for eligibility, and
arrange time and date for interview or conduct
interview "on the spot" if they are willing and
available.
The recruitment process was monitored to ensure
diversity of participants. The recruitment screener
questionnaire is contained in Appendix B.
Interview Protocol
Interviews were conducted using an interview
protocol, or discussion/moderator's guide. The
study's learning questions led the development of
this guide. The questions were carefully selected and
phrased to elicit the maximum amount of
information during each interview. The guide is
designed to generate open-ended conversation based
on the respondent's awareness levels, attitudes
about, and experiences regarding purchasing in
general, the "Greening of Government," and EPP.
The discussion guide was refined throughout the
course of the research, enabling NuStats to build
upon previous learnings or to hone in on potential
gaps. Copies of the discussion guides for each
research stage appear in Appendix C.
128 Appendix A: Research Design
-------
DISCOVERY STAGE
(DECEMBER 1999-MARCH 2000)
Research Objectives
Explore underlying motivations, barriers, and behaviors relating to Federal purchasing, in general.
Identify study participants for subsequent research stages.
Learning Question Area
1. General attitudes, behavior, and perceived information needs regarding purchasing.
Expected Outcomes Leading to Actionable Steps
Collect input to development of EPP strategy and policy.
Identify target outreach audience.
Research Plan
Study Population
Facilitators (EPP Staff)
Facilitators (EOIAG, Pioneers,
and Training Facilities)
Buyers
Total Participants
Date
12/12/99
12/13/99
12/13/99
1/24/00
2/7/00
2/15/00
3/8/00
3/15/00
3/18/00
3/20/00
3/21/00
3/24/00
Research Tactics Conducted
(All research conducted in Washington, DC)
One-on-One Dyads Conference Call One-on-One
In-Person Group Telephone
4
2 11 8
1 1
5 2 11 9
Research Logistics
Tactics
One-on-one
One-on-one
interview
interview
Dyad interview
One-on-one
One-on-one
interview
interview
Group interview
One-on-one
One-on-one
One-on-one
One-on-one
One-on-one
One-on-one
interview
interview
interview
interview
interview
interview
Study Pop.
EPP Staff
EPP Staff/Doers
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Facilitator
Environment
In-person interview
In-person interview
In-person interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Telephone interview
Other
Appendix A: Research Design 29
-------
EXPLORATORY STAGE
(MARCH 2000-MiD-MAY 2000)
Research Objectives
Observe greening and EPP behaviors and attitudes in depth.
Test EPP tools.
Develop research hypotheses.
Collect input for quantitative research.
Learning Question Areas
1. Attitude, behavior, and perceived information needs in defining environmental preferability.
2. General awareness of "Greening of Government."
3. General awareness of and feedback on EPP outreach materials.
Expected Outcomes Leading to Actionable Steps
Provide input to overall EPP strategy and policy.
Develop a hierarchy of least and most important factors used in a purchasing decision.
Identify outreach and policy strategies for making EPP tools and information available.
Collect input on outreach (message content, target audience, format, and delivery mechanisms).
Obtain feedback on strategies to remove barriers and increase motivation to make purchasing decisions.
Research Plan
Study Population
Learning Question
Areas
Research Tactics
(All research conducted in Washington, DC)
Dyad Interviews
Focus Group Interviews
One-on-One Interviews
Buyers Mixed
1,2,3
3
9
Requesters/Facility Managers
,2,3
Facilitators
1,3
Total Participants
11
21
Research Logistics
Date
Location
Tactics
Study Pop.
Environment
Other
Week of 4/3
DC
Short interview
Buyers
One-on-one interview
4/13/00
DC
Pilot interview
Dyad interview
Buyers
Hotel meeting room
NCMA Conference
4/13/00
DC
Pilot interview
Facilitators
One-on-one interview
Week of 4/17
Variable
Short interview
Facility Mgrs.
Telephone interview
4/19/00
DC
Dyad interview
Facility Mgrs.
Meeting room
Department of the Interior
Facility Mgr. Conference
5/2/00
DC
Focus groups (2)
Buyers (mixed)
Meeting room
OMB Front Line Forum
Week of 5/8/00
DC
Short interviews
Buyers
Requesters
One-on-one interviews
5/13/00
San Diego
One-on-one interview
Buyers
Meeting room
GSA Expo
130 Appendix A: Research Design
-------
CONFIRMATORY STAGE
(MlD-JUNE-JULY)
Research Objectives
Hone in on actionable steps and key implementation recommendations.
Test quantitative research outlines and instruments.
Recommend quantitative research approach.
Learning Question Areas
1. Behavior relating to EPP and defining EPP.
2. General awareness of and feedback on EPP Tools/Internet Tools Test.
3. Opportunities to fit EPP into day-to-day procurement activities.
Expected Outcomes Leading to Actionable Steps
Provide input to overall EPP strategy and policy.
Identify outreach and policy strategies for making EPP tools and information available.
Modify current outreach materials based on interest level, usefulness, and link to expected behavior changes that lead to incorporating
environment into PDPC.
Incorporate EPP experiences into EPP strategies, policies, and outreach materials.
Research Plan
Study Population
Learning Question
Areas
Research Tactics
(All research conducted in Washington, DC)
One-on-One Interviews
Dyad or Triad Interviews
Focus Group Interviews
Buyers
2
10
Buyers
1,3
13
Requesters: Bankcard holders,
program staff, building/
facility managers
Requesters: Bankcard holders,
program staff, building/
facility managers
1,3
15
Facilitators
1,3
Participants
51
10
Research Logistics
Date
Location
Tactics
Study Pop.
Environment
Other
Junel2-Junel6
Various
One-on-one interview
Mixed
Telephone interview
Pilot
Week of 7/10-7/21
DC
One-on-one interview
Buyers
Telephone/in-person interview
Tools/Full
Week of 7/10-7/21
DC
group interview
Buyers
In-person interview
Tools
Week of 7/10-7/21
DC
One-on-one interview
Requesters
Telephone/in-person interview
Tools/Full
Week of 7/17-7/21
DC
group interview
Requesters
In-person interview
Tools
Weeks of 7/17-7/28
Various
One-on-one interview
Vendors
Telephone interview
Full
Week of 7/24-7/28
Various
One-on-one interview
Mixed
Telephone interview
Follow-up
Appendix A: Research Design 31
-------
-------
Appendix B: Screening Protocols
EPP Interview Screener
Respondent Name:
Agency/Affiliation
Respondent Address:
Respondent City:
Respondent Phone: ( )
State: Zip Code:
Fax ( )
Hello, my name is of NuStats Research and Consulting. We are working
with a Federal agency to conduct market research related to Federal purchasing. As part of this effort, we
are contacting people like yourself who are involved in the request, specification, or purchase of products and
services to see if you might be willing to participate in a discussion group. The discussion will explore how
recent changes in acquisition reform have affected how certain products and services are selected and
purchased. By participating in this discussion group and sharing your opinions, ideas, and experiences, you
could contribute to or influence future purchasing practices.
Engage in a conversation that achieves:
understanding of the research objectives;
willingness to participate in the study; and
verification of agency, division, telephone, and email.
If person agrees to participate, say: "I'd like to ask you a few questions related to the study."
If person does not agree or can't participate because of a time/date conflict, ask: "Could you refer me to
someone else in your department who performs similar functions who might be willing to participate?"
1. Do you work for the Federal government in a civilian or military agency or as a Federal
contractor?
Q Yes (specify) Q No (TERMINATE)
[May be addressed in introduction]
2. Do you perform any of the following activities on a routine basis? (Check all that apply.)
Q Identify a need for or request product/services
Q Write specifications
Q Influence selection of products/services
Q Make purchasing decisions
Q Conduct life -cycle costing analysis
Q Purchase products and services
Q Conduct post-purchasing follow-up
Q None (TERMINATE)
Appendix B: Screening Protocols 33
-------
3. What is your job title or function? (Do not read from list. Check all that apply. If none apply, check
"other" and ask respondent for his/her job title/function. Read the list only if the respondent needs
prompting.)
Q Project Manager
Q Purchasing Manager
Q Contract or Purchasing Specialist
Q Supply Specialist or Manager
Q Bankcard Holder
Q Facility Manager, Engineer, or Architect
Q Other (specify)
4. For how long have you been involved in purchasing? (Terminate if less than 6 months.)
5. In the last six months, have you made a purchasing decision or selection for at least one of the
following products or services? (Check all that apply.)
Q Office Supplies
Q Office Equipment
Q Cleaning Products
Q Road and Parking Lot Renovation and Repair
Q Copier Paper
Q Paint
Q Engineering, Custodial, or Landscaping Services
Q Construction/Renovation
Q Large System Acquisitions (weapons, satellites, communications)
Q Other (specify)
6. Approximately how many purchase transactions do you handle or are involved in (e.g., as
requester, or service or product specifier)"!
In a typical month?
In a typical year?
7. Consider the last few "typical" purchases you were involved with or handled. Which of the
following categories best reflects those purchases?
Q Under $25,000
Q Over $25,000 but under $100,000
Q Over $100,000
Thank you for answering our questions.
Again, the discussion group will be held on (day) (date) from to (specify time) at
(name of facility) located at (address). The discussion will
last about 90 minutes. While we cannot pay you for donating your time, we will serve a light breakfast before
the discussion and will reimburse you for mileage or travel expenses. We will contact you on the day prior to
the discussion to confirm your participation.
34 Appendix B: Screening Protocols
-------
Appendix C: Interview Protocols
Discovery Stage Discussion Guide
Interviewees
Group 1: Providers of EPP Policy or Guidance
Group 2: Other Facilitators
Group 3: Contract Specialists
Perspectives on the EPP Measurement Project (Group 1)
1. Based on what you know about the EPP project we are undertaking right now, what do you think are this
project's objectives?
What would you like it to achieve?
What are your expectations?
2. What do you think you will be able to do with the results and products of this project?
3. What are the most valuable background informational materials you would recommend on the topic of
this project?
4. Who are the most valuable experts for advising this project? Why?
5. What are your recommendations for improving the project?
Status of the "Greening of Government" Initiative (Groups 1 & 2)
6. What does the "Greening of Government" initiative mean to you?
What is its current status in your opinion?
What other initiatives and efforts are related to this initiative?
Progress/Effectiveness of the EPP Program (Groups 7 & 2)
7. In your opinion, is the EPP program successful (e.g., Is it being widely implemented? Are agencies aware
of the program?)? Why or why not?
8. Who would you say are the key EPP "players" (and what are their roles) in the following segments?
Government policy/guideline personnel or offices.
Federal procurement/acquisition decision makers.
Private-sector personnel.
9. Who are the "antagonists" or "naysayers" of EPP?
What are their issues?
Do you recommend that we contact these people?
10. What do you think are the most serious barriers to the success of the EPP program in the Federal
government? Why?
11. How well informed on the EPP program requirements are E.O. 13101 stakeholders? Where do they get
their information?
Appendix C: Interview Protocols 35
-------
12. Which agencies/departments/personnel stand out in your mind as leaders in:
the "Greening of Government" initiative and
complying with EPP guidelines?
Federal Procurement System (Groups 2 and 3)
13. Define the "Federal acquisition community" (i.e., Who is included in the community and what are their
roles).
14. Within a government agency, who are the procurement "officials" (those with product decision-making
power)?
15. Describe the key processes for procuring products and services (e.g., charge card, GSA supply service, and
storeroom).
16. For this project, we are including the Federal acquisition community in focus groups, one-on-one
interviews, etc. Given the decentralized nature of procurement and acquisition practices within the
Federal government:
Of the many individuals you previously identified, whom should we target?
How should we contact these people?
17. To what extent do you believe that government purchasers are aware of either the "Greening of
Government" or EPP program requirements?
To the best of your knowledge, are your peers making purchasing decisions based on these
requirements?
What are their information and training sources?
What are the barriers or obstacles to carrying out these requirements facing procurement officials?
Exploratory Stage Discussion Guide
Introduction
Moderator disclosures: The moderator should introduce his/herself, the assistant, and notetaker and
explain their roles in the discussion. Inform the participants that the client is a Federal agency that is
trying to gain greater understanding about what motivates purchasing decision making.
Participant introductions: Ask the participants to introduce themselves, to specify their job function, and
to describe the most frequent purchase decision or product selection they make.
Warm-Up/Reconnaissance
Explain the purpose of session: to solicit insight on the motivations for and information needs regarding
making a purchase decision and/or product choice. Stress that today's discussion covers government
purchasingnot consumer purchasing. Emphasize that we want honest answersnot the "politically
correct" answers.
Explain warm-up.
Ask participants to specify the reasons for making a purchase decision and/or product choice.
Probe them on the dynamics of the Federal marketplace.
Ask who are the decision makers or product selectors (the originator of the request, the specifier, or the
purchaser).
36 Appendix C: Interview Protocols
-------
General Attitudes about Purchasing
Ask participants to remove the blue index card from the envelope (see Exhibit A). Tell them this activity
will help identify and rank the factors they consider when making purchasing decisions or selecting a
product or service.
Ask participants to rank the factors on the card, with 1 being the most considered and 9 being the least
considered factor. (There is a space to add one more factor and rank from 1 to 9.)
On the reverse side of the card, ask them to write down the products/services they were thinking of when
they ranked the factors.
On a flip chart with pre-printed factors, record other factors they added. Review high- and low-ranked
factors.
Discuss the least and most important factors.
What products were you thinking of for each?
Specify examples of when you used the highest-ranking factors.
Why don't you use other (low) factors?
EXHIBIT A
(BLUE INDEX CARD)
Price
Reputation of the Brand
Always Use the Product
Meeting Exact Specifications
Social/Environmental
Easiest to Purchase of All Available Choices
Small Business Set-asides
Mandatory Sources
Other
Defining Environmental Preferability
Ask participants to remove the yellow index card from the envelope (see Exhibit B). Ask them to record
their "top-of-the-mind" image or reactions to the words printed on either side (Green and Environment).
Who is responsible for defining these terms?
What are the connections between the two?
Ask participants to remove the red index card from the envelope (see Exhibit C). Ask them to record their
"top-of-the-mind" image or reactions to the words printed on either side (Environmental Purchase and
Green Products)
Who is responsible for defining these concepts?
What are the connections between the two?
Ask whether there a role for these concepts in making governmental purchasing decisions or product
selections. If so, ask why; if not, ask or why not.
Appendix C: Interview Protocols 37
-------
Side"
Side 2
Side"
EXHIBIT B
(YELLOW INDEX CARD)
Environment
Green
EXHIBIT C
(RED INDEX CARD)
Environmental Purchase
Green Products
Side 2
38 Appendix C: Interview Protocols
-------
Ask whether in their most recent purchasing decision or product selections, they have considered any
environmental factors (e.g., recycled content, energy conservation, low VOC, CFC-free). If yes, list the
factors on the flip chart.
What products or services were you thinking of when you mentioned each factor (record them on the
flip chart)?
Are these considerations made routinely, occasionally, or was it a one-time effort only?
Please provide specific examples.
Why did you consider these factors?
What barriers did you encounter to not including these factors in your decision making?
Describe a perfect world where all barriers were removed.
Would you use the environmental factors or attributes in a perfect world?
Refer to the list of factors.
Please record on a piece of paper the factors you consider to be the least and most important and
specify why.
Do you view or consider the factors singly or as multiple attributes?
Where do you obtain information on environmental factors or attributes?
Of the sources you use, what information is lacking or what would you like to see more of?
Attitudes and Opinions on the "Greening of Government"
Ask for a hand count on who is familiar with the "Greening of Government" initiatives.
Ask them to define the "Greening of Government." (Cite examples.)
Ask participants to remove the white index card from the envelope (see Exhibit D).
Rank the purchasing-related policies printed on the card in order of your familiarity with them, with
1 being most important.
Record your most challenging or rewarding experience with one item.
Refer to the list of purchasing-related policies. Ask the group to identify or name the purpose of each.
EXHIBIT D
(WHITE INDEX CARD)
Executive Order 12856
Executive Order 12873
Executive Order 14101
Federal Acquisition Regulations (FAR) environmental amendments
Executive Order 13123
Executive Order 13101
Executive Order 12902
Office of Federal Procurement Policy Letter 92-4
Appendix C: Interview Protocols 39
-------
Ask participants to report on the items they ranked as 1 and 9.
Are you carrying out the policies of the items you rated as 1? Provide examples and articulate what
makes it easy.
What are the barriers to carrying the policies out (including confusion and lack of motivation)?
Awareness of and Opinions on EPP Outreach Materials
Present the EPP outreach materials.
What materials are you familiar with?
Which ones are you not familiar with? Why?
Of the materials they are familiar with, ask which they have actually received.
How did you receive them (e.g., from EPA, from a colleague, on the Web site)?
Which delivery mechanisms work best?
Were they useful in helping you make a product/service selection? Why or why not? Please relate a
rewarding experience.
Ask what other information sources (non-EPA) participants are aware of or have used to make a
purchasing decision or product selection.
How did you obtain the materials (delivery, format)?
What did you like about the resource?
Ask what resources they need that currently don't exist.
Ask whether any of the materials they have seen or used, motivated them to make an environmental
purchasing decision or product choice. Why or why not?
Ask whether the materials, in general, help to alleviate some barriers to making environmental
purchasing decisions or product choices. If so, how?
Conclusion/Closure
Appeal for additional information.
Summarize the group's perceptions.
40 Appendix C: Interview Protocols
-------
Confirmatory Stage Discussion Guide
Partial Interviews (June)
Name:
Department:
Phone:
Fax:
Source:
Roles
Buyer: Contract Officer/Purchasing Specialist
Requester: Bankcard holder or Manager
Requester: Environmental Health and Safety Official, or Building or Facility Manager
Requester: Program Staff; Environmental Program
Facilitator: Policy; Pollution Prevention Coordinator
Other:
Hello, my name is . I am calling on behalf of the Environmental Protection Agency's
(EPA's) Pollution Prevention Division. We are working on a project to explore issues related to incorporating
environmental considerations into government purchasing. Would you be willing to participate in a 20-
minute discussionnow, or at a time that is more convenient for you?
// they are hesitant, explain why we selected them and how we obtained their name.
If they are still hesitant, say: We have conducted interviews with nearly 60 people in the Federal government
over the last two months and have arrived at a set of conclusions and recommendations for EPA. We would
like to test these recommendations before finalizing them.
// they still seem hesitant, say: EPA needs your insights and experiences so that the Agency can better
understand the "real world" related to purchasing so that it does not "work in a vacuum." EPA needs your
input so that it can create a program that meets the needs of those making purchasing decisions in the
government. This could make your lifeand the lives of others involved in purchasinga whole lot easier.
Would you be willing to participate? Now, or would you like to schedule a later interview?
Schedule Date and Time:
Warm Up
1. Of the following list of activities related to purchasing, which are you typically involved in?
Identify a need for or request a product or service
Write specifications
Influence the selection of products/services
Make purchasing decisions
Conduct "pre-purchasing evaluations" or life-cycle costing analysis
Identify vendors or sources
Purchase products or services
Conduct post-purchasing follow up
Other
Appendix C: Interview Protocols 41
-------
2. How long have you been doing this?
3. What types of products or services do you normally purchase?
General Procurement
4. Don't ask bankcard holders, but ask everyone else: What is the mechanism you most frequently use to
purchase products/services? (Probe on the use of that source and for which products and services.)
Small Purchase
Bankcard
Mandatory Source (PROBE: What is (are) that (those) source(s)?)
Small Business Set-aside
General Services Administration (GSA) or Defense Logistics Agency (DLA) schedules
Government-wide Contracts
Other (specify)
5. What factors do you consider when making a purchasing decision? (List all. Probe on Best Value, or ask
about what "Best Value" means to them.)
"Greening of Government"
6. Are you aware of or have you ever heard of the "Greening of Government" initiatives?
// they recognize the initiatives, probe on which ones.
If they don't recognize the initiatives, prompt them with the ones related to having to buy recycled
paper, ENERGY STAR, or green products.
How did you hear about them? During training? From management or internal Policy?
What other resources have you seen or used on the "Greening of Government"?
Do you consider green initiatives to be requirements? Why or why not? What do you think would
make them a requirement?
Awareness, Attitudes, and Behaviors about EPP
1. One of the Executive Orders requires Federal agencies to purchase products and services that are
"environmentally preferable." The Executive Order named EPA responsible for helping agencies and
providing guidance on considering environmental preferability in the purchase process. While EPA does
not certify specific products or create product lists, the Agency has developed a set of guiding principles.
Are you at all familiar with this concept?
8. When you think about "environmentally preferable products or services," what comes to mind?
9. The purchasing process involves a variety of people who make purchasing decisions at different points in
the process. We have observed that there is not agreement about who is responsible for making
environmental considerations or requesting that they be made. For each of the following entities, please
tell me if you think the responsibility lies there or not. (Probe what is the responsibility and whywhen is
the best point.)
BuyerContracting officer or specialist, purchasing specialist, approver of bankcard purchases
OriginatorProgram staff, bankcard holders
SourcesVendors, mandatory sources, GSA/DLA
10. A few questions back, we discussed factors that are routinely considered in making purchasing decisions.
We have observed that environmental considerations are not a high ranking factor for certain products
and services, but are higher ranked for office products.
Can you think of some other products or services where environmental considerations may rank
higher or lower?
42 Appendix C: Interview Protocols
-------
What might it take to get it considered more often?
11. If Best Value is not mentioned: Our previous research suggests that purchasing decisions in the Federal
government are made based on the "Best Value" for the goods and services needed. Some considerations
in Best Value include price, timeliness of delivery and availability, and the vendor's ability to meet
specifications. What would it take to incorporate EPP into those considerations?
12. Can you think of a specific instance of "green purchasing" within the Federal government? What was it?
Who made the decision? How?
13. Can you think of leaders or promoters of "green purchasing"? Who are they? What do they do?
In their department
Inside or outside of government
14. For keeping up with the practice of purchasing and procurement, what resources do you use most? (Probe
on the type of resource.) What is the format? What do you particularly like about it? How is it distributed?
15. If you were to make a recommendation to EPA regarding its program strategy on green purchasing as it
relates to Federal procurement, what would it be?
16. We'd like to do some further research on this issue. Would you be available for another interview with us
sometime in July?
17. Can you recommend anyone else in your office/department/agency with whom we can talk about this
topic? (Name and contact info.)
Thank you so much for your time and input. I appreciate your help. Have a good day.
Appendix C: Interview Protocols 43
-------
Confirmatory Stage Discussion Guide
Full Interview (July)
Discussion Guideline for Further Exploratory Probing on Attitudes
Hello, my name is . I'm calling on behalf of the Environmental Protection Agency's (EPA)
Pollution Prevention Division. We're talking with [Federal government employees] / [vendors of Federal
government agencies] to explore issues that relate to environmental considerations and purchasing decisions.
I'd like a few minutes of your time, either now or at a time more convenient for you.
// the interviewees are hesitant, explain why we selected them and where we got their names.
If they are still hesitant, say: We have conducted interviews with more than 60 people in the Federal
government over the last two months and have arrived at a set of conclusions and recommendations for EPA.
We would like to test these recommendations before finalizing them.
// they still seem hesitant, say: EPA needs your insights and experiences so that the Agency can better
understand the "real world" related to purchasing so that it is not "working in a vacuum." EPA needs your
input so it can create a program that meets the needs of those making purchasing decisions in the
government. This could make your lifeand the lives of others involved in purchasinga whole lot easier.
Would you be willing to participate? Continue the interview now or schedule a later interview.
Schedule Date and Time:
For all government employees:
1. First, I'd like to make sure we have your correct job role. What is your title and job function?
Title: Function:
(e.g. buyer, requester, facilitator, or vendor)
2. Of the following list of activities related to purchasing, which are you typically involved in?
Q Identify a need for or request a product or service
Q Write specifications
Q Influence the selection of products/services
Q Make purchasing decisions
Q Conduct "pre-purchasing evaluations" or life-cycle costing analysis
Q Identify vendors or sources
Q Purchase products or services
Q Conduct post-purchasing follow-up
Q Other (specify)
3. In the last six months, have you made a purchasing decision for at least one of the following products or
services?
Q Office supplies Q Office equipment
Q Cleaning products Q Copier paper
Q Road and parking lot renovation/repair Q Paint
Q Engineering, custodial, or landscaping Q Construction/renovation
Q IT systems and products Q Large system acquisitions
Q Conference planning Q Other (specify)
4. What is the most frequent purchase decision or product selection you make?
44 Appendix C: Interview Protocols
-------
5. What are the reasons or driving forces behind your having to make a purchase decision and/or product
choice? (Probe on how decisions are madeby the originator, by themselves, jointly, or as a team. Probe on
each to determine when and why it is done that way.)
Are your reasons different, depending on any of the following:
Whether you are purchasing a product versus a service?
The type of product and type of service?
The dollar value of the purchase?
The purchase mechanism?
6. As you make purchase decisions and/or request a product or service, what are the types of factors you take
into account? Record each factor below. For each factor, ask what type of product or service they were
thinking about, and ask where they get information for considering that factor.
Factor Product/Service Information Source
Explore their motivations for their ranking of each factor.
Ask whether their information sources are written materials or people. Ask them to cite examples for both.
1. If environmental considerations are mentioned (e.g., recycled paper, computer /ENERGY STAR, FAR
clauses), say: You mentioned that you consider [fill in with answer] when making a purchase /purchase
decision. Is that something you take into account for most purchases? What was the motivating factor
that made you consider that [those] environmental consideration^]?
8. Can you tell me about a specific recent purchase where you considered those factors? (Probe on the use of
single- verse multiple-attribute considerationswhy or weren't they made?)
9. If environmental considerations were not mentioned. How about environmental considerations, such as
local environmental concerns, or buying products with other environmental attributes? Do you think of
those factors in the purchase process? If yes, ask them to explain further; if no, ask them to explain why not.
10. Typically, at what point in the purchase process should those factors be considered? Would you say it's:
Q When there's a need or when a request is made?
Q When someone is conducting market research or selecting a vendor?
Q When a purchase is actually made?
Q Never? Why?
11. Where in the purchase process do you think responsibility lies for making environmental
considerations? // unsure, probe:
Do you think it lies with the people making the request, or perhaps with the people who makes the
actual purchase?
Are these people aware of this responsibility? If so, how were they made aware of it?
Do they accept this responsibility? Why or why not? What needs to be done to make them aware of
and accept this responsibility?
12. Are you aware of the "Greening of Government" initiatives?
If yes, ask: Can you think of any specific examples of initiatives?
If no, ask: Have you heard of Executive Orders or the FAR environmental amendments, or anything
similar?
Appendix C: Interview Protocols 45
-------
13. If there were no mandates or Executive Orders or FAR clauses related to environmental considerations,
would Federal government procurement and purchasing be easier or more difficult? Ask them to
explain their answer.
14. Do you think that mandating environmental considerations as they relate to procurement and
purchasing is the only way that agencies will carry out those initiatives? Why or why not?
15. The Executive Order related to our discussion right now is Executive Order 13101. Are you familiar
with it?
If yes, ask: How did you hear about it? What does it mean to you?
// the answer was no, briefly explain or clarify main components of the Executive Order. Ask: Does it
sound familiar now? Why? Which components are familiar? How did you hear about them?
16. Are you familiar with the term "environmentally preferable purchasing"?
If yes, ask: What does it mean? How well do you think the term is understood by your peers?
Q Quite well Q A little bit Q Not at all
Why is this so? Probe on whether it is because of a lack of information, no management/policy
support, indifference, too many other things to do, etc.
Regardless of the answer, explain or correct their previous response by getting them to see beyond the
single-attribute and multiple-attribute considerations. Does the term EPP work well for you? If not,
ask: Can you think of a better term or something that would be more meaningful to you or others?
Do you think it really doesn't matter what you call it, so long as the program works?
17. For several years, EPA has been developing tools and providing assistance to the Federal government
on conducting environmentally preferable purchasing. I'd like to get your ideas about how EPA can
improve on its efforts.
What might it take to get environmentally preferable purchasing practiced more widely or
considered more often?
Should environmentally preferable purchasing "piggy back" onto or be incorporated into other
initiatives or should it be presented on its own? Why?
Are there existing purchasing-related requirements and processes that environmentally preferable
purchasing can easily be incorporated into?
Given what you now know about environmentally preferable purchasing, what, if any, information
materials or tools do you think are needed? What are credible sources? What formats would you
prefer for receiving these tools?
Who is a credible source for information on procurement issues? On environmental purchasing
issues?
18. What information materials or resources do you depend upon most often for making purchasing
decisions or product/service selections? Why? What do you most like about that resource? Check all
that apply.
Paper Formats
Q Memo Q Periodicals/newspapers
Q Policy/white papers Q Newsletters
Q Magazines/journals G Other (brochures, reports, posters)
Person-to-Person Formats
Q Presentations at conferences Q Hotline/technical assistance
Q Presentations at meetings Q Exhibit booth
Q Agency/installations Q Other (specify)
Q In-person training
46 Appendix C: Interview Protocols
-------
Electronic Formats
Q Web-based Q E-mail
Q CD-ROM Q Listserves
Q Video G Other (specify)
19. Have you seen or used any of the written materials or tools about environmentally preferable
purchasing that EPA has produced? If yes, ask: Which ones?
How did you hear about them and/or how were they received?
Are they useful? If yes, ask: Why? Did they help you make a product selection or purchasing
decision?
Did anything about the design or format of the material stand out? If so, ask: What was it? Why
did it stand out?
20. Regardless of their previous answer, ask: Have you received material from another source on
environmentally preferable purchasing? If so, ask: From whom? How did you hear about it, and how
was it received?
21. Consider the many ways that information can be presentedwhich of the following formats do you
prefer? Why or why not?
Q Narrative vs. directive
Q Case study (detailed) vs. checklist
Q Product lists
Q Vendor lists
22. Given what you now know about environmentally preferably purchasing, what kinds of information
about it do you think would motivate you to incorporate environmental factors into your purchasing
decisions? Why would this information be a motivator?
23. One way EPA is promoting environmentally preferable purchasing is by sharing case studies. Can you
think of an example within your agency/department? Could you tell me more about that project?
Appendix C: Interview Protocols 47
-------
Confirmatory Stage Discussion Guide
Tools Test (July)
The Office of Pollution Prevention of the Environmental Protection Agency (EPA) is tasked with assisting Federal employees in making
environmental considerations in purchasingalso known as environmentally preferable purchasing (EPP).
EPA wants your opinions about its Internet resources related to EPP. Please use the following guide to explore a variety of on-line tools. At our
follow-up meeting, we'll talk about your reactions to the tools and get your thoughts on improving the resources.
Your follow-up time is scheduled lor , July , 2000 at a.m./p.m.
If you have any questions, please call Mia Zmud, NuStats Research and Consulting, at 1 -800-447-8287, ext. 2224. Thanks for your time!
Internet Resource Evaluation Guideline
Over the next few days, please visit this Web site: www.epa.gov/opptintr/epp/how-to.html
To try out the tools, click on the link on the left side of the page called "Tools." You'll go to the page with the
Greening Uncle Sam (G US) Purchasing Tool Suite. The tools were developed (or are under development) by
U.S. EPA's EPP program to assist purchasers in putting EPP into practice. First read this page, then try out
the following tools.
Database of Environmental Information for Products and Services
Assignment: The database is organized like a giant mall. We'd like you to go through the steps you'd
need to purchase something, or make a purchase decision if the item is not yet available on-line. Assume
you need to buy cleaning products for your office or facility. You can use either the browse or the search
function to find what you're looking for. Both of them lead to the same information, but offer different
ways to get there. We want your feedback on using both functions, so look for information on cleaning
products using both features.
Hint: Try the hardware store when using the Browse function, and select Cleaning Supplies (Industrial
Use) when using the Search function.
Three added features you can look through are: Contract Language, Voluntary Standards, and
Environmental Attributes. These give Federal employees some ideas for how to include EPP language in
contracts, what the voluntary standards are, and the environmental attributes of the product. Your input
on these links will greatly assist us in evaluating the Web site and the EPP program itself.
Comments: Use this space, or a separate piece of paper, to write down any comments you have about the
site as you're going through it.
Promising Practices Guide for "Greening" Contracts
Assignment: This is an on-line source for green purchasing tips, strategies, and success stories. This
information was compiled to make it easier to identify and purchase:
Recycled-content products
Energy-efficient products
Bio-based products
Environmentally preferable products and services
Currently, this guide includes 16 success stories (or case studies) highlighting how Federal agencies have
successfully incorporated environmental concerns into the purchasing process. Each success story includes
details about the purchasing process, identifies the lessons learned, and provides links to additional
48 Appendix C: Interview Protocols
-------
information, such as the contracts or specifications. Success stories can be sorted by agency and/or by
product category. The guide also includes an extensive list of related resources.
We'd like you to review the success stories to see how other Federal agencies incorporate environmentally
preferable products into their purchasing. One example, EPA's Region 10 Remodels with EPP, reviews
how Region 10 incorporated a wide variety of green features into its refurbished executive office suites.
The project showcases the latest innovations in "green construction" and environmentally preferable
purchasing. While you're there, please check out the "At a Glance" section, which has a link to the
contract language used for Region 10.
Comments:
For the above tool, please evaluate the following items:
Had you ever visited this site and used this tool before?
Is this tool easy to use and understand? Do you like the format? Why or why not?
Does seeing other agency's efforts help you in your job?
Would this tool help you make purchasing decisions? For which products and services?
Tips for Buying Green with the Government Credit Card
Assignment: For those of you who hold a government credit card (Purchase Card), this tool was created to
offer some green tips for EPA's Purchase Card Training Course material. EPA's Manual for Credit Card
Holders includes tips to help you make "greener" choices when buying products using your credit card.
Please take a few moments to review the following topics:
Buy products with recycled content
Buy products with reduced packaging
Look for the Energy Star label
Ask if the product contains hazardous materials or toxic chemicals
Look for other information on the environmental features of products
This section should help you understand how to include environmental considerations in purchases made
with your Federal credit card.
Comments:
For the above tool, please evaluate the following items:
Have you ever visited this site and used this tool before?
Is this tool easy to use and understand?
Would it help you in your job?
Would it help you make purchasing decisions?
Interactive Training Tool
Much of the material on the EPP site is available in an interactive version. The interactive version can be
found at: www.itgco.com/gentt/. Click on "View the Interactive Version of This Training Tool."
To view the interactive version, you'll need the pluggin Flash Player installed and enabled on your
computer. If you don't currently have Flash, you can download it free at:
http://www.macromedia.com/software/
If you are unable to view the Interactive version, please enter through the "View the Text Version of This
Training Tool."
If your computer doesn't have a sound card, this interactive tool will show test and graphics, but the audio
will not be available for you to hear. You can still view the site, without sound.
Appendix C: Interview Protocols 49
-------
Once you're in the training tool, see Section II, Current EPP Mandates. Please review one of the
following topics:
Executive Order and FAR Requirements
EPA's EPP Guidance
For the above tool, please evaluate the following items:
Have you ever visited this site and used this tool before?
Is this tool easy to use and understand? Why or why not?
Would it help you in your job?
Would it help you make purchasing decisions? For which products and services?
Cleaning Products Pilot Project (CP3)
Assignment: The CP3 Web site provides information related to biodegradable cleaners and degreasers
offered in the GSA's (General Services Administration's) Commercial Cleaning Supplies Catalog. Federal
employees can browse and order cleaning products using GSA Advantage!, GSA's on-line ordering system.
We'd like you to go through this Pilot Project tool, with the intent to make a purchase decision about a
biodegradable cleaning product, using the Purchasing Decision Wizard, Option 3 (weighted attribute).
This will help you rank various product attributes.
The URL is: http://www.epa,gov/oppintr/epp/cleaners/select/matrix.htm
Comments:
For the above tool, please evaluate the following items:
Have you ever visited this site and used this tool before?
Is this tool easy to use and understand? Why or why not?
Did you feel comfortable setting the threshold of relative importance levels?
Would it help you in your job?
Would it help you make purchasing decisions?
Would it be valuable to you for other product categories as well?
We'll contact you again at the appointed time (see front page) to get your opinions about and reactions to
the pages you just reviewed. Please be completely honest in your assessment about the content,
layout/design, and practical applications to your job. Your feedback will help EPA improve the site, as
well as provide valuable insight into how to best communicate the EPP program to Federal agency
employees. Thanks for your time!
50 Appendix C: Interview Protocols
-------
Appendix D:
Secondary Research Bibliography
Burman, Allan V. December 1998. "Who's Who in Acquisition." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Burman, Allan V. April 1999. '"Smart Pay' Pays off." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Carter, Craig R., and Joseph R. Carter. 1998. "Interorganizational Determinants of Environmental
Purchasing: Initial Evidence from the Consumer Products Industries." Decision Sciences 29(3).
Federal Acquisition Institute (FAI). 1998. Report on the Federal Acquisition Workforce Fiscal Year 1997.
Washington, D.C.
Federal Register. "Executive Order 13101Greening the Government Through Waste Prevention, Recycling,
and Federal Acquisition." Wednesday, September 16, 1998:49643-49650.
Government Executive. 2000. Procurement Preview.
Kittower, Diane. 2000. "High IQ Plastic." The Public Purchaser. July/August:24-28.
Laurent, Anne. September 1997. "Sudden Impact." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Laurent, Anne. June 2000. "E-Invasion: Dot.corns Are Swarming into the Government Market." Government
Executive June:27-34.
Maxwell, Alison. October 1997a. "Green Government." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Maxwell, Alison. October 1997b. "Greenbacks for Green Firms." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Maxwell, Alison. August 1998. '"Planet GSA' Aims to Go Green." Government Executive [On-line]. Available:
www.govexec.com [12/2/99].
Office of Federal Procurement Policy (OFPP). No date. "Professional Development," Acquisition Reform
Network (ARNET). [On-line]. Available: www.arnet.gov/profdevelop.html [12/7/00].
The Reeves Agency and Jacobs, Jenner & Kent. 1999. EPA Communications Study. U.S. EPA Purchase Order
No. 8W-2455-TTSA.
Saldarini, Katy. January 2000. "Feeling the Consequences of Procurement Reform." Government Executive
[On-line]. Available: www.govexec.com [1/24/00].
U.S. Department of Defense (U.S. DoD). Office of the Secretary of Defense Under Secretary of Defense -
Acquisition and Technology. 1995. Recycling, Waste Prevention and Acquisition Benefit the National
Environment Available:
-------
U.S. DoD and Swedish Armed Forces. 1999. Environmental Considerations in the Systems Acquisition
Process: A Handbook for Program Managers. Available: and
[12/8/00].
U.S. Environmental Protection Agency (U.S. EPA). No date. "Environmentally Preferable Purchasing (EPP)
Program's Communications Plan - A Living Document."
U.S. EPA. February 1997a. Cleaning Products Pilot Project. EPA 742-R-97-002. Available:
www.epa.gov/oppt/epp.
U.S. EPA. May 1997b. Selling Environmental Products to the Federal Government. EPA 742-K-97-002.
Available: www.epa.gov/oppt/epp.
U.S. EPA. November 1997c. EPP Update. EPA 742-F-97-002. Available: www.epa.gov/oppt/epp.
U.S. EPA. September 1998. EPP Update. EPA 742-F-98-019. Available: www.epa.gov/oppt/epp.
U.S. EPA. March 1999a. EPP Update. EPA 742-N-99-001. Available: www.epa.gov/oppt/epp.
U.S. EPA. March 1999b. Guide for Measuring Compliance Assistance Outcomes. EPA 300-B-99-002.
U.S. EPA. May 1999c. "EPA Guidance on Conducting Inspections of Federal Facilities for Compliance with
Section 6002 of RCRA." Memorandum. Washington, B.C.
U.S. EPA. July 1999d. Defending the Environment at the Department of Defense. EPA 742-R-99-002.
Available: www.epa.gov/oppt/epp.
U.S. EPA. September 1999e. EPP Update. EPA 742-N-99-002. Available: www.epa.gov/oppt/epp.
U.S. General Accounting Office (U.S. GAO). 1998. Report to Congressional Committees. Acquisition Reform:
Implementation of Key Aspects of the Federal Acquisition Streamlining Act of 1994. GAO/NSIAD-98-81.
Washington, B.C.: U.S. Government Printing Office.
U.S. General Services Administration (U.S. GSA). 1998. White Paper: Acquisition Sources and Alternatives.
Washington, B.C. Available: [01/22/00].
U.S. GSA. 1999. Federal Procurement Report: Fiscal Year 1998 Through Fourth Quarter (October 1, 1997,
Through September 30, 1998). Washington, B.C.
U.S. National Aeronautics and Space Administration (NASA). JE/Environmental Management Bivision.
1999. Affirmative Procurement Plan for Environmentally Preferable Products. Washington, B.C. Available:
www.nasa.gov.
U.S. NASA. Ames Research Center. Purchasing Bankcard Program: Training Packet. Washington, B.C.
Available: www.nasa.gov. No date.
U.S. Office of Personnel Management (U.S. 0PM). 1999. The Fact Book - 1999 Edition: Federal Civilian
Workforce Statistics. OWI-99-2. Available: http://www.opm.gov/feddata/factbook/index.html [12/8/00]
White House Task Force on Greening the Government Through Waste Prevention and Recycling. March
1999. Strategic Plan to Implement Executive Order 13101: Greening the Government Through Waste
Prevention, Recycling, and Federal Acquisition. Available:
[11/30/99].
52 Appendix D: Secondary Research Bibliography
-------
Appendix E: Interview Tracking
Interviews Tracked by Agency and Role
Agency
AID
DOC
DoDAF
DoDAR
DoD DLA
DoDMC
DoD NAV
DoD-OTH
DOE
DOI
DOJ
DOL
DOT
ED
EPA
GSA
HHS
JWOD
NASA
0PM
Treasury
USDA
VA
Vendor (JWOD)
Vendor (Commercial)
Vendor (Contractor)
Total
Buyer
2
1
7
4
1
3
5
3
2
7
4
1
1
2
1
3
11
5
1
1
66
Requester
3
2
1
2
2
3
5
1
3
1
5
2
30
Facilitator
1
2
2
1
0
0
4
3
1
1
4
2
1
2
1
1
1
2
2
3
3
37
Total
2
5
11
7
2
5
7
7
8
13
5
2
1
2
8
6
17
2
1
1
5
2
5
2
3
3
133
Appendix E: Interview Tracking 53
------- |