&EPA
United States
Environmental Protection
Agency
Prevention, Pesticides
and Toxics Substances
(7506C)
EPA-730-R-95-001
September 1995
I
Office of Pesticide Programs:
Report on Customer
Satisfaction With
Program Services
Targeting OPP Customer Service Improvements
ENVtRONMENTAL
&
PUBLtC
PUBLIC
Recycled/Recyclable • Printed with Vegetable Based Inks on Recycled Paper (20% Poslconsumer)
-------
-------
ACKNOWLEDGMENTS
Univer ?W™ °f Pes!!cide PTO&ams (OPP) ^ould like to thank Dr. Denise Scheberle of the
su*ey 4^T °^^ f°r ^ aSS1StanCC * successMy Completing the "General Public"
OPP CUSTOMER SURVEY WORK GROUP
Ameesha Mehta/FOD: Project Leader
Karen Angulo/EFED
Marjorie Fehrenbach/OPP-IO
Michael Hardy/FOD
Anne Lindsay/PSPS
Peter Caullkins/RD
AlanGoozrier/BEAD
Susan Lewis/RD
Richard Petrie/EFED
Penelope Fenner-Crisp: Advisor
-------
TABLE OF CONTENTS
5
I. GLOSSARY OF TERMS
.... 6
II. EXECUTIVE SUMMARY
7
III. INTRODUCTION
A CUSTOMER SURVEY FRAMEWORK ?
B! OPP'S INITIATIVE
ffl. THE NATIONAL TELECOMMUNICATIONS NETWORK SURVEY 10
... 10
A. INTRODUCTION 10
B. METHODOLOGY n
C. FINDINGS ' " ' .13
D. CONCLUSIONS
IV INTERGOVERNMENTAL RELATIONSHIPS BETWEEN EPA HEADQUARTERS AND ^
REGIONAL OFFICES AND STATES
.... 15
A. INTRODUCTION 16
B. METHODOLOGY 16
C. FINDINGS • • • • • ' •; . . 18
1. statftProprarr-g «H State-Level Support ...18
a) Federal-State Relationships
b). State-Regional-Federal Relationships
.... 21
D. CONCLUSIONS 21
1 St?tesperceive '....... 21
2. gtat?P ttenomrnend; 21
3. ttfigtrms Perceive." 25
4. Ktj-gtons Recommend:
V. NATIONAL SURVEY OF PESTICIDE PROGRAM SERVICES 25
' .... 25
A. INTRODUCTION •••••' 25
B. METHODOLOGY 25
1. Sample : ..;... 26
2. Questionnaire •" : 26
3. Ttftsfponse tn the Survey • • • •
• . 26
C. SURVEYRESULTS •••••' ' ' 26
1_ fiegpondents 31
2. Jrppnrtance of V^tf*™ Services
-------
E.
3' Program Performance Ratings . 34
4. Overall Approval Rating 35
5. Benchmarking 36
D. ANALYSIS
36
1. Performance Gaps 37
CONCLUSIONS
VI. RECOMMENDATIONS
39
37
I. GLOSSARY OF TERMS
Benchmarking: refers to analyses of customers, competitors, trends in the market, differences in
products or service characteristics and organization's performance. It allows organizations to focus
attention on where it is most needed.
Core Processes: any process or system, in place, that centers on the review of applications and data
submitted by the pesticide industry, leading to decisions to approve or deny registration or reregistration.
Customer Satisfaction: an emotional and intellectual reaction to one or more characteristics of a
product, service, or quality.
Customer Service Standards: set by OPP for services we provide to all our customers It is used to
judge the quality of the service provided.
Performance Gaps: a method of comparison whereby the difference between the importance rating and
performance rating is assessed. This method allows decision-makers to prioritize program improvements
and allocate scarce resources to targeted areas.
Performance Standards/ Goals: Not the same as individual performance stand ards, though two could
be connected. It is a specified outcome or result on one or more criterion. It is the desired state or level
or future performance of a service.
-------
H. EXECUTIVE SUMMARY
EPA's Office of Pesticide Programs (OPP) is committed to improving the quality of our customer
services In the fall of 1994, OPP conducted a series of baseline "customer surveys m order to identify
"hatl most important to all customers, and better target the areas where improvements are
e OPP's customer service initiative is also in response to the President's Executive Order
which requires all Federal agencies to identify their customers, the kind and quality of semces
desired by their customers, their customer's level of satisfaction with existing services, and to compare
their own performance against the "best in the business."
OPP conducted three customer surveys that were designed to encompass the diverse spectrum of
are affected by and interested in pesticide regulatory and implementation issues (e.g.,
stiy, States, environmental groups, fanners and other user groups, other government
congreSonal staff, the public, etc.). The results of a preliminary "pilot survey,' conducted m
mid 1994, were used in the development of the final survey instruments.
Several opportunities for improvement were identified from the survey results. The general
public (survey #1) rated as satisfactory or above OPP's performance in responding to the public the
qul ry oSo'rmaL that they received from OPP, and OPP's ability to protect the environment and
food supply OPP received lower ratings for the public's ability to access pesticide information. The
EPA Reg ons and States (survey #2) identified under funded mandates, few opportunities ^ early in^ut
decision making process, and OPP's poor telephone *y?^^™££m
srant and Environmental/Public Interest community (survey #3) rated OPP s overall
? 4 4 on a scale of 10, identifying the lack of timely decisions, lack of consistency in policy
CtUUl W
-------
III. INTRODUCTION
th v 7he/reS!dent! Exec«tive Order 12862 requires all Federal Agencies to identify their customers
the kind and quality of services desired by their customers, their customer's level of salfac ont^h
exis ing services, and to compare their own performance against "the best in the business - A M™ ch
1995 Memorandum from the President (part of Reinventing Government II) reaffirmed the
sizK^ssrto fxecutive °rder 12862'caiiing for the pubucation °f cust°mer -
standards by mid-1995 and a customer service report assessing performance against those standards
A. CUSTOMER ASSESSMENT FRAMEWORK
SEisfa °n is a i
TSTr SEsa °n s a "**™** ^d systematic approach a, illustrated by the
re. It involves identifying the customers who and should be served by the organization;
Identify
0 u s torn ers .
& Services
•^-*
Develop
P erform anoe
(8u rvey
M ethodc)
Conduct
Baselin e
C u stom sr
Assecem enf
Establish &
Post
Custom er
Service 8 td 's
(Benchmark)
Establish
Perfo rm an ce
G oals/S td's
CUSTOMER ASSESSMENT FRAMEWORK
' t (SUrV6y meth°ds) md C0nductin8 baseline assessments to determine
the kind and quality of services they want and their level of satisfaction with existing services- estaSL
^posting customer service standards and goals, and measuring results against £££?
Vhe ^ " thC busineSS' s^^g front-line employees on the
fin t 'f l^ bUSineSS; making fnformatio» ^ inf 'rmatio« -rvices
sily accessible, and finally, systematically addressing customer complaints.
B. OPP'S INITIATIVE:
^f6/PeStidde Pr0§rT (°PP) iS C0mmitted t0 imProvinSthe ^ua% of our customer
, oart of th,s commitment, and in response to the President's Executive Order 12862 and
Task Force, OPP create^S^ ™> *"**•
of conducting a series of baseline assessments and recommending areas where ii
The OPP workgroup began planning for the customer
5
service survey early in 1994. The
-------
* stomer^ho ™4Med by and h*£tod in pesticide regulatory and imptementatton .ssues:
State/Local pesticide regulatory officials
EPA Regional & Headquarters Offices
Other Federal Government Agencies
Congressional Staff
Industry/user group representatives including:
Chemical manufacturers
Formulators
Distributors (agricultural, biological, structural, lawn care)
Farmers
Certified pesticide applicators
Environmental public interest groups (including groups
representing environmental equity issues)
Press and media
Representatives of the public health and medical fields
International community
General public
The pilot survey proved to be very useful in developing the final survey Particularly ™P°^
were Jcritiques o7the survey instrument itself. A significant percentage of the respondents found the
Ce*t o te J too long, somewhat confusing, and containing many questions that were relevant to some
types of customers.
As a result of the pilot surveys preliminary findings, OPP abandoned the "one-instrument-fits-aU"
survey ££±? afpro^h and developed three survey instruments. Each of the three surveys targeted a
grouping of customers with similar service needs, as follows:
• Snrvev#l: Users of the National Pesticide Telecommunications Network (NPTN) (primarily the
general public).
• Survey #2: EPA Regional Pesticide Program Officers and State Pesticide Program Directors.
• Survey #3: Users of pesticide program services including: pesticide registrants, producers, and
formulators, and environmental and public interest groups.
The chapters that follow provide an in-depth look at each of the 3 1
methodologies, findings, and an analysis of the survey results. The last chapter
workgroup's recommendations for improvement in targeted areas.
OPP
-------
-------
A&C&/W m fffMMl PM/M UM OF
MT/OMt p£Sr/C/I>£ mECO/UM(/MMT/OMff£WOR«
AMKCf/W*
-------
1
IV. THE NATIONAL TELECOMMUNICATIONS NETWORK SURVEY
A. INTRODUCTION
The Office of Pesticide Programs (OPP) has conducted this survey to measure the satisfaction of
services provided to its "general public" customers. This chapter focuses on the responses of private
citizens who were polled by EPA's pesticide hotline, the National Pesticide Telecommunications Network
(NPTN).
NPTN is a toll-free telephone service that provides a variety of information about pesticides to
anyone in the contiguous United States, Puerto Rico, and the Virgin Islands. The service is designed to
provide accurate and prompt responses to requests for information about pesticides. Answers are given
on the telephone or in the next day's mail. Any requested information that is outside NPTN's expertise or
authority is referred to the appropriate contact within the Agency.
OPP used NPTN for this customer service survey because the service is regularly used by a broad
cross section of the general public. OPP recognizes that NPTN may not reach all types of private
citizens, and does not consider this survey to represent the opinions of all private citizens affected or
associated with OPP's services. Nevertheless, OPP believes that the use of NPTN for this survey has
provided a starting point (baseline) for measuring customer satisfaction of private citizens.
B. METHODOLOGY
NPTN agreed to ask 150 callers to the hotline if they would agree participate in OPP's customer
service survey. A systematic caller selection process was used, where every four caller to NPTN was
asked if they would agree-to participate in the survey.
The following is a summary of those questions asked:
4 How did you obtain the Hotline number?
* How often do you contact the EPA or the Hotline?
+ Demographic data, including city, state, gender, race, education, profession, number of
persons in the household, pets?
* How satisfied were you with the verbal information received?
+ How satisfied were you with any written material received?
* Was the information understandable?
4 Is pesticide-related information accessible to the public?
+ Do you have confidence in EPA's ability to protect the food supply?
+ Do you have confidence in EPA's ability to protect the environment?
+ Overall, is OPP responsive to the public?
-------
C. FINDINGS
The following charts provide summaries of the survey responses obtained by NPTN.
Concerning the verbal information received: 78% of the respondents were very satisfied,
21% were somewhat satisfied, and 1% were not satisfied.
After Contacting, how satisfied with
information received?
Not Satisfied 11%
Somewhat satisfied j^B 21%
Very Satisfied
78%
0% 20% 40% CO1/. 80V. 1190%
Percent of Respondents
Accessibility of pesticide information: 12% were very satisfied, 42% were somewhat
satisfied, and nearly 46% of the respondents were not satisfied.
Do you feel that pesticide information
is readily accessible to the public?
Not Satisfied
Somewhat satisfied
Very Satisfied
0% 20% 40% 60% 80% 100%
Percent of Respondents
10
-------
Confidence in EPA to ensure food safety: 20% of the respondents were very satisfied
61% were somewhat satisfied, and 19% were not satisfied.
How much confidence do you have in EPA
regulating pesticides - safe food
No confidence
Some Confidence
A lot of Confidence
0% 20% 40% 60% 80% 100%
Percent of Respondents
Confidence in EPA to protect the environment: 19% were veiy satisfied, 62%
somewhat satisfied, and 19% were not satisfied.
were
How much confidence do you have in EPA
regulating pesticides-safe environment
No confidence
Some Confidence
A lot of Confidence
0% 20% 40% 60% 80% 100%
Percent of Respondents
11
-------
OPP's responsiveness to the public: 31% were very satisfied, 62% were somewhat
satisfied, and 7% were not satisfied.
Overall, how would you rate EPA-OPP
Program responsiveness to the public
Not Satisfied • 7°/
Somewhat satisfied
Very satisfied
62%
I I I I II
0% 20% 40% 60% 80% 100%
Percent of Respondents
D. CONCLUSIONS
OPP recognizes that this survey has its limitations. The surveys sample size was limited tc> 150
persons and a majority of the respondents were white females with children and/or pets. Therefore, the
SSShfiited the need for better communication to the public regarding the availability of the
Hotline services, especially to low income and minority populations.
In addition many of the callers were "first-time callers," which limits the accuracy of responses
for one foUow^g question, "satisfaction of verbal information received." As with other surveys of this
nature thereTs an hiherent degree of bias because NPTN was conducting the survey while at the same
Te SsfsSe^th potential problems. The average caller may have felt some pressure to respond
favorably to the person whose assistance they needed.
Overall OPP received satisfactory or above satisfactory ratings for every question except the one
oertaining to accessibility to pesticide information. OPP believes that this survey even with its
Sons prides an adequate baseline for measuring the general public's satisfaction with the services
that OPP provides.
12
-------
13
-------
MTIRffOIMffMNTMREMT/ONM/PS
STATtS
RCG/0M
FEBRUARY 1995
14
-------
V.
INTERGOVERNMENTAL RELATIONSHIPS BETWEEN FPA
HEADQUARTERS AND REGIONAL OFFICES AND STATES
INTRODUCTION
- °f thiS CUSt°mer Survey is to acc»r*tely characterize the perceptions that
the nan H Pr°gramfdirectors and Reg^ Pesticide program offices have about their pro-ams Id
n oT ? f T6 °f ferSovemme^l relationships with EPA's Office of Pesticide Programs (Om
dev±±f "f fTZf thf! Perceptions> one raust first have an appreciation of the history 17 '
development of federal and State pesticide programs, and some of the issues, that they face today.
red.tr Jo! ** T^ ff ^ V •** federal Sovernment to ^gulate pesticide distribution through
registration was the Federal Insecticide, Fungicide and Rodenticide Act (FIFRA) of 1947 The
rZ±6ntS H° FIFRA " l %2 rCSUlted " ^ Changes k how the fede^ government approached the
he1±s "S T7T ^^ PeStiCidu S by CreatinS a partnershiP between *• ^^ govemmtt and
Itr S^ , J I ! amendments authorized the development and maintenaince of cooperative
agreements to State governments for the establishment of State pesticide programs The States who
pr°8ram «*» have experienSd dSm
s and deletions of programs mandated by the statute, and many resource fluxuations These
frequent changes have made it difficult for OPP, Regions, and States toV«ivelyde^op and maintain
good working intergovernmental relationships. ucveiop ana maintain
affected how they approach the implementation of pesticide programs, resulting in the numerous program
and policy varurtions that currently exist among Regions, among States, and befween O pTheadquZs
±t InH^81011' T *• ^^ Pr0gram and P°Iicy insistencies have proved to be problemS [he
past and have strained intergovernmental relationships. uuiemanc in tne
B. METHODOLOGY
In June 1994, OPP received copies of a survey about intergovernmentd relationshios
15
-------
this report. Note, also, that between June 1993 to December 1994 OPPTS included OHMhe Office-of
Son Prevention, and Toxics (OPPT) and the Office of Compliance Monitoring (OCM)
in December of 1994, OCM was moved to the Office of Enforcement and Compliance Assurance
(OECA).
This study incorporates data from three sources:
1) Surveys mailed to State Pesticide Program Directors;
2) Interviews with EPA Regional Pesticide Program Offices and Headquarters staff in the
pesticide programs; and . .
3) EPA documents, including the guidance documents for State pesticide cooperative
agreements.
Forty-four State pesticide program directors (88%) responded to the survey questionnaires^
Phone Sews were conducted with 9 EPA regional offices and 11 States. Phone calls were made to
State pesticide program officials who chose not to respond to the survey m order to determine
^te^S^ the survey itself influenced their decision not to respond. Their responses
suggest that there is not a non-respondent bias in the survey results.
C. FINDINGS
The findings in this report are grouped into two categories:
n State Programs and State-Level Support. State perceptions or responses about the
quality of the State program and State-level support for the program goals; and,
2) Federal-Regional-State Relationships. Perceptions or responses about the nature of the
State-Federal and the State-Regional-Federal relationship.
Note- to maintain the consistency of Dr. Scheberle's original report, the term perceptions is used
instead of responTeT The usage of the word "perceptions" is not intended to invalidate the responses of
State Program Directors or Regional Program Officers.
1. Stflt* Programs and Stqt?-T ftvel Support
State pesticide programs are established programs; 71 percent have been in operation at
least 15 years. Located in State agriculture departments, many State pesticide pro-ams ^absorbed
pesticide regulatory duties in the mid-1970's. State pesticide programs are large with about half of the
States operating programs with 20 or more staff.
Perceptions.
• 79 percent of State pesticide directors agreed or strongly agreed that the number
of State pesticide personnel were too few.
16
-------
The availability of federal funding is viewed as very important by State directors.
Perceptions
• 79 percent believe that EPA grants are very important to their programs
• 57 percent of State pesticide directors agreed or strongly agreed that the amount
of federal funding was too little. 39 percent responded that federal funding
adequate.
; was
Pesticide directors perceive high levels of administrative, legislative and public
support.
Perceptions;
• 82 percent believe that their top-level State administrators support the pesticide
programs.
• 48 percent feel that State legislators view the regulation of pesticides as an
important policy issues and support the programs.
• 66 percent believe that citizens consider the State pesticide programs were
addressing an important public health and environmental issue.
State pesticide program directors expressed a high degree of confidence in their
program.
Perceptions:
• 74 percent of State pesticide directors agreed that their staffare adequately trained
to run a pesticide program.
• 77 percent believe the State programs work effectively.
• 91 percent indicated that the pesticide program in their'State is now more effective
than a few years ago.
• 48 percent of State pesticide directors agreed or strongly agreed that certified
pesticide applicators in their State were well trained and did a good job
• 68 percent agreed or strongly agreed that the responsibility for regulating
pesticides most appropriately rests with the States rather than the federal
government.
2.
Federal-Regional-State Relationships
a). Federal-State Relationships.
Perceptions of State nesticide program directors;
• EPA Regional Offices are supportive of State programmatic efforts.
• 84 percent believe that their EPA regional office supports their
17
-------
program.
• 61 percent feel that EPA regional staff fairly evaluate State
cooperative agreements.
• 57 percent of pesticide directors agreed or strongly agreed that
EPA regional staff are valuable resources for their, program.
State and regional program staff are in frequent contact.
• 80 percent of State pesticide directors maintain daily or weekly
contact with EPA regional staff.
The level of perceived support and cooperation decreases when the
State directors assess their relationship with EPA Headquarters staff
or talk about the EPA in general.
• 48 percent believe that EPA Headquarters staff support their
program, much lower than the perceived regional support.
One in four pesticide directors believe that the EPA pesticides
program works effectively, and not a single State director believes that
a stronger national program is warranted.
One State director commented: "The emphasis of the federal program is
based on delivery of commitments in a form understandable to Congress,
not necessarily to produce effective State programs. Recently, the stress
on EPA officials at Headquarters and regions has been to justify everything
they do The resulting effect on States is much stronger oversight by the
EPA to the point of pettiness. This has eroded the relationship between
EPA and States, as well as the amount of time State personnel can spend
accomplishing program objectives."
EPA's communication with the States.
• 75 percent of State pesticide directors believe that the EPA does
not clearly communicate program goals and requirements,
effectively.
Pesticide program directors express both of the following concerns
with greater frequency, which may partially explain the much higher
percentage satisfaction with the Regional program officers.
• Headquarters staff simply aren't available (don't answer or return
phone calls).
Communication tends to be
opportunity for State input.
phone calls).
Communication tends to be one direction (top-down), with little
18
-------
EPA's arbitrary decision-making and perceptions about the direction
01 the pesticide program.
These comments are typical of pesticide directors' concerns:
• "Like all other relationships, open lines of communication are
critical. It is imperative that current information be readily available
and appropriate technical assistance be accessible for the
relationship to work effectively."
• "The EPA should seek more State input prior to developing new
initiatives."
• "EPA should coordinate release of documents, make releases more
timely, and be certain there are enough copies for the States to
distribute. This is simply a matter of poor planning and
communication."
EPA tends to increase requirements under cooperative agreements,
without increasing funding or State flexibility.
Unfunded and/or under-funded mandates to States is the most
frequently expressed complaint among pesticides program directors.
Concerns about flexibility are often expressed in two ways:
• The methods used to evaluate State programs.
« In the ability of States to focus on State priorities.
Some variants of the phrase "bean-counter" are used often by State
directors when expressing their concerns about the federal-State
relationship, such as the following comments;:
• "EPA is more interested in bean counts to show numbers The
State is more interested in accomplishments with real impacts An
example is the EPA requirement that we tab; 300 ground water
samples. Dedicating staff to that means less focusing on the
hundreds of pounds of excess pesticides that will be disposed of
improperly."
• "As long as the State can accomplish more than the feds ask for
everything is fine. But, the feds tend to be bean counters and don't
necessarily look at the whole program picture "
19
-------
b). State-Regional-Federal Relationships
p?{,inn?l Project ftfffryr*' Perceptions:
• Perceive themselves as "buffers" in the system: simultaneously
responsible for maintaining State performance to achieve national
goals while also assisting State program directors in running their
program. .
• Perceive their position as more collaborative and less
confrontational toward State directors. This may, at times, put
them at odds with EPA Headquarters.
• Comments of Regional Project officers:
• "There's more than just geographical distance that makes
Headquarters (staff) seem remote. There's a philosophical
difference. I trust my State, and see my role as onepart ot a
partnership. Sometimes, I think Headquarters (staff) are so
concerned about their own accountability, they forget that
States have come a long way, and basically want the same
thing we do.
• RegionalstaffagreethatEPAHeadquartersoccasionallysends
inconsistent messages about program goals and priorities, and
would like to have greater input into and influence on the central
decision-making process.
D. CONCLUSIONS
Many general conclusions can be drawn from this survey.
1. Staff Perceive;
a).
b).
c).
d).
Their staff, while too few, are adequately trained and run an effective State
pesticide program. State programs are continuing to improve;
EPA regional staff are valuable resources who are frequently contacted
who areconsidered to be supportive of State programmatic efforts, and are
fair in their appraisals of State programs;
EPA Headquarters staff are less supportive of State programs, are not
available (don't return phone calls), and don't provide adequate opportunity
for State input into decision-making;
The federal pesticides program does not work effectively and a stronger
federal role is not warranted.
2.
a).
All mandates must be adequately funded (no increase in requirements
without commensurate increase in funding);
20
-------
b). More flexibility in tailoring federal funds to support State priorities;
c). Less rigid accountability (bean-counting);
d). Improved Headquarters communication of national program goals and
requirements, improved EPA Headquarter stiff responsiveness (return
phone calls), and more opportunity for State input into the headquarters
decision-making process.
3. Regions Perceive;
a). Political constraints exist on both State program directors and on
headquarters;
b). There is a need for cooperative relationships between Regions and States.
4. Regions Recommend;
a). Recognition of their expertise by headquarters;
b). Headquarters must send consistent messages about program requirements.
c). Greater opportunity for their input into policy decisions.
The following tables provide a summary of characteristics and perceptions of State
Pesticide programs.
Table 1: Characteristics of State Pesticide Programs
Program longevity
Six years or less
at least 15 years
Personnel
Six or fewer
at least 20
Contact with Regional staff
Daily/weekly
Federal funding is very important to the program
Federal funding is adequate
Matching requirements (a)
2%
71%
7%
52%
80%
79%
39%
15%; 50%
(a) The matching requirements for die pesticides grant depend upon the program: federal govianment contributes 85% of the
cost for groundwater, worker protection and endangered species programs; 50% for certification program.
21
-------
Table 2
Perceptions about State pesticide programs and State-level support.
Pesticides
Administrative support for program
State program is effective
Legislative support for program
Citizen support for program
State staff is adequately trained
State program is more effective than
three years ago
States staffing is adequate
% Agree
82
77
48
66
74
91
21
n=44
Table 3
Perceptions about Nature of the Federal-State relationship
Perceptions related to Regional Offices
Regional staff valuable resource
Regional staff are supportive of program
Evaluated fairly in program review
Perceptions related to Headquarters
EPA clearly communicates program goals
Headquarters staff are supportive of program
EPA Program is working effectively
Perceptions related to program strength
Need for a stronger national program
Need for a stronger State program
% Agree
57
84
61
22
48
25
0
19
22
-------
23
-------
A&Eff/M MT/ffMT/ON IEWI
fMRO/VME/VT/ll £'Pl/Sl/C'/WREST
TO
OPP
PROGRAM
&RWCES
/H/ty/995
24
-------
VI. NATIONAL SURVEY OF PESTICIDE PROGRAM SERVICES
A. INTRODUCTION
EPA's Office of Pesticide Programs (OPP) has as its main function the issuance of regulatory
decisions for the registration and reregistration of pesticides under the Federal Insecticide, Fungicide and
Rodenticide Act (FIFRA). The objective is to safeguard human health and the environment from the
risks associated with the beneficial use of pesticides. In its effort to improve the services to registrants
and environmental and public interest groups, OPP revised its pilot survey amd sent it to 700+
respondents.
B. METHODOLOGY
L Sample
The target population for this survey included industry "registrants" and
environmental/public interest groups. Half of the registrants in OPP's "company name and address"
database ( about 700 ) were randomly selected and sent the survey, as well as 40 environmental and
public interest groups.
2. Questionnaire
Survey recipients were asked to rate OPP's performance on core processes within its
Registration and Reregistration programs (which include special reviews, reregistration eligibility
decisions, product reregistration, new active ingredients, biologicals, fast tracks and non-fast tracks)
Each core process had two categories to be ranked - performance and importance. The performance
category rated the level/quality of service provided, whereas the importance category rated how
important the service is to the respondent. Ranking was from one to five; one represented "strongly
disagree" - to - five "strongly agree." Each category asked basically the same types of questions
regarding timeliness, understandability, consistency, communication of decisions, whether the process
has improved and if the process is open to the public.
Respondents were further asked to "benchmark" OPP's services against other comparable
government/private sector organizations whose services, in the respondent's opinion, were viewed as
high quality." Factors to be considered for the Benchmarking were courtesy/professionalism timeliness
attention to detail, telephone responsiveness, understandability of our product output, consistency
competence, product knowledge, openness to customer concerns and relevance.
The survey included a section where respondents were asked to provide an overall rating
of product/service performance ranging on a scale from 1 to 10, where 1 represented "highly
dissatisfied," and 10 represented "highly satisfied."
The survey also had a section devoted to identifying the type of respondent (e g pesticide
producer, pesticide formulator, environmental group, public interest group and other.)
25
-------
3. Response to the Survey
The questionnaires were mailed out in mid-February and responses were requested by
March 17,1995. Atotal of 326 responses were received (47 percent).
C. SURVEY RESULTS
1.
Respondents
Of the 326 responses received - 307 were recognized as complete and usable as of the
cutoff date By respondent category the numbers responding were: 112 pesticide producers, 138
pesticide formulators, 4 environmental interest groups representatives, 1 public interest group
representative and 52 other or unknown.
Table 1: National Survey of Users of Pesticide Program Services
Number and Percent of Total Tabulated Questionnaires by
Respondent Category
Respondent Group
Pesticide Producers
Pesticide formulators
Environmental Interest Groups
Public Interest Groups
Other/Unknown
Total
Number Percent
112 36.5
138 45.0
4 1.3
1 0.3
52 16.9
307 100.0
2. Importance ^f Program Services
Respondents were asked to rate 45 customer service statements as to the importance of
that service being provided as part of OPP's Reregistration, Registration and Special Review core
processes. For example the first statement under Special Review Regulatory Decisions was:
"Risk assessments are understandable."
The respondent was asked to circle one response out of 5 that were provided with 1
representing "not at all important," 2 representing "not very important," 3 representing "neutral," 4
representing "fairly important" and 5 representing "very important". The 7 subareas in the core
processes listed were:
26
-------
Reregistration
Reregistration Eligibility Decisions (REDs)
Reregistration of End-use Products
Registration
New Active Ingredients/Tolerances
Biologicals
Fast Track Applications (Me-Too's/Amendments)
Non-fast Track Applications (New Uses/Formulations)
Special Review
Special Review Regulatory Decisions
s as a group (including pesticide producers and pesticide formulated
noD. °'—tlon of Pesticide products is the most imoortant cnfp «,•„,>»/•
OPP's program. Their highest program agereeat f II f ""P0™"1 core process/service area of
Too's/Amendments) with an imnottan^ oSL AP A°I i ,tors rated 1S for Fast Track Applications (Me-
v^/ vTiui an uupuildnce SCOre Or 4 31 on the S nnint cr>a1o TU ^ -i . v-iv
program area indicated is non-fast track application th 4 • • m°St imPortant
received a 4.19 and new active in&tv>/r * + i a Rcregistration of end-use products
• <»wuvc ingreuient tolerances 4 06 Snepial rox/i^nr t>cr» j i • < .
received, 4.05, 3.99 and 3.59 respectively Therefo '' review, KfcDs and biologicals
important side of the scale All core registrants rated all prognam areas on the
important and very important. None of tSatTmenL n^h ^^^ ^ REI>S Were rated between fairly
the registrants. statements on the survey were raited as being unimportant by
Core Process/Service Area Importance
Ratings by Registrants
Fast Track (Me-Too's/Amendments)
Non-Fast track Applications (New Uses)
Reregistration of End-Use Products
New Active Ingrediant/Tolerances
Special Review
REDs
Biologicals
•he
.. ... aggregate of all
point scale. The next most important
were rated at T '^k'f~^°7C"^ of end-use products received a 4.04.
were rated at 3.85 followed by new active ingredients/tolerances with a
27 :
-------
3.84. Non-fast track appucationsCnewuses^
^"^^^^
offte statements on the survey were rated as being urumportant.
Core Process/Service Area Importance
Ratings by Enviro/P I Groups
BEDs
Special Review
RtrefMratioii of End-Use Products
Fart Track (M«-Too'a/Amendments)
New Active Ingredient/Tolerances
Non-Fast track Applications (New Uses)
Blologicals
The statements on the survey
fall into eight categories of customer service standards:
scientifically sound
adequate guidance - clearly written
timeliness
consistency . .
understandability/effective communication
adequate resources
process has improved
open to the public
boTh with a rating of 4.18 on the 5 ^^^^"^Lted received a rating of 4.06.
4.10 and 4.13. Having guidance ^^f1^16^^dS got ratings of 4.02. The only
28
-------
Service Standard Aspect Importance
Ratings for Registrants
scientifically sound
adequate guidance - clearly written
timely
consistent
understandable/effectively communicate
adequate resources
process has improved
open to the public
The most important service standard aspect across all program areas as indicated bv
Pf IiVnt*reSt,?r°UpS 1S ^x™^/^™ communication with a rating of
the 5 point scale. Timeliness and consistency follow both with ratings of 3.92 Scientifically
sound and open to the public both got a 3.90. Having adequate resources is rated at 3 87 The
process being improved received ratings of 3.75. The lowest rating was for adequate' '
guidance/clearly written with a 3.66. However, this is still well above the neutral rating of 3
Therefore all aspects of customer service rated on the survey are identified as important to the
environ/public interest groups.
Service Standard Aspect Importance
Ratings for Enviro/P I Groups
understandable/effectively communicate
timely
consistent
scientifically sound
open to the public
adequate resources
process has improved
adequate guidance - clearly written
A ' • / u'
the environ/pubhc interest groups with two exceptions. Environ/public interest gnmps
±1PU, * ^I'^^^ble/effectively communicated" higher at 3.90 aJ U J
registrants rated these aspects at 3.62 and 4.06 respectively.
°f
29
-------
-------
gsssssss
-------
88888858
88888888
-------
-------
statements.
was:
3. ffrftgram Performance Ratings.
Resoondents were asked to rate OPP's customer service performance using 45
P pie "he &st of the 45 statements under Special Review Regulatory Decisions
•'Risk assessments are understandable."
Respondents were instructed to circle one response per statement using a scale of 1 to
major program areas of Reregistration, Registration and Special Review.
Registrants generally "disagreed" with statements regar
end of performance ratings with 2.78 and 2.73 respectively.
Core Process/Service Area Performace
Ratings for Registrants
REDs
Biological*
Special Review
fUrtfllstratton of End-Use Products
Hew Active Inflrediant Tolerances
Fast track (Me-Toos's/Anwndmenis)
Non-Fast Track Applications (New Uses)
i i
s
Ratines from environmental/public interest groups are generally neutral, not
disagreement wirti the statements regarding OPP's level of performance.
n of end-use products receives the highest performance rating wi h a
REDs wt a . an no-
category falling below the neutral rating was new active ingredient tolerances with a 2.87
30
-------
Core Process/Service Area Performance
Ratings for Enviro/P I Groups
Registration of End-Use Products
Special Review
Fast track (Me-Toos's/Amendments)
Biologicals
REDs
Non-Fast Track Applications (New Uses)
New Active Ingrediant Tolerances
aSp6Ct S6tting the highest ratinS is bein§
the public withalTh e ges ratnS s en§ °P^ to
Th/irv ! A A' 1& u6 Only aSpCCt fated above neutral (toward the 4 rating of 'agree ")
The service standard receiving the lowest rating is timeliness with a 2 54
Service Standard Aspect Performance
Ratings for Registrants
open to public
adequate guidance - clearly written
understandable/effectively communicate
process has Improved
consistent
scientifically sound
adequate resources
timely
31
-------
For the environmental/public interest groups the service standard aspect getting
the highest rating is being open to the public with a 3.41. Four other aspects were rated above a 3.
Sen~a 3.31, understandable/effectively communicated with a 3.26 adequa*
guidance/clearly written with a 3.22 and process has improved with a 3.13, The aspect ot service
receiving the lowest rating is having adequate resources with a 2.58.
Service Standard Aspect Performance
Ratings for Enviro/P I Groups
open to public
consistent
understandable/effectively communicate
adequate guidance - clearly written
process has Improved
scientifically sound
timely
adequate resources
Furthermore, registrants rated every service standard aspect of performance lower
n/puw£ interest groups except for having adequate resources. This is rated more toward
of levels of OPP's performance at 2.42, than registrants rating of 2.70.
The following Table 3 provides a summary of importance ratings delineated by
customer service standard and customer group.
32
-------
03
-------
CO
•s
CM
%
CD
T3
1
a>
W
>*-
O
"3
1
i
o
'
If
*?,
JL o)
o
g
g g
•S I
a d
i
§- 3
O
I S
D)(/) in
g C
(3 O
S-st
iM
o x> g
Q. -g z
(D II
£ «
£ 0)
CO C)
35 sx
T3 g
O .2
E
z
•g *
•>
s I 5
se
I If
S (D It
^OC-
§•8
°&
^s I
-"I
M
TJ
0)
.2
S
CO
(P
I
O
cvi
gggggggg
CO
CO CO
CO CO CO CO CO CO CO CO
S 3 S S S'S
cvi
1C
cvi
S
ssssssss
8fC f. O O O m
Sj «*{ 10 v> 55 ,f»
en
cvi
5. a
•5 S 9 2
"•wgo-s-fts-is
g g a ^ | f> g S
a i 1111 a 1
o o .•? £J g^a^ v
S
.
g
IIIII ill
o
-------
1
-------
-------
4. Overall Approval Rating
Respondents using a 10 point scale, with 1 representing "toghly dissatisfied" and 10
representing "highly satisfied," the following question received a 4.39.
"Overall, how do you rate our product/service performance?"
The rating category respondents most selected was a "4" (54 respondents). Almost
70 percent of the respondents rated OPP below the mid point on the "overall approval" scale.
Table 4:
Rating Category
1 Highly dissatisfied
2
3
4
5
6
7
8
9
10 Highly satisfied
Missing/Unknown
Total
Overall
Number
22
34
46
54
44
43
22
15
6
1
20
307
Approval Rating
Percent
7.7
11.8
16.0
18.8
15.3
15.0
7.7
5.2
2.1
0.4
100.0
Cumulative
Percent(%)
7.7
. 19.5
35.5
54.3
69.6
84.6
92.3
97.5
99.6
100.0
100.0
33
-------
Overall Approval Rating Average=4.39
Percent for each 10 point category
10
9 (0.35%
(2.09%;
(5.23%!
(7.67%;
.67%)
(14.98%!
(15.33%;
11.85%)
(16.03%)
(18.82%)
The 5 environmental\public interest groups gave a slightly higher rating for overall
performance with an average 5.0 rating. There were 3 responses with a rating of 3, and 2 responses
at 8.
5.
Benchmarking
Benchmarking refers to the analyses of customers, competitors, trends in the market,
differences in products or service characteristics, and the organization's performance. This method
of analyses allows organizations to focus service area improvement on where it is most needed.
The overwhelming areas identified for improvement are timeliness and telephone
responsiveness. Over 65 respondents specifically state that "timeliness" is an issue and over 60
highlight phone calls not being returned. A significant portion of the respondents state that other
agencies/companies are generally able to respond in a faster and more reliable time-frame. One
respondent state that the endless reassignment of personnel may be good for cross-training, but is to
the efficiency of the organization. Another example given regarding poor timeliness is not receiving
study results as soon as possible - especially when there are problems. The respondent indicated new
studies/information could be ongoing while other reviews progress. This could shorten time to
registration.
Concerning telephone responsiveness, numerous respondents indicate that they did
not receive a timely response to phone calls; that calls often are not returned for several days. This
appears to be a source of great frustration. The general indication is that the Agency needs to work
on customer responsiveness, i.e., returning phone calls. In addition, the manner in which we
communicate was highlighted. Some feel courtesy/professionalism is lacking in our communication
style.
34
-------
The third element most frequently mentioned is the lack of consistency for both regulatory
decisions and science reviews.
Some respondents provided general comments under the Benclimarking section. Praise is
given to the rejection rate analysis which provided needed guidance to better conducting the studies,
Lack of resources was also acknowledged. One comment states "many of us in the industry recognize
that most of the issues (timeliness, telephone responsiveness, consistency) are reflections of decreasing
Agency resources in a time when Agency responsibilities are certainly not decreasing." Another comment
states that although the Agency appears to be understaffed, they feel that the Agency has wasted
resources in other areas. "I feel the Agency is trying to do too many things for too many people."
Table 4 which summarizes counts from the competitive Benchmarking and service
activities are provided in the Appendix IV.
D. ANALYSIS
!• Performance Gaps
With many dimensions of data available, a means of prioritization provides valuable
information for management decision-makers to concentrate scarce resources for program improvement
One method of comparison is to look at the difference between ratings. This is called the performance
gap. Subtracting the performance rating from the importance rating gives a, negative performance gap for
every aspect rated under every program component of the survey. If the gap were to be positive it
would indicate that OPP's level of performance exceeds what customers want. With importance being
equated as customer's expectations, customer expectations are not being met.
Table 5: Difference of Performance from Importance (Performance Gaps) by Core
Process/Service Area
Core Process/Service
Area
Biologicals
REDs
Special Review
New Active
Ingredients/Tolerances
Reregistration -
End-use Products
Non-Fast Track
Applications (New
Uses/Formulations)
Fast Track
Applications (Me-
roo's/Amendments)
Importance Rating
3.65
3.97
4.04
4.08
4.19
4.26
4.29
==========
Performance Rating
======^======
299
3.07
2.89
2.79
2.81
2.76
2.78
==============
Difference (Gap)
0^66
0.90
1.15
1.29
1.38
1.50
1.51
35
-------
By program core process/service area, Biologicals show the smallest performance gap of
only 0 66 This is followed by REDs with a 0.90. The area with the largest performance gap is Fast
Track (Me-Too's) with a 1.51. This is closely followed by Non-Fast Track Applications (New
Uses/Formulations) with a 1.50.
When rating OPP's services to the public, timeliness receives the greatest gap between the
importance rating and the performance rating (1.61); the smallest gap was for openness to the public
(0.40) followed by "understandable/effective communication (0.98).
!SSS«^==============^
Table 6: Difference of Performance from Importance (Performance Gaps) by Service
Standard Aspect
Ot
Customer Service Standard
ublic
Importance
Rating
_—
3.67
Performance
Rating
___
3.27
Difference
'Gaf
_^>.
0.40
Understandable/effectively
4.06
3.08
0.98
ih
sroved
4.01
2.88
1.13
Scientifically sound
4.18
2.93
1.25
Adequate guidance/clearly written.
4.16
2.89
1.27
4.11
2.79
1.32
Adequate resources
4.02
2.69
1.33
2.51
1.61
E. CONCLUSIONS
These survey results provide the Pesticide Program with very useful information. The
respondent group included pesticide producers, formulators, environmental interest groups and public
interest groups. The results indicate that the Agency has not met the expectations of the registrants or
the environmental groups. The overall approval rating was a 4.39 on a scale of 10.
The environmental/public interest groups gave a slightly higher rating for overall
performance with an average 5.0. Over 70 percent of the respondents rated the program below the mid
point of the scale. The analysis of the program's current performance compared to the importance rating
resulted in negative gaps for every service standard ranked. The service with the largest gap was
timeliness of decisions, followed by consistent decisions. The core processes with the largest gaps were
Fast Track Applications followed by Non-fast track. Additionally, comments provided in the
Benchmarking segment clearly state that telephone responsiveness and professionalism needed
improvement.
36
-------
Registrants rated scientifically sound written decisions and receiving Agency policy
guidance which is clearly written and easily accessed as the most important to them. For Agency
programs, registrants rated fast track application then non-fast track as the most import. On the
performance side, with the exception of REDs, all other categories were raited below average. (Less than
a 3 rating).
The environmental/public interest groups rated effective co>mmunication of decisions
followed by timely decisions as the most important to them. For Agency programs, they rated
reregistration eligibility decisions and product reregistration as they're most important. On the
performance side, timeliness of decisions received the lowest rating.
37
-------
VH. RECOMMENDATIONS
Several opportunities for improvement were identified from the survey results. The general
public (survey #1) rated as satisfactory or above, OPP's performance in responding to the public, the
quality of information that they received from OPP, and OPP's ability to protect the environment and
food supply. OPP received lower ratings for the public's ability to access pesticide information. The
EPA Regions and States (survey #2) identified under funded mandates, few opportunities for early input
into OPP's decision making process, and OPP's poor telephone responsiveness as their areas of concern.
The Registrant and Environmental/Public Interest community (survey #3) rated OPP's overall
approval at a 4.4 on a scale of 10, identifying the lack of timely decisions, lack of consistency in policy
decisions, and poor telephone responsiveness as their areas of concern. The Registrants and
Environmental/Public Interest group also indicated that no service, provided by OPP, met their
expectations.
In providing three recommendations for targeted areas of improvement, the OPP workgroup
Examined the following: service areas that each customer group identified; poor service standards that
were consistently noted by all three customer groups; if the service area improvements were within OPP's
domain; whether the improvements in one area amplify or detract the improvements in another service
area; the upcoming OPP reorganization in 1996; and, other ongoing streamlining/re-engineering efforts.
Poor telephone responsiveness was noted by both the States and Regions, and the Registrants and
Environmental/Public Interest community. Though the general public noted its inability to access
pesticide information as its major area of concern, improvements in the telephone responsiveness of the
program would assist in increasing their information accessibility. The level of telephone responsiveness
will also be affected in the upcoming reorganization of OPP. In addition, any improvements in the
telephone responsiveness also enhances the earlier involvement of States and Regions into the decision-
making process which then amplifies timely and consistent regulatory decisions. Hence, by targeting
improvement efforts (resources) towards this service standard, OPP will realize greater benefits.
The second targeted area for improvement includes timely and consistent regulatory decisions,
identified as the second major concern by the Registrants and Environmental/Public Interest community.
Other service areas identified by States and Regions were earlier involvement in the decision-making
process and under-funded mandates. The OPP workgroup chose the earlier involvement into the decision
making process over under-funded mandates,' since this service area is in OPP's domain. Thus, the under
funded mandates issue could be addressed by the earlier involvement of all parties in the decision-making
and priority goal setting process. In addition, the above service areas are also of concern during the
upcoming reorganization.
Therefore, the OPP Customer survey workgroup recommends the following 3 service areas for
improvement:
1. Telephone responsiveness;
2. Timely and consistent regulatory decisions; and,
3. Earlier involvement of States and Regions in the decision-making and priority setting.
38
-------
The information gained from these surveys will assist OPP in setting mew customer service
standards. The OPP workgroup recommends that to set new, effective and realistic customer service
standards for the above service areas, staff and managers must be involved in the process. The
workgroup also emphasizes that the Program must clearly communicate it's expectations to both its
external and internal customers, especially in light of the upcoming reorganization. This will then enhance
the credibility of the organization and its improvement efforts.
Finally, effective and lasting improvements in OPP's ability to provide services to the satisfaction
of its myriad of customers can be attained through the systematic integration of these data into the
priority and budget planning process (i.e., adequate resources), reorganization activities, and other
ongoing streamlining efforts.
39
-------
-------
-------
01
m
-05'
9- ro
CD
W
CD
o
o —
o -o
is a
»'
Q.
ffi
a
a
o'
(Q
CD
Q- Jo
31 w
------- |