TRIANGLE
INSTITUTE
RTI/1864/18-03F
NATIONAL SURVEY OF HOUSEHOLD PESTICIDE USAGE PILOT STUDY:
EXECUTIVE SUMMARY AND OVERVIEW
by
David Budescu
Martin Rosenzweig
Richard Waddell
Research Triangle Institute
Research Triangle Park, NC 27709
Contract No. 68-01-5848
Task 18
Task Manager:
Project Officer:
John Smith
Linda Deluise
Design and Development Branch
Exposure Evaluation Division
Office of Pesticides and Toxic Substances
Environmental Protection Agency
Washington, DC 20460
.•ARCH TRIANGLE PARK, NORTH CAROLINA 27709
-------
RTI/1864/18-03F
NATIONAL SURVEY OF HOUSEHOLD PESTICIDE USAGE PILOT STUDY:
EXECUTIVE SUMMARY AND OVERVIEW
by
David Budescu
Martin Rosenzweig
Richard Waddell
Research Triangle Institute
Research Triangle Park, NC 27709
Contract No. 68-01-5848
Task 18
Task Manager: John Smith
Project Officer: Linda Deluise
Design and Development Branch
Exposure Evaluation Division
Office of Pesticides and Toxic Substances
Environmental Protection Agency
Washington, DC 20460
-------
Disclaimer
This document has been reviewed and approved for publication
by the Office of Testing and Evaluation, Office of Pesticides
and Toxic Substances, U.S. Environmental Protection Agency.
Approval does not signify that the contents necessarily reflect
the views and policies of the Environmental Protection Agency,
nor does the mention of trade names or commercial products
constitute endorsement or recommendation for use.
-------
Table of Contents
EXECUTIVE SUMMARY
I. BACKGROUND 1
II. DATA COLLECTION AND ANALYSIS 1
PROJECT OVERVIEW
1. BACKGROUND 4
1.1 Purpose of the National Household Pesticide Usage
Survey (NHPUS) 4
1.2 Purpose of the Pilot Study of the NHPUS 5
2. DATA COLLECTION AND ANALYSIS 6
2.1 Data Collection Activities 6
2.2 Data Analysis 10
3. FINDINGS 14
3.1 Effectiveness of the Sample Design 14
3.2 Fieldwork 14
3.3 Data Analysis 16
3.4 Recommendations 19
-------
NATIONAL SURVEY OF HOUSEHOLD PESTICIDE USAGE PILOT STUDY:
EXECUTIVE SUMMARY
I. BACKGROUND
The Pilot Study for the National Survey of Household Pesticide
Usage (NHPUS) is in essence a FEASIBILITY STUDY, however the utility of
the data is only limited by the variability due to small samples.
PURPOSE:
To ascertain if quantitative and statistically valid data
on home pesticide use can be collected.
OBJECTIVES:
To evaluate the survey instrument,
To develop and evaluate training and training materials,
To determine response rates,
To develop cost information,
To generate statistical design information for the
National Survey.
To develop and demonstrate basic analytical methodology.
II. DATA COLLECTION AND ANALYSIS
PURPOSE:
To collect data required to decide if the NHPUS is a
feasible project, and if so, to design the National
Survey.
SAMPLE:
Three States — California, Michigan, and Louisiana, each
an independent survey in itself covering a range of
pesticide usage.
States were divided into relatively homogenous pesticide
use regions (coastal areas, mountain, lowlands, etc.).
Grouping by size of locality was also a factor since
needs for major urban areas were expected to be different
from small towns.
Target sample size by State was 240 households. Number
of completed interviews by State: California - 182,
Michigan - 179, and Louisiana - 163 (524 total).
-1-
-------
Limited disaggregation of data is possible; a measure of
the effect of sample size is shown by number of empty
cells in the analysis tables.
FIELD WORK:
Data collection period was October 1980.
Response rate: 80.5%.
Average cost per interview: $23.12. Eighteen (18) field
interviewers and four (4) field supervisors used.
PROBLEMS ENCOUNTERED:
Language problems with potential respondents who spoke
English poorly or not at all; inadequacies with portions
of the training materials; interviewer attrition; incle-
ment weather; and heavy agricultural activity discouraging
participation.
The following data items were difficult to collect:
quantity of pesticides used (41% nonresponse), household
income level (21% nonresponse), and data on recall of
usage.
Information on professional exterminator services was ,
surprisingly sparse.
ANALYSIS:
Standard analysis procedures were generally used. However,
some analyses required use of EPA data tapes including:
(1) the Product Label File,* and (2) the Inverted Chemical-
Registration Number File, which proved extremely useful..
Software was developed to integrate these files into the
analysis.
Only the most basic analysis was undertaken for this
study.
FINDINGS:
The Sample Design: A qualitative review of available
information is reassuring, producing no indication of
problems. Detailed analyses of sample design will be
performed under Task 20.
Fieldwork: An extensive training guide enhanced the
training program by insuring standard training across
sessions/trainers. Problems with training program/
materials were identified and rectified.
-2-
-------
Ability tp_ collect desired data: Contrary to expectation,
respondents were reluctant to take interviewers to storage
sites which raises the possibility of under-reporting.
Respondents felt (probably optimistically) that six
months was the maximum recall period.
Data analysis: Demographic characteristics are easy to
collect, with the exception of income. Questions relating
to frequency of use, safety precautions taken, and equip-
ment used were no problem. Quantity data is not collec-
tible using current methodology.
RECOMMENDATIONS:
Questionnaire: Several question wording and reformatting
changes are suggested. A major change is suggested in
inventory methodology, and consideration of use of tags
to identify containers inventoried by round (particularly
aerosols) and use of some form of weighing machine are
suggested for obtaining quantity data.
Training and Fieldwork: Sessions should be longer. More
specialized and consistent pesticide training needed.
Field supervisors should be trained consistently and at
one session. The number of allowed callbacks should be
increased, possibly to eight.
Sample Design: The sample design is being developed
under Task 20.
Data Analysis: The size of the data file suggests substan-
tial efficiency can be achieved by appropriately dividing
them (subsetting) into more manageable lengths. Tables
seem to contain too many categories and should be reduced
in dimension. Consideration should be given in planning
for additional analysis, e.g., comparisons between domains,
regressions, and so on. Reviewers should be encouraged
to indicate their analytic requirements.
-3-
-------
PROJECT OVERVIEW
1. BACKGROUND
1.1 Purpose of the National Household Pesticide Usage Survey(NHPUS)
*TO PROVIDE STATISTICALLY VALID QUANTITATIVE PESTICIDE USAGE DATA
*TO DEVELOP DEMOGRAPHIC INFORMATION ON USER HOUSEHOLDS
*TO COLLECT SPECIFIC PESTICIDE BY TARGET PEST BY SITE BY QUANTITY
BY FREQUENCY OF USE DATA
*TO FIND USER INFORMATION SOURCES, SAFETY PRECAUTIONS, METHODS OF
USE AND DISPOSAL
The overall objectives of this survey is to provide statistically
valid quantitative pesticide usage data for households in the
United States, state-by-state. This is the first of several nation-
wide surveys to be funded by EPA to provide current quantitative
pesticide usage data on urban/non-farm use sites. This is part of
a general initiative by EPA to provide current quantitative usage
data on all such sites over a three year period on a recurring
basis. We plan to repeat these surveys approximately every three
years in order to maintain relatively current data on the usage of
pesticides on urban/non-farm sites in the United States. This
initiative is designed to complement an initiative by USDA and the
state agricultural universities which generate pesticide usage data
on farm/rural pesticide use sites.
The Household Pesticide Usage Survey," as well as other surveys
undertaken under this general initiative, is to provide pesticide
usage data in the public domain for all users of such information,
including government agencies at the Federal, State and local
levels as well as the private sector. Presently quantitative usage
data are very limited, out-of-date and not reliable for meeting the
needs of most users of such information.
Within EPA, quantitative usage data is needed as a basis for human
exposure and risk analyses, for environmental exposure and risk
analyses, for analyses of the benefits to pesticide users as an
input to risk/ benefit decisionmaking, for program planning and
prioritization, for general monitoring activities, for integrated
pest management education and training programs, for basic research,
and for planning of enforcement activities. Many governmental
units at the Federal, State and local levels from time-to-time have
data needs in one or more of these areas. Private industry and the
general public also need such data to participate in or react to
various regulatory educational and related activities by government
agencies and universities.
The major areas upon which data are to be obtained in the Household
Usage Survey are demographic information on the household, the
-4-
-------
specific pesticide/site/target pest/quantity/frequency data on the
pesticides actually used, user information sources, safety precau-
tions, application technology, container disposal, and unused
chemical disposal. Information is needed on each of these areas by
one or more of the data user groups. I./
1.2 Purpose of the Pilot Study of the NHPUS
*TO EVALUATE THE SURVEY INSTRUMENT
-TO EVALUATE INTERVIEWER TRAINING AND FIELD MANUAL
*TO DETERMINE RESPONSE RATE
*TO DEVELOP INFORMATION ON COSTS AND LENGTH OF INTERVIEW
*TO PROVIDE INFORMATION NEEDED FOR DESIGNING THE NHPUS.
The quality of the data collected from the field in the National
Survey depends largely on the quality of the instrument and the
effectiveness of the procedures used to administer it. In addition
to operationally testing the instrument to be used in the National
Survey, an examination of the feasibility of the entire system of
data collection procedures should be made. A pretest of the data
collection methods and procedures as proposed in the Pilot Study is
the only way the problems of unreliability or invalidity of measure-
ment can be surfaced prior to the collection of data upon which the
National Study conclusions will be drawn. Additionally, the Pilot
Study will provide information needed to identify any deficiencies
in the interviewer training program which could affect the quality
of the data being collected.
1.2.2 Objectives
— f
The objectives of the Pilot Study are to evaluate interviewer
training and to pretest methods and data collection procedures in
preparation for the proposed National Survey. Specific objectives
of the Pilot Study are:
(a) To evaluate the survey instrument. The effectiveness of the
instrument is dependent upon the content and the wording of
the questions. The Pilot Study will provide information
needed to refine the instrument so that it will be as effective
as possible when it is administered in the National Survey.
(b) To evaluate the interviewer training program and the field
manual. Since some of the interviewers to be used in the
National Survey may have little or no interviewing experience,
it is essential to have both an effective training program and
an effective field manual. The identification of problems
encountered during data collection in the pilot study will be
invaluable in providing information that can be used to improve
the training program and the field manual.
^/Memorandum dated March 27, 1980 from the Chief, Economic Analysis
Branch, BFSD, OPTS (TS-768).
-5-
-------
(c) To gain information on response rates. The refusal and non-
contact rates in the Pilot Study will be useful for determining
the need for evening, weekend and call-back interviewing.
Nonresponses in a survey will necessarily cause some bias in
the survey estimates. The level of the bias is dependent upon
the number of nonrespondents and the similarities between the
pesticide use practices of the respondents and the nonrespon-
dents. Obviously, if nonrespondents have different pesticide
use practices than those who respond, estimates computed from
the collected data will be biased. Because there is no way to
determine the similarities between the two groups, the most
effective way to minimize the bias is to obtain high response
rates. Obtaining high response rates generally involves
estensive interviewing. The Pilot Study response rates will
be essential for determining the extent to which follow-up
procedures are needed.
(d) To provide information on time and costs associated with the
collection and processing of data. The administration times
of the interviews in the Pilot Study will be used to compute
an estimate of the average time required to conduct a single
interview. Also, total interviewer time including travel time
between interviews can be evaluated. These time estimates
will aid in the establishment of a time frame for the National
Survey. Additionally, the Pilot Study will provide data on
the costs associated with data collection, data processing,
and data analysis.
(e) To provide estimates of the variability associated with each
stage of sampling. These variance estimates can be used to
evaluate the effectiveness of the sample design and to provide
estimates of sample sizes at each stage of sampling for the
National Survey.2/
2. DATA COLLECTION AND ANALYSIS
2.1 Data Collection Activities
2.1.1 Description of the sample design
Survey samples employ lists called sample frames. These are the
means of identifying the members of the population of interest, the
target population. The sample is selected by drawing at random from the
list. Conceptually, complete and current lists provide the basis for
scientific sampling. It is the selection of sample members according to
chance which provides the probability basis for general statements about
the population from the sample data (called statistical inference).
Unfortunately, lists of all the people living in localities of the
United States do not exist. Consequently, we divide the land area into
small segments and select some of them at random. This is called
area sampling.
2/National Household Pesticide Usage Survey OMB Submission Package.
May 20, 1980. SRDC, Research Triangle Institute, Research Triangle
Park, NC 27709.
-6-
-------
It is generally more convenient not to select these segments
directly, but to first select a larger area, perhaps a State, then a
county, a part of a county (called a County Census Division), and so on.
Each of the successively chosen areas is called a stage. Any sample
with more than one such level is called a multi-stage sample.
The larger units selected at the first stage units, often the
county, are designated as of greater significance, and are called the
primary sampling unit (PSU). Smaller areas within the PSU may be desig-
nated the secondary sampling unit (SSU), and so forth. These sampling
units possess two important characteristics, (i) they exhaust the land
area of the stage in which they lie (e.g., the totality of county lands
exhaust all the land in the State where they lie), and (ii) they do not
overlap.
Another important distinction is made: If several counties are
selected from a State, they are called a stage of sampling, i.e., we
have sampled from among the counties. If all the counties are selected
from a State, the counties are called strata. One has stratified the
sample within the State by taking from every county. This can occur at
any level, or several levels. If we stratify by State, we must sample
from every State. If we take only some States, then States are a stage
of sampling. In practice, the construction of strata may become rather
complex, however the distinction made here does not fail.
The Pilot Study employed a two-stage area sample. The first stage
was stratified by size of locality and topography, i.e., each State was
partitioned into regions hopefully, homogeneous in pesticide use.
Larger, medium and small localities were grouped (stratified) within
each topograhic region. The first stage units (the primary sampling
units) are the Minor Civil Divisions (MCD's) or County Census Divisions
in this study. The second stage of sampling consisted of units called
block groups (which are groups of city blocks) or enumeration districts
(small areas in localities not organized into city blocks). The selected
households were taken from the block groups or enumeration districts
which fell in the sample.
2.1.2 Fieldwork
Data collection took place between October 1 and 31, 1980 in three
States--California, Michigan, and Louisiana. Eighteen (18) field inter-
viewers were used during data collection, supervised by four RTI field
supervisors. Four training sessions, conducted jointly by the field
supervisors and two survey specialists, were held during the latter part
of September. These sessions lasted three days and covered all areas
necessary for proper completion of the survey questionnaires and other
field forms. Representatives from EPA were in attendance at all
sessions.
Lead letters were mailed to sample households before data collection
began. These letters informed the residents of the nature of the survey,
the need for their cooperation, and told them that an interviewer would
be calling at their homes in order to conduct the interview. Most
interviewers reported that when the respondent had read the letter
-7-
-------
before the visit, his/her attitude toward participating in the study was
very positive. A similar letter was also sent to various public officials
in the sample areas. In addition, news releases were distributed to
local radio stations and newspapers providing coverage for the areas.
More field problems were encountered in Louisiana than in the other
Pilot Study States. Unforeseen problems with the training materials
were detected during the Louisiana training, which fortunately was the
first session scheduled. Resolution of these problems was made before
the remaining sessions were begun. There were .interviewer-related
problems in Louisiana, also. Several persons never worked after being
trained, therefore, other interviewers had to work larger case loads
than anticipated, which to some extent, increased field costs. Sample
dispersion and travel conditions in Louisiana posed a problem for call-
backs on nonrespondents as well as increased travel costs considerably.
The primary problem encountered in California was language barrier,
especially in the southern part of the State where 7% of the sample
households were non-English speaking. Weather and apple harvest season
posed the most problems in Michigan. A freak snow storm hindered data
collection for a short period and many potential respondents were too
busy with apple harvest to participate.
It should be noted similar field problems are not uncommon for
household surveys and a relatively successful response rate of 80.5% was
achieved. This rate is quite good considering that only four callbacks
were allowed for nonrespondents. It also is a positive reflection on
the quality of training, efficiency and experience of the field staff,
and proper project planning. Response rates by State were: California —
82.4%, Michigan -- 82.9%, and Louisiana -- 76.2%. The average cost per
interview was $23.12; 524 questionnaires were completed from 727 sample
housing units.
2.1.3 Survey Questionnaire Discussion
Initially, plans for questionnaire development called for the
design of between two and four instruments to be tested in the Pilot
Study. Each questionnaire alternative was to embody a different metho-
dological design and each design option was to test various recall
periods.
The major areas of data which all instruments were designed to
collect were: demographic information on the household, specific pesti-
cide/site/ target pest/quantity/frequency data on pesticides used during
the recall period, user information sources, safety precautions, applica-
tion technology, container disposal, unused chemical disposal, and
target pest/site/frequency/ cost data on pesticides applied by commerical
pest control operators. For purposes of this study, the term "pesticide"
included all categories of pesticides except plant growth regulators,
antitranspirants, and disinfectants without EPA registration numbers.
A series of three pretests were conducted using the various ques-
tionnaire designs until a final questionnaire was developed and submitted
to OMB for review/approval. The primary data collection instrument used
during the Pilot Study was known as the Core Questionnaire and contained
five sections:
-8-
-------
the Household Section — collected basic demographic data on
household members, and obtained information on the kinds of
buildings/outdoor areas maintained on the premises, primary
household ethnicity, and classification of type of residence.
An introduction to the inventory process closed this section.
the Usage Inventory Section -- contained 17 questions concern-
ing all pesticides used during the recall period, which for
the Pilot Study was June 1 - September 30, 1980. Only pesti-
cides for which containers were still on the premises were
listed in this section. A complicated and extremely detailed
question/ matrix concerning indoor/outdoor usage, number of
applications, and site/pest appeared in this section.
the Usage Inventory Section (Pesticides Not on Premises)
this section was almost identical to the preceding section
with the exception of several questions concerning disposal
methods for pesticides/containers.
the Professional Exterminator Section — collected information
on the use of professional exterminators for pest control,
either under contract or by other means.
the Recall and Income Section -- the final section asked the
respondent how long he/she would be able to reasonably recall
usage information. In addition, household income data and
telephone number were obtained and information on respondent
attitudes was requested from the interviewer.
Two secondary data collection instruments were used: (1) the
Continuation Section, which was basically a subset of the Core
Questionnaire (used to record data on more than nine pesticides used
during the recall period), and (2) the Nonusage Inventory Form, which
was used to record basic data on pesticides found on the premises but
not used during the recall period.
Specific recommendations for changes to the survey instruments are
discussed in Section 3.4 of this overview. A general evaluation of the
instruments follows.
2.1.4 Questionnaire Evaluation
Generally speaking, the Core Questionnaire was effective in gather-
ing the required data. Several problem areas have been identified and
plans for correcting these problems will be incorporated in the revisions
in preparation for the National Survey. Each section of the questionnaire
will be covered.
(a) Household Section: Basically, there were no major problems
encountered in this section.
(b) Usage Inventory Section: Many of the questions relating to
pesticide storage and reading of instructions elicited positive
data. However, some problem areas are obvious, such as the
-9-
-------
quantity questions. Forty-one percent of the completed ques-
tionnaires had missing values for these questions. This
largely reflects the frequency of use for aerosols since these
questions were skipped for such products. Also, there is a
problem with reporting by standard units of weight.
Generally, most of the other questions in this section elicited
useful data and caused few field problems or problems in
analyses. Using the Pilot data, it will be possible to elimi-
nate some response options, add others, and streamline and/or
reword some questions in order to better define the objectives.
(c) Usage Inventory Section (Pesticide Not on the Premises): This
section proved to be of little use during the Pilot Study.
Almost 63% of the respondents reported no pesticides used, and
28% reported only one used. For those reporting usage informa-
tion, the quality of the data is questionable because complete
names were not given, registration numbers cannot be obtained,
and other information appears to be quite sketchy.
(d) Professional Exterminator Section: The section presented no
major field problems. The reported data are of some interest,
however. Eighty-two percent of the respondents reported
having no professional exterminator contract and 94% said that
no services were used during the recall period.
(e) Recall and Income Section: The most problematic question in
this section was the income question. Twenty-one percent of
the respondents either refused to answer the question or
reported "Don't Know." The positioning of the question may
have created the problem; however, questions of a similarly
sensitive nature frequently have lower response rates. Ninety
percent of the respondents provided a telephone number upon
request.
The inventory methodology has been an area of concern for some
time. There is some reason to believe that under-reporting may result
because of two principal factors: (1) respondents quickly learn that
the more pesticides identified, the longer the interview, and (2) the
response cards fail to educate respondents adequately. Possible solutions
to the under-reporting may be the use of a screening questionnaire,
better response cards, and illustrations of common pesticide containers.
Other recommendations resulting from the Pilot Study include reducing
the size of the document, including the Nonusage Inventory Form as part
of the Core Questionnaire, eliminate the Continuation Section and simply
use additional questionnaires as needed, and improve instruments within
documents.
2.2 Data Analysis
2.2.1 EPA data files
Certain features of the analysis plan require information that is
collected and maintained by EPA. Copies of two EPA tapes were obtained
by RTI:
-10-
-------
(1) The Product Lable File, which relates each pesticide (identi-
fied by a unique EPA Registration Number) with a formulation
code, a list of active ingredients, and a list of pesticide
classification codes.
(2) The Inverted Chemical-Registration Number File, which relates
each active ingredient to its chemical (not commercial) name,
and a list of pesticides (identified by their EPA Reg. Nos.)
containing the active ingredient.
For the purpose of future surveys, like the National Study, it would be
useful to obtain copies of two additional tapes:
(3) A file relating EPA Registration numbers to commercial names
of pesticides, and
(4) A file relating State Registration Numbers to EPA numbers.
2.2.2 Demographic Characteristics
The reciprocal of the selection probability of any given unit in
the sample is commonly called the sampling weight of the element. The
weight can be regarded as an inflation or expansion factor by which the
aggregate of the sample observations estimate the entire target popula-
tion. Sampling weights were calculated, and adjusted for nonresponse,
for all the observations in the sample. Since each weight represents
the number of households in the State represented by a given observation
in the sample, all the responses from a questionnaire were multiplied by
the appropriate weight in order to obtain estimates for the State. All
reported results are based on these "weighted" responses.
The first part of the analysis consists of a set of tables providing
statistics describing the population sampled in each State. The results
'can be compared with the Census data for the States to evaluate the
weighted sample with respect to certain demographic characteristics.
To complement the control data, similar tables were generated based
on the data from the 1970 U.S. Census. Note, however, that the 1980
Census data when published will be more appropriate for comparison than
the 1970 Census data.
A common pitfall for such comparisons is that some will take them
as proof or disproof of whether or not a sample is "valid" or "represen-
tative." The matching or mismatching with control data on selected
factors should not be interpreted as proof one way or the other as to
how "good" the sample is for estimating the factors under study.
The demographic tables and the Census data can be found in the
Appendix, (Tables 1-4) and a discussion of the results can be found in
the Technical Report (Section 3.3).
-11-
-------
2.2.3 Analysis of type of pesticide
The EPA Pesticide Product File classifies each product according to
its potential use. Products are not uniquely assigned to one category--
60% of them are classified in more than one way. For this purpose, EPA
uses 40 different categories. For the purpose of the present analysis
these categories were combined into four general classes: insecticides,
herbicides, fungicides and others. Additional types of pesticides were
eliminated from the analysis because few, if any, products in this
category were reported.
The use of different types of pesticides was analyzed with respect
to different characteristics of the household (type of residence, income
of family, frequency of pesticide use), the way the pesticides are
stored, the source which recommended purchase of the pesticide and the
site of application and type of pest controlled.
The results of this analysis are presented in tables 5-10 in the
Appendix, and a discussion of the estimates obtained can be found in
Section 3.6 of the Technical Report.
2.2.4 Analysis by active ingredient
The EPA Product File links each pesticide with a list of its active
ingredients. The number of active ingredients listed for the different
products varies from one to 18. For the pesticides reported in the
present study, a total of 161 different active ingredients were identi-
fied. Of those, 107 were reported in California, 84 in Louisiana and
112 in Michigan. The majority of these ingredients was found in a small
fraction of the pesticides listed in the State. Display 1 presents a
list of the ten most prevalent active ingredients. For the purpose of
the Pilot Study only these ten ingredients were analyzed.
Display 1. Active Ingredients Selected for Analysis
Active ingredient EPA number
Ethanol 1501
d-trans-Allethrin 4003
o-Isopropoxyphenyl methylcarbamate 47802
Carbaryl 56801
N-Octyl bicycloheptene dicarboximide 57001
Petrolum distilate, oils, solvent, or hydrocarbons 63503
Piperonyl butoxide 67501
Pyrethrins 69001
2,2-Dichlorovinyl dimethyl phosphate 84001
Resmethrin 97801
-12-
-------
The use of each of the ten active ingredients was analyzed with
respect to some of the household characteristics (type of residence,
family income, household size), and some of the principal user characte-
ristics (sex, age). Also, the frequency of use of any given active
ingredient was analyzed by its storage site, source of recommendation,
formulation, equipment used in application, safety precautions taken at
use, reading and understanding of the instructions on the lable, and
purpose of use (preventive measure or existing problem).
The results of the analysis can be found in tables A, D, and E in
the Appendix and a discussion of their interpretation is included in
Section 3.7 of the Technical Report.
2.2.5 Quantity data analysis
The EPA file lists the percentage of weight each active ingredient
constitutes of the pesticide. Estimates of the quantity of a given
active ingredient used were obtained from the total quantity used in a
given household (as well as the total quantity used indoors and outdoors)
multiplied by this fraction.
Several serious problems were encountered in this stage of the
analysis. First, a considerable number of households did not report
valid quantities. Several factors could have caused this fact--
uncertainty regarding the date of use, inability to estimate quantity
used, or report of quantity in nonstandard units (e.g., squirts).
Second, the units of measurements varied from one pesticide to another—
some were reported as weights (ounces, pounds), and others as volumes
(fluid ounces, pints, quarts, tablespoons, cups). Display 2 represents
the number of pesticides for which valid usage quantities were reported.
Considering the fact that these values are aggregated across the three
States, it appeared that for only one of the
Display 2. Number of Respondents Reporting Valid Quantities
Active
Ingredient
1501
4003
47802
56801
57001
63503
67501
69001
84001
97801
Totals:
All
reports
82
78
169
92
70 .
324
122
95
116
84
1232
"Good"
quantities
21
5
11
78
13
46
22
24
7
0
227
-13-
"Dry"
only
9
4
10
61
10
25
16
17
4
0
156
"Fluid"
only
12
1
1
17
3
21
6
7
3
0
71
-------
ten active ingredients (56801) was the number of pesticides reported
large enough to justify an estimate at the State level.
In estimating quantity used, several assumptions were made:
(1) For all fluid quantities, it was assumed that all the pesti-
cides have the same density (volume/weight ratio) and that
this density is one. It is however recognized that pesticides
vary in their densities.
(2) It was assumed that whenever quantities used were reported in
non-technical units (e.g., tablespoons, cups, etc.), these
values can be converted to ounces of fluid ounces according to
the regular conventions. However, we realize that "a table-
spoon of pesticide x" may not always correspond to 0.5 fluid
ounces of the substance.
(3) It was assumed that all quantities reported reflected use of
the pesticide itself. It should be mentioned, however, that
many pesticides are diluted or dissolved in water, or other
liquids prior to use, and that the ratio of dilution is likely
to vary from one product to another and from one user to
another.
Tables B and C in the Appendix present the average quantity of
Carbaryl (56801) used, as reported in ounces and fluid ounces
respectively.
3. FINDINGS
3.1 Effectiveness of the Sample Design
This analysis will be undertaken in Task 20, however since the
multi-stage area sample involving stratification is a standard design
employed many times before, its characteristics are well known. Conse-
quently, no problems or great inefficiencies are expected. As informa-
tion on various costs and the variability occurring at each stage of
sampling is developed, greater effectiveness can be achieved.
A qualitative review of the data suggests that stratification to
obtain homogeneous pesticide-use regions is effective in reducing vari-
ability. Stratification by size of locality also is important in both
increasing precision and improving coverage.
3.2 Fieldwork
3.2.1 Effectiveness of Training
Generally, the training program proved effective. With a few
exceptions, the topics were covered in training in the same order as
they were performed in the field. One goal of the training was to
provide consistency in all sessions. To achieve this, an extensive
training guide was developed for use by the trainers. This guide
provided a script that could generally be read verbatim. This method
proved most helpful and was highly praised by the field staff during the
sessions and at the debriefing after data collection.
-14-
-------
In addition to the guide, several audio-visual aids were used.
These included a video presentation on general interviewing techniques,
overhead transparancies, and displays (using chalk-boards, tablets,
etc.)- Despite the emphasis placed on consistency, two areas of training
were not consistent from session to session, because a detailed script
was not developed. The first area was the Field Supervisor (FS) training
which was guided only by a prepared agenda. The major topics covered
during the FS training session were a review of the Field Supervisor
(FI) training agenda, a thorough review of the sections assigned to the
FS for FI training, a brief review of the other sections of the FI
manual, and a thorough review of the FS Supplement to the manual. The
second area of inconsistency was the Pest and Pesticide section of the
FI training. Because three different pesticide experts were used for
this portion of the training and because each used his own approach to
the same general subject matter, different areas were emphasized and, in
some cases, different material was introduced.
Because the National Survey is likely to require more than four
training sessions with more than two principal trainers, consistency
will be of paramount importance. Consequently, planning for the National
Survey should include changes in the training that will ensure consistency
in the FS and Pest and Pesticide training areas.
The only other major changes required for the National Survey (in
addition to streamlining many topics) are: 1) Increase the length of
the FI training session to four or five days in order to spend more time
on mock interviews, overcoming objections at the door, re-showing the
video on general interviewing techniques and showing a video on probing
techniques; 2) Re-schedule the edit exercise so its value can be trans-
ferred to the interview situation; and 3) Develop exercises using actual
pesticide containers to teach accurate recording of label and quantity
information.
3.2.2 Ability to Obtain Desired Data
The consensus during questionnaire development was that respondents
would allow the interviewers to go about the premises to identify pesti-
cides. It was on this supposition that the inventory methodology was
developed. However, the Pilot Study statistics clearly show the respon-
dent's preference for bringing the pesticides to the interviewer. The
methodology did not work as anticipated and this raises the possibility
of under-reporting, as stated earlier. To get some measure of under-
reporting, interviewers were instructed to probe at the end of the
interview in order to possibly identify any missed pesticides. Of the
questionnaires reviewed, 68% reported no missed pesticides, 19% reported
one missed, 11% reported two missed, and 1% reported three or more
missed. Therefore, while under-reporting is still a problem, an improved
questionnaire methodology should be implemented to include probing.
Eighty-five percent of the respondents felt that they could recall
information for six months, but the percent dropped significantly for
longer periods, leading one to theorize that a six month recall period
may be the maximum usable without jeopardizing the quality of data.
-15-
-------
The average number of pesticides used was four, thus lessening
concern that during periods of high use (i.e., late spring through early
fall), reporting may be more difficult. However, the data reflect
problems for any recall period in reporting quantity of use. As pre-
viously stated, 41% of the questionnaires had missing values for these
questions. Clearly, this subject should be carefully reviewed and
additional methodologies developed and tested for collecting such
information.
3.2.3 Factors Affecting Response Rate
The primary factors affecting the response rate are:
(a) The inventory methodology. Respondents seem to learn quickly
that the more pesticides they identify, the longer the
interview.
(b) The number of callbacks. All field staff reported that the
response rate could be increased if more callbacks are allowed.
Eight is the number suggested.
(c) Language barriers. Non-English speaking people reduced the
response rate, particularly in Louisiana and Southern
California.
(d) Lead letters arriving too far in advance of FI visit. When
letters arrive too far in advance, their effectiveness is
diminished. It might be better for FIs to mail the letters
directly to the housing units closer to the scheduled field
visit.
(e) Factors beyond normal control. These include weather problems,
field staff illness, etc.
3.3 Data Analysis
3.3.1. Aspects of the analysis that were performed successfully
(1) A prerequisite for most of the analysis was the identification
of the reported pesticides on the EPA files. Of the 1286
pesticides reported on the premises 215 (16.7%) had non-EPA
numbers, and 49 (3.8%) had EPA numbers which were not found on
the EPA tape. These pesticides were not included in the
analyses performed. Hard copy lists of these products were
generated. The vast majority of pesticides listed (1022 =
79.5%) had valid registration numbers which were matched with
EPA files. This high proportion of "matches" demonstrates the
high quality of the data collection effort, and the feasibility
of the proposed method of analysis.
(2) Generally speaking, it was demonstrated that all the tables
involving the demographic variables, and the analysis by type
of pesticides, can be performed successfully without major
difficulties. From the perspective of the present project,
-16-
-------
these aspects of the analysis can be considered satisfactory.
For the purpose of the national study, or other State or
regional efforts, some minor modifications (to be described in
the next section) may prove to be useful in the sense that
they are likely to make the results more meaningful and easier
to interpret.
(3) Tables A, D, E, demonstrate the capability to obtain valid and
meaningful estimates of the frequency of use of various active
ingredients, and to cross-tabulate these estimates by the
safety precautions and equipment used during application and
by the characteristics of the user and the household. We are
particularly pleased with the fact that frequency-of-use data
were obtained for most pesticides reported, and by the fact
that aggregate estimates for active ingredients (for a variety
of pesticides) were obtained.
(4) Although this was riot one of the purposes of the Pilot, we
have attempted to identify the most frequently used pesticides.
A list of the 18 most popular pesticides can be found in
Display 3. Note that 10 pesticides account for 28.5% of the
total reported, and the first 18 account for 37.8% of the
total.
3.3.2 Aspects of the analysis which require slight modifications in the
methodology
(1) A relatively good correspondence was obtained between the
demographic data and the Census information for the States.
The accuracy of these comparisons can be further improved by
using data from the recently completed 1980 Census, and by
changing the reporting categories of these variables to better
match the categories used by the Census Bureau.
(2) A problem in all the tables of the analysis by type of pesti-
cide is the large number of cells with zero or missing entries.
This fact indicates that there is a tradeoff between the
sample size and the level of detail that can be achieved in
the analysis. The number of categories used in these tables
should be reduced. This should improve the quality of all the
tables, and in particular of tables 2 and 8 which contain the
analysis of pest controlled by site of application by type of
pesticides - pests should be collapsed into 5 to 7 categories
and sites of applications should be combined to yield four to
five categories.
(3) The active ingredient analysis suffers from a similar problem.
First the number of active ingredients used in the analysis
should be reduced to two or four. The choice should be based
on the frequency of report in the present study, and chemical
considerations, and should be made in close cooperation with
EPA. Second, the number of categories used for the reporting
variables should be reduced. As a rule of thumb the number of
categories for each variable should be kept under seven.
-17-
-------
Display 3. The Most Frequently Reported Pesticides
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Reg. No.
4822-111
777-53
4822-10
475-136
1730-36
432-536
4822-84
5813-1
3573-2
4822-38
239-565
239-2346
777-10
777-25
4822-136
239-739
475-156
777-29
Name
Raid Ant & Bug Spray
Lysol Spray
Off
Black Flag Ant & Roach Spray
Pine Sol
Raid House & Garden Spray Formula III
Raid Professional Strength
Clorox
Comet Chlorinol
Raid House & Garden Spray
Ortho Tomato & Vegetable Dust
Ortho Rose & Floral Dust
Lysol Regular
Lysol De-odorizing Cleaner
Raid House & Garden Spray Formula II
Ortho Malathion 50 Insect Spray
Black Flag Insect Spray
Lysol Toilet Bowl Cleaner
Frequency
of report
5.28%
4.99%
3.43%
2.86%
2.45%
2.35%
2.06%
1.96%
1-76%
1-47%
1.27%
1.17%
1.17%
1.17%
1.17%
1.08%
1.08%
1.08%
3.3.3 Aspects of the analysis which require major changes in methodology.
(1) EPA plans to conduct a national survey of use of pesticide
control operators. Although the focus of the present study
was to investigate nonprofessional use of pesticides some
information was collected about services provided by profes-
sional operators. For each State, estimates of the number of
households which have a contact with such operators were
attempted, and the average cost per contract found. The
majority of households (80% in CA, 77% in LA, and 89% in MI)
did not report contracts and many of those who have could not
remember, or did not report, their costs. Given the small
sample sizes and the small proportion of household reporting
costs it seems that valid estimates of these parameters must
be obtained by other means. In addition, we have tried to
estimate the number of households using professional services
on a non-contract basis by the pest controlled and the site of
application. Only 51 questionnaires reported the type of pest
controlled. Of those about two thirds were roaches, and no
other pest was reported by more than 4 households. The large
majority of households interviewed (91% in CA, 92% in LA, and
99% in MI) did not use any professional services. Since the
estimated number of households using different types of
services (i.e., different combinations of site of application
and pest controlled), and the estimated costs are based on a
very small number of observation, it is recommended that this
information be obtained by other means.
-18-
-------
(2) Most tables were based on the pesticides on premises that have
been used between June and September 1980. However, data were
collected about the number of households storing pesticides on
the premises but not using them, and the number of households
reporting having used pesticides and discarding the container
prior to the date of the interview. Only 258 reports of
pesticides not on the premises were recorded. Since in these
instances EPA Reg. Nos. are not available, and information
about the type of the pesticide, active ingredients, etc.,
cannot be obtained, these observations are of little relevance.
Also, we suspect that a recall problem exists in these cases.
It is recommended discontinuing this part of the questionnaire,
unless the National Study includes periodic visits to the
sample households. This option promises more accurate follow
up on the use of pesticides.
(3) Overall, 556 reports of pesticides were listed in the three
States as being on the premises, but not being used. One
hundred and sixty-seven of them (30.0%) did not have EPA
registration numbers, (either non-EPA numbers or no number
reported), and 326 of the reported pesticides (58.6%) were
identified in the EPA files. At the present time, these
pesticides have not been used in the analysis in any signifi-
cant way. This situation is likely to change only if the
National Study includes repeated visits to the same households
at fixed intervals. However, if this option is not be included
in the design, we see no major use for these data and recommend
dropping this part of the instrument.
(4) As shown in Display 2, RTI has encountered serious problems in
obtaining data for quantity estimation. When valid responses
were obtained the estimation procedure required many assump-
tions. It is our conclusion that the methodology used will
fail to generate valid estimates of quantities used and it
should be replaced by an alternative one. One possibility is
tagging and weighing opaque containers, particularly aerosols,
on a periodic basis.
3.4 Recommendations
3.4.1 Survey Instrument
The following changes are suggested for the .Core Questionnaire.
Item Number Recommended Change
4 Add reference dates. Explain "maintain."
5 Put at end of questionnaire.
6 Put at end of questionnaire. Change categories from
"Row house/garden apartment" to "House or apartment
with direct ground level access" and "Apartment house
(high rise building or housing unit without direct
-19-
-------
ground level access)" to "Apartment or house without
direct ground level access."
7a Allow standard abbreviations.
8 Needs to be added to Nonusage Inventory Form.
9 Write registration number on same line as EPA, USDA,
or State.
10 Since no computer file exists which links brand
name to formulation, may be deleted.
11 Put in question form.
12 Change wording to "How did you learn about this
product?"
13 Preface question with "As you know, labels are
sometimes hard to read."
14 Change wording to "Were the instructions clear?"
Add DK to response choices.
15 • Add "Pets" to indoor and outdoor site codes; add
"Germs" to indoor pest list; allow multiple site
codes; possibly delete question on number of times
the product was applied; let interviewers code pests
rather than having them preceded down the page;
repeat question on p.9.
I6a Change wording to "Who used or applied this product
most often? Was it a male or female? How old is
(he/she)?" Allow multiple responses if applied equally
by more than one person.
I6b Change wording to "Did you use anything other than
the original container to apply this pesticide?"
If yes, "What?"
I6c Change wording to "Were any safety precautions used
when this product was applied?" If yes, "what were
they?"
17 Revise with three waves of data collection in mind.
18 Delete this section. Possibly replace with one
general question.
26 Ask separate questions for respondent contracts,
landlord contracts, and community services such as
rodent control; differentiate between a contract
and a request for service without a contract.
-20-
-------
3.4.2 Training and Fieldwork
The following are recommendations for changes to the training and field
work.
1. Conduct a centralized training session for the field supervisors.
2. Conduct five-day training sessions for inexperienced interviewers
and four-day sessions for experienced interviewers.
3. Administer more home study quizzes before training session to
reinforce contacting/locating the respondent, recording conventions,
questionnaire specifications, and pests and pesticides. This is
especially crucial for experienced interviewers missing the first
day of training which covers contacting/locating, production reporting,
and standard recording conventions.
4. Develop more consistent pest and pesticide training.
5. Increase time spent on reading pesticide labels.
6. Spend more time on overcoming objections at the door using mini-mock
interviews.
7. Begin mock interviews at 9:00 a.m. so interviewers will feel fresh
at this time.
8. First mock interview should be conducted as a demonstration using
the trainers as interviewer and respondent.
9. Develop 5 or 6 other mock interviews.
10. If training sessions are large, split group into triads for mock
interviews.
11. Have trainer stay an extra day in each site, if possible, to assist
interviewers who seem to be having problems.
12. Train interviewers in probing techniques using the video tape
developed by Mathematica and mini-mock exercises.
13. Simplify and streamline training on production reporting.
14. Schedule the in-house edit exercise at a more appropriate time
then at the end of the training session. It might be good to do
a second mock interview between the trainers having the Field
Interviewers coding the responses. They could then edit those
questionnaires.
15. Conduct a separate training session for Rounds 2, 3, and 4 of data
collection. The session should be scheduled just before the start
of Round 2.
-21-
-------
16. Allow more than four callbacks.
3.4.3 Sample Design Recommendations
Complete recommendations regarding the sample design for the NHPUS
are being developed in Task 20, which is a continuation of this project.
Those activities are described both in the Task 20 Workplan and in
evaluation of the alternatives specified in the letter from John Smith
to Frank Potter dated January 1, 1981. However, some observations based
on the analysis of the Pilot Study date can be made.
*THe stratification used achieved gains in precision
Stratifying the sample on size of locality which provides some
control on population density reduced variability by 40 or 50 percent in
some cases. It is also likely that topographic stratification to isolate
regions relatively homogeneous in pesticide requirements produced addi-
tional gains in precision, however, these data are not yet analyzed.
*The effect of sample size on the ability to construct complex
tables is clear: small samples mean zero cell numbers
Tables that have either many levels for each factor, or have many
factors, as:
Family Size
12345678.
0-2999
3000-4999
5000-6999
Income 7000-8999
9000-10999
11000-12999
or
State x Pesticide x Site x Age of User x Sex of User x ... ,
have many cells. To fill them will require large sample sizes.
For example, (5 States) x (10 sites) x (4 age groups) x (2 sexes)
x (3 frequency of use categories) x (10 target pests) = 12000 cells.
The usual rule-of-thumb is 5 observations per cell, so here we are
talking about a sample of 5 x 12000 = 60,000 households. The lesson is:
Small samples provide for small tables, large or complex tables require
large samples.
3.4.4 Data Anaysis
(1) In order to increase the precision and the validity of the
estimates, a serious effort should be made to obtain from EPA,
or other sources, files linking State Registration Numbers to
EPA numbers and, thus, to active ingredients and type codes.
-22-
-------
(2) In order to (a) increase the efficiency of the analysis and
(b) reduce its cost, the number of categories of the variables
of interest should be reduced and, if possible, be kept under
seven.
(3) A small number of active ingredients, pesticides and types of
pesticides of interest should be identified prior to the
analysis.
(4) The experience of the Pilot Study has shown that it is not
practical nor efficient to work with the large raw data file.
Instead, the data set should be subsetted into smaller files
which correspond, in terms of the information they contain, to
the different facets of the analysis. It is recommended that
prior to the analysis the content of these data sets be decided,
and software capabilities for merging and/or combining the
different files be developed.
(5) Further recommendations will be generated based upon the first
part of Task 20.
-23-
------- |