imm' —~ n_ _jn_ * * "DEPARTMENT i^Lr * V'GOvCiShmEhT Of THE
Vtrr^ ¦ CLEANOF ENEBGYA |SDiST(5lCTQfCCX0MfiJA
United States iCltVL/W ¦ GWROMOT DC MURIEL BOWSER, MAY
Environmental Protection ' 1 1
Agency
CURBSIDE DISPOSAL EDUCATION
CAMPAIGN PILOT: CASE STUDY
MAY 2022
EPA-842-R-22-004
-------
TABLE OF CONTENTS
Executive Summary
Acknowledqements
Introduction and Project Scope
Methodoloqv of Data Collection
Project Results
3
4
5
12
20
Survey Data Analysis
20
District 311 Request Analysis
28
Further Data Collection Context
34
Focus Group Takeaways
40
Limitations
44
Replicabilitv and Recommendations
Recommendations
46
46
Proiect Checklist
48
Cost Estimation
49
Citations
51
Appendix
53
Appendix A. Sticker Distribution Materials
53
Appendix B. Data Collection Route and Sticker Distribution Maps
55
Appendix C. Additional Images Taken During Data Collection
58
Appendix D. Initial Data Analysis
59
-------
EXECUTIVE SUMMARY
The Curbside Disposal Education Campaign Pilot took place from July 2020 to May
2021 and was rolled out in Washington, D.C., through a partnership between the EPA's
Trash Free Waters Program and the local District government, including the D.C.
Mayor's Office of the Clean City, D.C. Department of Public Works, and D.C.
Department of Energy and Environment. The primary goal of this initiative was to
educate residents about proper waste containment and encourage behavioral changes
to reduce unintentional leakage associated with curbside municipal trash collection.
A total of 8,000 DPW-serviced, single-family homes in four target neighborhoods were
selected to receive a campaign sticker, which was accompanied by material explaining
the purpose of the campaign and how to apply the sticker to their municipal trash cans
for a point-of-contact reminder about best practices. Average litter scores were
measured by project partners on a weekly basis for 22 weeks along 1 -mile
representative routes in each of the four neighborhoods. To assess impact
measurement, litter scoring was conducted for the 11 weeks leading up to sticker
distribution and the 11 weeks following distribution. Project partners also used these
representative routes to collect weekly data on compliance with the four specific
recommendations outlined on the stickers. For comparison purposes, the above data
collection took place along control blocks within each of the target neighborhoods
consisting of households that did not receive a campaign sticker. After sticker
distribution in the treatment area, the total number of stickers applied to cans along
routes was also counted. An analysis suggests that although the improvements
tended to be small to moderate, this educational program had an overall positive
impact on the target communities. In particular, there was a statistically
significant reduction overflowing cans and overflowing and open cans combined
across all neighborhoods.
The intention of this study is to provide important findings and recommendations to
inform successful adaptation and adoption of the Curbside Disposal Education
Campaign Pilot approach in other interested communities.
3
-------
ACKNOWLEDGEMENTS
We would like to give a special thanks to the below individuals, all of whom played a
critical role in the successful planning and implementation of the Curbside Disposal
Education Pilot.
Washington, D.C. Government
Julie Lawson, Director, D.C. Mayor's Office of the Clean City
Lorena Kowalewski, Associate Director, D.C. Mayor's Office of the Clean City
Matthew Robinson, Environmental Protection Specialist, D.C. Department of Energy
and Environment
Katherine Antos, Senior Policy Advisor, D.C. Department of Energy and Environment
Annie White, Office of Waste Diversion Manager, D.C. Department of Public Works
Tim Harwood, Senior Data Visualization Analyst, Office of the Chief Technology Officer
Michael Bentivegna, Data Team Program Manager, Office of the Chief Technology
Officer
Tomash Bukowiecki, Data Scientist, Office of the Chief Technology Officer
National Environmental Education Foundation (NEEF)
Nick Bradford, Program Director or Research & Innovation
Sara Espinoza, Vice President of Programs
Amy Skalmusky, Vice President of Marketing & Communications
Industrial Economics (lEc)
Greg Englehart, Associate
Eric Ruder, Principal
Others
Randy Hartmann, Senior Director of Affiliate Operations, Keep America Beautiful
Katie Register, Executive Director, Clean Virginia Waterways of Longwood University
This project was funded by the U.S. EPA's Trash Free Waters Program.
For any questions, please contact Layne Marshall at Marshall.Layne@epa.gov.
4
-------
INTRODUCTION AND PROJECT SCOPE
Some research suggests that trash spillage resulting from lax residential curbside
disposal practices is potentially a significant contributor to pollution of our waterways.
EPA's Trash Free Waters (TFW) program was interested in exploring how this issue
might be addressed through a low-cost public education campaign with a municipality
as a potential pilot for other municipalities to learn from. The TFW program approached
the Washington, D.C. Mayor's Office of the Clean City (MOCC), the Department of
Public Works (DPW), and the Department of Energy and Environment (DOEE) about a
potential campaign. District employees indicated that unintentional spillage was indeed
an issue, and therefore agreed to partner on this campaign.
Several common trash disposal practices can increase leakage of household trash and
recyclables into the environment due to factors such as animals eating holes in trash
bags and wind blowing trash out of cans. These factors include:
a) Residents not closing trash can lids;
b) Residents not bagging the trash they put in trash cans;
c) Residents setting trash cans outside too far in advance of collection times and
therefore increasing exposure time to factors that may cause spillage;
d) Residents using trash cans with no lids or that are otherwise damaged.
The TFW program and District Government decided to address these behaviors through
a public education campaign.
Sticker Design and Messaging
The TFW program and District government partners started working with the National
Environmental Education Foundation (NEEF) to brainstorm campaign messaging,
format, and design that Washington, D.C. (and eventually other municipalities) could
implement without exorbitant cost and staffing needs. The group determined that an
eye-catching and informative sticker placed on curbside trash can lids would serve as
the best method of message delivery. The sticker served as a point-of-contact reminder
about proper set-out behaviors that could be easily referred to. The stickers were 12.5
inches x 4.625 inches and were designed to fit on the lid of District-provided trash cans
(both small cans for semi-weekly pickup and larger cans for weekly pickup).
The campaign sticker (see Figure 1) articulated four simple actions to reduce
unintentional trash spillage associated with curbside disposal:
5
-------
1) Keep your lid closed, and don't overflow the can.
2) Bag your trash before putting it in the can.
3) Put trash outside shortly before pickup.
4) Request repairs or replacements by calling 311.
Cleaner communities
and waterways
start here
lid closed and do not overfill the can.
^ KeeP
JU Bag your trash before putting it in the can.
(Do not bag recyclables; place items loose in your blue can.)
(H Place trash in can outside shortly before pickup.
0
Call 311 or visit 31 l.dc.gov for assistance
with cans needing repair or replacement.
"TrashFrooDC
cloancity.dc.gov
zerowasto.dc.gov
H CIT Y I
DC —"
L/w ^i-
Y~m ; fcVGOVSRTNMEl
KUL?
In partnership with the U.S. EPA's Trash Free Waters program
www.epa.gov/trash-free-waters
Figure 1. The Curbside Disposal Education Pilot Project campaign sticker design, distributed to 8,000 District
households.
The chosen behavioral messages were developed in accordance with Community-
Based Social Marketing principles that have a proven record of effectiveness. For
example, behaviors were framed positively rather than negatively (e.g., "Put trash
outside shortly before pickup" rather than "Don't put your trash outside too early").
These behaviors were identified through both research and first-hand eye-witness
accounts by District Government employees as to what they perceived as the most
important problem behaviors to address.
The behavioral messages were written in plain language to be as direct and
straightforward as possible. The logo of each District Government partner was included
so recipients of the sticker could know this material was coming from credible local
government sources in partnership with EPA's TFW program.
In addition to the behavioral messages, the slogan "Cleaner communities and
waterways start here" (with an arrow pointing to a picture of a trash can) was chosen to
connect clean, healthy neighborhoods and nearby waterways and appeal to local pride
and a sense of community. Finally, ancillary information (including the parenthetical
about refraining from bagging recyclables and directing recipients to District webpages
and #TrashFreeDC for more information) was added.
6
-------
The sticker purposefully used a similar font and color palette to existing District
campaigns around waste management and littering that had brand recognition with
residents (Trash Free DC, Zero Waste DC, and Not in My DC) to increase the credibility
of the messages and identification with other trash-related campaigns.
Sticker Logistics and Distribution
Eight thousand stickers were disseminated to Department of Public Works-serviced
single-family homes in four pre-selected neighborhoods: Brightwood, ParkView,
Rosedale/Kingman Park, and Trinidad (See Figure 2). Each of these high-density
neighborhoods received roughly 2,000 stickers. See Appendix B for more detailed maps
denoting the estimated boundaries of sticker distribution in the four target
neighborhoods.
7
-------
The four target neighborhoods were selected based on the three criteria below:
1. Pre-existing waste management issues.
2. Prevalence of high-density single-family homes (District government partners
were not interested in targeting residents of apartment buildings or businesses.
Larger detached homes took up too much geographical spread to make data
collection efficient).
3. Representative population size (to aid with expansion and/or replicability in other
communities).
Staff and coronavirus-related limitations prevented District staff and campaign
volunteers from having direct conversations with residents regarding the campaign
goals and ensuring that all stickers were actually applied to residents' trash can lids.
Because direct verbal interaction was not feasible, the stickers were placed into eye-
catching cardstock presentation cards. These presentation cards identified the problem
of trash in the environment associated with curbside disposal behaviors. The cards also
encouraged residents to apply the sticker to their trash can lids as a reminder of best
practices (see Figure 3). A simple set of directions was added to the back of the sticker
to reduce confusion about specific placement on trash can lids (See Figure 4).
8
-------
Hello, Neighbor!
We know you want to keep your community and
local waterways clean, and this free sticker can help.
Please place it on your green trash can lid as a reminder
about how to properly dispose of trash.
Mayor Bowser's Office of the Clean City has partnered with the
U.S. Environmental Protection Agency's Trash Free Waters Program
to reduce the amount of trash falling onto our neighborhood streets
and sidewalks. Trash that spills on streets and sidewal ks attracts rats
and other pests. It can also be carried by water or wind into our
sewers and ultimately wind up in local rivers and streams.
The attached sticker lists a few simple steps you can take to help
keep your local community and waterways free from trash.
To read more about how you can help keep your community clean,
please visit the Mayor's Office of the Clean City at deancity.dc.gov
or zerowaste.dc.gov, and follow *TrashFreeDC on social media.
Figure 3. Presentation card with slots to hold the educational sticker in place.
fBi
DCi
• 01 IM
JHlWITttCTOF LtXUh*BlA
MURIEL BOWSER, MAYOR
9
-------
How to apply your sticker:
O Clean: Clean the lid of the trash can with a mild cleaner
and allow to dry.
Peel: Carefully peel off the backing from this sticker.
O Apply: Place your decal on the lid of the trash can. Start at
one end, guiding the sticker to slowly fall into position.
O Flatten: Use a squeegee or credit card to remove
the air bubbles.
Figure 4. Graphic on the back of the sticker with directions on how to properly apply the sticker on a can.
District Government staff), Advisory Neighborhood Commissioners (ANC) and
community volunteers (i.e., Adopt-A-Block groups) all pitched in to help distribute the
8,000 stickers to residents in the select neighborhoods. Volunteers were given a project
overview and walk sheets (map of routes/addresses) and directed to deliver the
materials to residents in the select neighborhoods by leaving the presentation card and
sticker between the porch door and front door or in the door jam of each home. This
process took a little over one week.
Additional Campaign Messaging
In addition to the 8,000 stickers directly distributed to households in the four targeted
neighborhoods, project partners also disseminated campaign messaging via social
media and a District Government landing page. Through this approach, District
residents who did not receive the campaign sticker were provided an opportunity to
learn about best curbside disposal practices.
An array of District Government social media accounts - the Mayor's Office of the Clean
City, D.C. Department of Public Works, D.C. Department of Energy and Environment,
and Zero Waste D.C. - helped spread the recommended disposal behaviors via Twitter
and Facebook. Below are some example posts.
10
-------
r
Call 311 or visit 311.dc.gov
to request residential bulk
collection or for assistance
with cans needing repair
or replacement.
-sissr iawDC ^
Help us keep our
city trash free!
1. Keep garbage can lid closed and do not overfill the can.
2. Bag your trash before putting it in the can.
(Do not bag recydables; place items loose in your blue can.)
3. Place trash in can outside shortly before pickup.
4. Call 311 or visit 3t1.dc.gov for assistance with cans needing repair/replacement.
6*—-
HMtow lCmumslbc
Cleaner communities
and waterways
start here
In partnership with the U.S. EPA's Trash Free Waters program
Miou
CCmi
|GOVS8»MNT Of THE
MUmK>WE?HAVOO
Figure 5. Examples of social media posts shared by District Government accounts during the campaign.
The D.C. Mayor's Office website featured a short overview of the pilot project along with
a more comprehensive description of the four recommended disposal behaviors for
District residents. The webpage also encouraged citizens who did not receive a sticker
to contact District staff about printing and distributing stickers for their neighborhoods'
trash cans. The website can be viewed at:
https://communitvaffairs.dc.gov/paqe/cleaner-communities-and-waterwavs.
11
-------
METHODOLOGY OF DATA COLLECTION
The most critical element of a pilot project is testing the campaign's effectiveness and
then determining what modifications need to be made before expanding it to a larger
population or a different community. To measure the success of the Curbside Disposal
Education Pilot, two MOCC staff members collected data on a weekly basis along 1 -
mile representative routes in each target neighborhood (Brightwood, ParkView,
Rosedale/Kingman Park, and Trinidad).
These 1-mile representative walk routes were determined by selecting areas with a high
prevalence of single-family homes and service alleyways. Staff optimized effort by
avoiding streets with large apartment complexes and commercial spaces and instead
emphasizing how many alleyways could be walked within the short distance.
Approximately 1,022 households along the four representative data collection routes
made up the "treatment group" and received a sticker. Control blocks were determined
within each of the four target neighborhoods and consisted of approximately 285 homes
that would not receive a sticker. Control blocks were typically at the start or end of each
representative data collection route to ensure simple separation during data analysis.
See Appendix B for more detailed maps denoting the 1-mile data collection routes in
each of the four target neighborhoods, as well as maps showcasing the overlap
between data collection route and treatment group.
of each neighborhood can be found in Appendix B.
12
-------
Generally, from August 17 to November 13, 2020, MOCC staff members walked each of
these four routes once a week to observe trash spillage prior to the launch of the sticker
campaign. From November 16, 2020, to February 5, 2021, the MOCC staff walked the
designated routes to collect data after the stickers were disseminated. The Brightwood
and ParkView neighborhoods routes were walked the day before trash collection, and
the Rosedale/Kingman and Trinidad neighborhood routes were walked the day after
trash collection. It is important to note that several holidays impacted the DPW service
schedule during the data collection period, meaning residential curbside pickup would
"slide" to a day or two later than usual. To limit the holiday variables, District staff shifted
their routine walk to ensure that the time between data collection and trash pickup was
consistent with the typical schedule. Another variable the holiday schedule adjustments
may have introduced is related to the size of the trash cans. Some of the neighborhoods
had a semiweekly pickup, and therefore, they had smaller trash cans. Homes on a
weekly pickup schedule typically had "super cans," which allow for a greater volume of
trash. It is likely that holiday slides impacted households with a semiweekly pickup
schedule more than homes serviced on a weekly basis because the smaller can volume
could translate to more overflow.
For each block along the four 1-mile representative routes, data were collected based
on six indicators:
1) Litter index score
2) Total number of cans
3) Number of overflowing cans
4) Number of open cans
5) Number of cans with a campaign sticker applied to the lid
6) Additional information such as photographs or notes of significant issues
These data indicators are explained in more detail in the following section.
Data Indicator 1: Litter Index Score
Assessing the level of litter, on a scale of 1 to 4, along 1-mile representative routes in
each neighborhood.
To determine if the campaign stickers and educational material led to a reduction of
alleyway litter, MOCC staff members conducted observational litter surveys for eleven
weeks prior to sticker distribution and conducted another eleven weeks of litter surveys
after the stickers were distributed.
13
-------
Litter levels were determined using a rapid visual assessment protocol similar to the
Visual On-Land Trash Assessment for Stormwater. Alleyways and street fronts were
evaluated on a 1 to 4 scale, with 1 meaning there was practically no spillage on the
street, and an individual could pick up any litter under 5 minutes, and 4 meaning there
was a great deal of litter on the street which would require professionals to clean up.
Any overflow trash placed on the ground outside of a can because it could not fit inside
was incorporated into the litter index score. Examples of different alley conditions and
their respective assigned litter levels are depicted below.
Level 1 of the Litter Index Score
Effort required to clean: One person could complete under 5 minutes as a walk-by
pickup effort.
14
-------
Level 2 of the Litter Index Score
Effort required to clean: Two+ people could complete during a dedicated pickup effort.
Level 3 of the Litter Index Score
Effort required to clean: A concentrated community cleanup event.
15
-------
Level 4 of the Litter Index Score
Effort required to clean: A team of professionals from the city would be needed to be
called to efficiently clean up the debris.
Data Indicators 2-4: Counting Cans
District staff collected weekly data regarding compliance with several sticker
recommendations by counting the below indicators on each block,
A) Totai number of cans
The total number of cans was counted on each block to determine if more
residents properly stored their cans before and after collection. This indicator
provided insight into the sticker recommendation "Place trash in can outside
shortly before pickup."
B) Number of overflowing cans
A can was quantified as "overflowing" when trash was "piled up like an ice cream
cone" so high it was impossible to shut the lid properly. This indicator provided
insight into the sticker recommendation, "Keep your lid closed and don't overflow
the can."
16
-------
C) Number of open cans
A can was quantified as "open" when the can could be closed securely with the
lid down but was not. This indicator provided insight into the sticker
recommendation "Keep your lid closed and don't overflow the can."
17
-------
Data indicator 5: Number of cans with a campaign sticker applied to the lid
Assessing the prevalence of campaign stickers.
For the eleven weeks following the distribution of the stickers and educational material,
the number of observable trash can campaign stickers was counted weekly. This metric
was later used to calculate the percentage of cans with stickers.
Data Indicator 6: Additional Information
Sharing supplementary details and photo evidence.
District staff also made note of any significant issues like illegal dumping/ bulk,
construction debris, or the presence of rodents. Photos were taken intermittently
throughout each route to provide further insight on alleyway conditions.
18
-------
Project partners were unable to collect and analyze data related to one of the four
sticker recommendations: "Bag your trash before putting it in your can." While legality
was not a concern because the District owns the cans provided to D.C. DPW-serviced
homes, this assessment would be extremely time consuming and make the data
collection team more noticeable to residents (which could influence resident behaviors
and impact campaign results). Therefore, District staff could not collect data to provide
insight into the campaign's effect on this specific behavior.
19
-------
PROJECT RESULTS
Survey Data Analysis
An analysis was conducted to measure the impacts of the Curbside Disposal Education
Pilot Project on curbside trash spillage. As explained in the preceding section, data
were collected in the four target neighborhoods for eleven weeks prior to distribution of
the campaign stickers and educational material. Then data were collected for another
eleven weeks following campaign material distribution. Metrics included in the survey
data analysis are as follows:
1) Litter index score
2) Total number of cans
3) Number of overflowing cans
4) Number of open cans
5) Number of cans with a campaign sticker applied to the lid.
For each of the four neighborhoods, the weekly collected data points for the five metrics
listed above were compiled from August 17 to November 13, 2020. Then again from
November 16, 2020, to February 5, 2021, to provide a comparison before and after the
stickers and educational material were distributed. Of course, Metric 5 data were only
collected after the stickers were distributed.
Metrics 1, 3, and 4: Assessing Litter Levels and Unsecured Cans
The first part of the analysis, below, looks specifically at Metrics 1, 3, and 4. Metrics 3
and 4 - overflowing and open cans - were collected individually but combined during
analysis to provide a broad overview of the issues of unsecured trash can lids.
Project partners compared the neighborhoods that received stickers (i.e., treatment
group) to those that did not receive stickers (i.e., control group) to evaluate the
differences in these groups before and after the treatment inflection point of November
2020 - when stickers were distributed. Since there were existing differences between
the groups in the pre-treatment period, project partners used a difference-in-differences
regression analysis to determine the program's effects more accurately. The difference-
in-differences analysis method accounts for differences between observed groups prior
to treatment and controls for these differences when determining treatment effects. For
the purpose of this analysis, project partners limited these regressions only to alleys, as
there were very few instances where cans were serviced along a street front. Where
cans along street fronts were present, project partners found the impacts limited and
generally not reflective of where stickers were placed. For each model, project partners
ran two separate versions, one including location-specific fixed effects (to offset any
unexplained variation across sites) and one with no such effects.
20
-------
Our analysis suggests that the program had an overall positive impact, although the
improvements tended to be small to moderate. Using the difference-in-differences
regression analysis, project partners found improvements in 13 of the 16 measured
metrics (Metric 1, Metric 3, Metric 4, and Metric 3 and 4 combined, across four
neighborhoods) (See Figure 7). Five of these findings were statistically significant at the
90% confidence level (one of the five was only significant in the fixed-effects model).
Project partners also found significant negative effects in the number of overflowing
cans across all neighborhoods and the combined number of overflowing and open cans.
Simplified regression results are available below in terms of magnitude of change,
direction of change, and whether the change was statistically significant or not (Figure
7). A table of all findings related to this sub-analysis, including those not statistically
significant, can be found in Appendix D.
Neighborhood
Metric
Magnitude
Direction
Significant
Litter Index
-0.395
1
Yes
Brightwood
Score
Brightwood
Open Cans
-1.286
1
Yes
Overflowing +
-1.640
1
Yes
Brightwood
Open Cans
Overflowing
-2.616
1
Yes*
Park View
Cans
Overflowing +
-3.769
1
Yes
Park View
Open Cans
Combined (All
Overflowing
-1.364
1
Yes
Neighborhoods)
Cans
Combined (All
Overflowing +
-2.159
1
Yes
Neighborhoods)
Open Cans
Figure 7. Table of statistically significant findings from difference-in-differences regression analysis for average litter
index score and number of overflowing and/or open cans observed during data collection before and after treatment.
(*=Only statistically significant in the fixed effects model).
Metrics 2 and 5: Total Number of Cans and Cans with Campaign Sticker
Project partners also used a difference-in-differences regression analysis like the above
on Metrics 2 and 5 to assess the program's effects on the number of cans and the
number of cans observed with the campaign sticker applied along each neighborhood's
data collection route. There was no statistically significant change in the total number of
cans counted - a metric used to provide insight on if the campaign successfully
encouraged residents to properly stored their cans before and after collection (i.e.,
placed/returned the cans from the curbside promptly). A total of 109 maximum stickers
21
-------
were counted by project partners along the representative neighborhood routes. The
maximum number of stickers counted for each block in any given week was compiled to
account for variability (i.e., it was likely that data collection walk-by timing did not
universally coincide with can placement out on the curb, so the maximum number of
stickers observed serves as an indicator of campaign reach rate).
Can Total
Number of
Households
Maximum
# of
Stickers
Neighborhood
Magnitude
Direction
Significant
Treatment
Control
Brightwood
0.0596
T
No
338
28
29
Park View
1.0264
T
No
212
85
30
Rosedale
Kingman
-1.429**
1
No
218
102
26
Trinidad
-7.487
1
No
254
70
24
Combined (All
Neighborhoods)
-3.536
1
No
1,022
285
109
Figure 8. Table of findings from difference-in-differences regression analysis showing the number of cans observed,
number of households along the data collection route, and the maximum number of stickers observed on cans.
(*=Only statistically significant in the fixed-effects model; **Direction of sign changes in fixed-effects model).
Key Findings
a) Overall, there were improvements in 13 of the 16 neighborhood metrics (Metric 1,
Metric 3, Metric 4, and Metric 3 and 4 combined, across four neighborhoods) as
well as across all four metrics at the combined project level (See Figure 7).
a. There was no statistically significant change found in the amount of litter
prevalent before and after stickers were distributed along the
representative data collection routes in the four target neighborhoods
combined. However, a statistically significant decrease in the average litter
index score was found in the Brightwood neighborhood.
b. There were statistically significant improvements across all neighborhoods
at the combined level regarding both the number of overflowing cans as
well as the total number of overflowing and open cans combined.
c. Although not statistically significant, the number of cans decreased in both
Rosedale-Kingman and Trinidad neighborhoods, indicating the sticker
messaging could have led some people to put cans away more quickly
after collection (See Figure 8).
22
-------
d. None of the three "negative" effects (i.e., increased/worsening litter scores
or increased number of overflowing or open cans) were found to be
statistically significant (See Appendix D).
b) The maximum number of stickers observed by District staff along the four
representative data collection routes was 109 in total (See Figure 8). Because
approximately 1,022 households in the treatment group along the four
representative routes combined, project partners can infer around 10.6% of
households that received the campaign materials applied the sticker to their trash
cans. However, this number is likely higher, as greater compliance with sticker
recommendations would also mean cans spent less time out in alleyways and
street fronts where data collection occurred, and stickers could be counted.
Discussion
Metrics 1. 3. and 4
Results of the analysis show seven statistically significant findings in the improved
direction (i.e., a lower litter index score or fewer overflowing cans), meaning the
campaign had an overall positive effect on the treatment group. This can be compared
to no statistically significant findings in the "wrong" direction (i.e., higher litter index
score or more overflowing cans). Of the remaining non-significant results, the majority
were in the improved direction.
A potential contributing factor of finding no statistically significant change in the amount
of litter prevalent before and after stickers were distributed along the representative data
collection routes in the four target neighborhoods combined could relate to the timing of
data collection (See Appendix D and Figure 9). Brightwood and Park View
neighborhood routes were observed prior to trash pickup, while Rosedale Kingman and
Trinidad were observed after. This may explain the differences in the levels of
improvement that could be observed between these neighborhoods. For example, the
finding of no litter index score improvements in the Rosedale Kingman and Trinidad
neighborhoods may be attributed to the fact that data were collected following trash
collection, and therefore the alleyways looked less littered because D.C. DPW crews
removed excess trash. The small number of control group households in Park View and
Trinidad may have also limited the ability to detect statistically significant effects in those
neighborhoods and at the combined level.
23
-------
3.0 I
2.5 -
oj 2.0 -
o
u
c/l
X
-O 1.5 ¦
1
L_
CD
¦M
¦M
~ 1.0
0.5 -
Scores
< *•
1^2
>* > > r
1.81
' • " ' • » " ••••
I »• • • • • • •
s .'.v <•. ¦.? •
••1*3
53
• •
o.o J
¦Treatment - Avg
¦Control - Avg • Treatment • Control
Figure 9. Graph showing the average litter index score across all four neighborhoods over time. The blue vertical line
signifies sticker distribution.
Despite the above, there was a statistically significant decrease in the average litter
index score for Brightwood - which decreased by an order of magnitude of
approximately 0.4 comparing before and after treatment time periods (See Figure 10).
1.0 •
9/1/2020
10/1/2020
11/172020
¦Brightwood - Control (Avg)
Brightwood - Control
Brightwood - Treat (Avg)
Brightwood - Treat
Figure 10. The average litter index score for Brightwood decreased by an order of magnitude of -0.4 when
comparing pre- and post-treatment time periods. Sticker distribution timing is denoted by the blue vertical line.
24
-------
In addition, all four neighborhoods showed statistically significant improvements in the
number of overflowing cans (See Figure 11) and the number of overflowing and open
cans combined (See Figure 12). For example, in the ParkView neighborhood, the
difference-in-differences regression suggests that blocks that received stickers
experienced a statistically significant reduction in the total combined number of open
and overflowing cans of ~3.77 cans. Prior to the treatment period (receiving a sticker),
those same blocks were reported as having an average of ~9.73 open and overflowing
cans each week. Since the campaign sticker recommended "Keep your lid closed and
do not overfill your can," the decrease in overflowing cans suggests that residents
shifted their behavior after being informed by the educational campaign.
20.0 -I
18.0 ¦
16.0 ¦
£ 14.0 ¦
rc
u
on
| 12.0 ¦
o
^ 10.0 -
o
o
s- 8.0 ¦
CD
_Q
E
z 6.0 ¦
4.0 ¦
1
2.0 -
Ove
•
• •
•
•
• •
•
. •.
* - . * 4 . •
• •• •
92# . •.*•••• v..
rf lowing
•
•
•
•
• * •
• • ^ m • _ •»
• • • • •
• •• • . •
• *•••»•• „„„
_ • t • • 1.81
•• • • • •
1.53* „ _
• 1 • •• ••
• 1.43
Treatment - Avg Cont
rol - Avg • Treatment • Control
Figure 11. The number of overflowing cans across all four neighborhoods combined decreased by an order of
magnitude of-1.364 when comparing pre- and post-treatment time periods.
25
-------
25.0
<= 20.0
u
on
c
fe 15-°
>
o
"O
c
fO
Q_ 10.0
o
CD
_Q
E
D
5.0
Open + Overflowing
•Treatment - Avg
¦Control - Avg • Treatment • Control
Figure 12. The number of open and overflowing cans across all four neighborhoods combined decreased by an order
of magnitude of-2.159 when comparing pre- and post-treatment time periods.
While District staff did not observe a statistically significant decrease in the amount of
litter surrounding the cans (categorized in the litter index score), project partners can
assume that less litter escapes into the environment if more cans are closed and
secure.
Cans with closed lids are presumed to be less vulnerable to spillage (i.e., waste is more
likely to blow out of open cans and open cans provide easier access to pests and other
animals). The lack of significant change in the amount of litter surrounding cans can
likely be attributed to weekly data collection timing and/or the relatively stable weather
experienced during the period of data collection.
Metrics 2 and 5
Although not statistically significant, the number of cans decreased in both Rosedale-
Kingman and Trinidad neighborhoods. This could indicate the sticker messaging led
some people to put cans away more quickly after collection, as recommended via the
inclusion of "Place trash in can outside shortly before pickup" on the sticker.
Since it was too difficult to determine whether all cans with stickers were placed out on
any given day of data collection, project partners determined the maximum number of
26
-------
stickers as an indicator of the campaign "contact" rate by summing the maximum
number of stickers counted for each block in any given week. For example, the
maximum number of stickers counted at 200 Jefferson St. Alley was seven on February
9th. The maximum number counted at 1200 Owen PI. Alley was six on both December
18th and January 8th. This method likely undercounts the total number of stickers to a
small extent since it is likely that in some neighborhoods, there was never a week where
all cans with stickers were placed on the curb at the time of data collection. In addition,
it is important to note that refraining from applying the sticker does not mean a
household threw away the educational material before reading it. In some cases, the
sticker and ancillary messaging were still read by recipients, which may have exhibited
certain positive behavior changes.
The number of homes included in the treatment group and control group in each
neighborhood was quantified after data collection was completed using online software
to show a more accurate representation of the treatment effect. The number of
properties along each alley varied greatly - for example, in Brightwood, the number of
households along the data collection walk route varied from 28 to 65 properties per
block ID. When comparing the number of stickers applied versus the total sample size,
project partners found that calculating the total number of households along each data
collection route was more representative than solely counting the number of cans
present because many residents own and put out several cans, but each household
only received one sticker.
Despite the relatively small sticker application rate of 10.6%, project partners still
managed to find positive impacts of the campaign and believe the initial results are
promising. Compared to other District government canvassing efforts (e.g., regular
email distributions and door-to-door canvassing for senior Covid-19 testing), this
campaign was relatively successful in terms of reach rate. For example, suppose 10.6%
of the homes in the treatment area showcased the campaign sticker on their can, and
data collection routes accounted for approximately 12.7% of households that received
the campaign materials. In that case, it can be inferred that more than 1,000 trash cans
in the four target neighborhoods chose to display the sticker.
In addition, D.C. DPW services about 105,000 homes, while the Curbside Disposal
Education Campaign Pilot's treatment group size was only around 1,022 homes. If
sticker distribution increased to even a fifth of D.C. DPW-serviced households, it could
translate to an impressive impact on the city and could drive even more statistically
significant results. District staff have expressed interest in potentially expanding the
campaign to be District-wide.
27
-------
District 311 Request Analysis
As previously mentioned, 311 is a telephone and online portal wherein District residents
may contact District employees to address issues such as trash can repair and
replacement, alley cleaning, etc. (See the left-hand column of Figure 15 for a more
extensive list of 311 request options). The intention of this part of the analysis is to
determine if more 311 requests were submitted in the target neighborhoods after sticker
distribution, as prompted by recommendation #4 on the sticker: "Call 311 or visit
311 .gc.gov for assistance with cans needing repair or replacement."
District staff compiled a dashboard of 311 service requests submitted during the
treatment period, from November 2020 to February 2021, after stickers were distributed.
The 311 service requests were categorized by Single Member District (SMD), or
subdivisions of District wards/neighborhoods represented by different Advisory
Neighborhood Commissioners, and the requests from SMDs that approximately overlay
the boundaries of the four target neighborhoods were pulled out for this analysis. To
compare service requests submitted during the previous year, the District average for
311 service requests was subtracted to isolate the effect of the campaign.
After controlling for annual trends and seasonality, project partners found the effect of
the education program across the four target neighborhoods increased by 2.2
percentage points (See Figure 13). The seasonal impact was controlled by comparing
the average rate of change in the number of requests from month to month during the
treatment period of November 2020 through February 2021. The annual trend was
accounted for by taking the difference in the rate of change from the previous year for
the same months (November 2019 through February 2020).
Year
District
Target
Neighborhoods
District
y/y
Target
Neighborhoods
y/y
Difference of
Target
Neighborhood
s - District
2020-2021
-7.8
-6
-4.6
-2.4
2.2
2019- 2020
-3.2
-3.6
6.5
5.3
-1.2
Figure 13. Degree of change in 311 requests when controlling for annual trends and seasonality.
The highest potential degree of change in the number of 311 requests was found in the
Trinidad neighborhood (+5.3), while the lowest was found in Park View (-1.7) (See
Figure 14).
28
-------
Brightwood
Effect: 3.2
Parkview
Effect: -1.7
Trinidad
Effect: 5.3
District of Columbia Neighborhood Impact
Percentage Point Effect
Figure 14. Map visualizing the degree of change in 311 requests according to each target neighborhood.
For the purposes of this analysis, project partners incorporated a broader group of 311
service request data than explicitly mentioned on the sticker. The campaign sticker
encouraged residents to use 311 for help with can repair or replacement specifically, but
several additional types of service requests were included in the analysis because of
District staff interest in their correlation with the overall campaign message/goal of
cleaner communities, such as alley cleaning and rodent inspection and treatment (See
Figure 15). As depicted in the table below, more service requests were submitted in the
priority neighborhoods compared to the District average in several categories: alley
cleaning, bulk collection, recycling cart repair, rodent inspection and treatment,
sanitation enforcement, and supercan delivery.
29
-------
Service
Request
Brightwood
Rosedale
Kingman
Park
View
Trinidad
Across
Service
Totals
Average Number
of Requests Per
Month
Alley Cleaning
13.9
-7.1
-18.0
-0.4
3.1
60-82
Bulk Collection
-4.2
-5.9
3.1
6.2
0.9
608-744
Rat
Replacement
Containers
98.4
56.8
-51.1
31.2
-1.4
1-10
Recycling Cart
- Repair
-51.1
-5.2
51.7
32.3
14.9
6-8
Recycling Cart
Delivery
-32.4
-8.3
-55.9
-22.6
-24.3
30-46
Rodent
Inspection and
Treatment
14.5
9.7
3.4
11.3
8.2
120-152
Sanitation
Enforcement
22.9
37.2
-3.3
11.2
9.2
59-104
Supercan -
Delivery
11.8
146.5
-24.8
-18.1
5.5
33-38
Supercan -
Repair
53.2
4.0
6.5
160.3
-5.6
5-16
Trash Cart -
Delivery
-17.1
-24.1
-33.0
7.4
-13.1
37-51
Trash Cart
Repair
18.4
48.9
-66.9
-61.5
-29.6
6-14
Within
Neighborhood
Totals
3.2
0.6
-1.7
5.3
2.2
Average
Number of
Requests Per
Month
236-308
176-208
294-
361
317-360
Figure 15. This table shows the percentage point change in service requests for the 4 target neighborhoods relative
to the District average, while comparing the treatment period of November 2020-February 2021 to November 2019-
February 2020.
The graph below shows the key takeaways of 311 requests from November 2020
through February 2021 in the four target neighborhoods (See Figure 16). The drastic
increase in the bulk collection could be related to the Coronavirus pandemic, as many
residents worked from home and took this time to clean out their property. While repair
30
-------
of cans was higher in these neighborhoods compared to the District average (2.4),
delivery of new cans was lower (-8.5). Few people were moving to Washington, D.C.
during the beginning of the Coronavirus pandemic, which could contribute to a smaller
number of can delivery requests in specific neighborhoods. In addition, a nationwide
shortage of supercans due to supply chain issues during the pandemic affected the
number of new cans available for delivery.
Effect on Service Type: All Priority Neighborhoods sizeof Effect
Sanitation Enforcement
Rodent I nspection and Treatment
Alley Cleaning
Trash/Super Can/Recycling Cart Repair
BulkCollection
Rat Replacement Containers
Trash/Super Can/Recycling Delivery
M2-4
¦ 0.9
-1.4
-10 -S
-6 -4 -2 0 2 4 6 8 10
Percentage Point Difference from Non-Priority Neighborhoods
Figure 16. Degree of change in 311 requests compared to District average. For the purposes of this visualization,
several complementary service requests were combined into one line item, such as repair of trash carts, super cans,
and recycling carts.
As shown in the graph below, a surprising amount of certain 311 requests were
submitted on behalf of residents in the four target neighborhoods compared to the
District (See Figure 17). For example, 38% of District-wide rodent inspection and
treatment requests submitted from November 2020-February 2021 were made in these
neighborhoods.
31
-------
Total Number of Requests forTreatment Period Across Non-Priority and Priority
SMDs. (2020 vs. 2021)
¦ 2021 Priority SMD Requests ¦ 2020 Priority SMD Requests ¦ 2021 City Requests ¦ 2020 City Requests
Alley Cleaning
299
1877
Bulk Collection
¦ 2,760
17,063
Rat Replacement Containers
31
102
Recycling Cart - Repair
27
137
Recycling Cart Delivery
162
1743
Rodent Inspection and Treatment
1 554
¦ 1,448
Sanitation Enforcement
346
¦ 1,305
Supercan - Delivery
145
¦ 1,027
Supercan - Repair
36
246
Trash Cart - Delivery
181
1528
Trash Cart Repair
43
255
Figure 17. District 311 request submissions for Single Member Districts (SMDs) overlaying the four target
neighborhoods compared to the rest of the District, comparing November 2019-February 2020 ("2020," control
period) and November 2020-February 2021 ("2021," treatment period).
District Sanitation Enforcement Analysis
Sanitation enforcement was the most impacted service type in the priority
neighborhoods with a +9.2 degree of change after treatment compared to the District
average according to the 311 service request data analysis detailed above (See Figure
16). District residents can request sanitation enforcement and report improper disposal
of trash or solid waste. In response, District Solid Waste Education and Enforcement
Program (SWEEP) inspectors issue ticket violations, including leaving trash/recycling
containers in a public space (alleyways and street fronts) before 6:30 pm the day before
collection or after 8 pm on collection day. This behavior was specifically outlined on the
campaign sticker, which encouraged residents to "place trash in can outside shortly
before pickup." An increase in the number of sanitation enforcement requests in these
neighborhoods could mean that after being informed by the campaign sticker, residents
were able to take action and promote better behaviors from their neighbors. It is also
possible that newly appointed Advisory Neighborhood Commissioners (ANC) were
eager to report mismanaged trash because the pilot's treatment period coincided with
ANC elections.
32
-------
After identifying this finding, District staff compiled additional data specific to sanitation
enforcement in the wards, including the four priority neighborhoods. The below analysis
outlines the number of sanitation enforcement tickets distributed in the four wards,
which incorporate the smaller target neighborhoods between November 2020 and
February 2021, and compares that to the same four-month period in the previous year.
These data were only available on a ward-level basis and did not completely represent
the specific block IDs included in the treatment groups that received the stickers. Two
sanitation enforcement ticket types were included in this analysis: a) R110 - Solid Waste
not properly stored/contained for collection and providing food or breeding ground for
rodents or causing a potential fire hazard; and b) R220 - Solid Waste Containers out for
collection at wrong time or place.
The average number of sanitation enforcement tickets per year (January-December)
per ward dropped from ~50 tickets in 2019 to ~19 in 2020. This is likely attributed to
reduced District capacity during the Coronavirus pandemic, which may also be seen in
the sanitation enforcement row comparison of 2020 and 2021 in Figure 17 above.
Findings show that although there were fewer citations year to year District-wide as well
as specifically from November 2020-February 2021 in the wards that encompass the
four priority neighborhoods compared to the same four-month period a year prior, the
tickets that were distributed after treatment were more often made in the priority
neighborhoods compared to the rest of the District. In fact, 69 tickets were distributed in
the four wards in the four months following sticker distribution (November 2020-
February 2021), totaling $2,700.
33
-------
Further Data Collection Context
District staff observed changes over the 22-week data collection period that may or may
not be as evident in the data analysis described in the prior section. The below is
intended to provide further context, experiential insight, and additional findings from on-
the-ground data collection.
District staff reported that impressionistically, they noticed an impact in the overall
cleanliness of alleyways and street fronts along the 1-mile representative routes in each
target neighborhood before and after stickers were distributed.
After sticker distribution, dozens of examples of homes with cans exhibited the
campaign sticker and seemed to show proper disposal behaviors. In one neighborhood,
a District staff member noticed that one household put the sticker on their can right
away (this was noted the week after distribution), but no other stickers were counted on
the street. The following week, the neighbors on each side of the "early adopter" had
applied their stickers. As project partners had hoped, this suggests that one resident
inspired by the campaign could influence others to take positive action. Project partners
also recount several households along the data collection routes that suddenly started
storing their cans away from the alley (outside of waste collection time) after the
campaign stickers were distributed.
34
-------
Unfortunately, campaign stickers were evident but improper disposal practices such as
can overflow or spillage were noted. In addition, several recycling carts were spotted
with the sticker applied to the lid despite explicit directions provided on the back of the
sticker instructing recipients to place it on a trash can, not a recycling cart. To avoid
confusion, the sticker showcased a green can graphic, and the accompanying
campaign materials deliberately referenced a "green trash can," but this was still not
enough in a few cases. Though many sticker recommendations also apply to the
disposal of recyclable material, the campaign was intended to focus on residential trash.
35
-------
District staff discovered that dumping of bulk debris, mainly associated with home
cleaning/moveout/foreclosure, was a significant problem in the four target
neighborhoods. Discarded furniture such as mattresses and sofas often piled up in
alleyways and, in some instances, remained for several weeks without collection. For
example, one Brightwood alleyway improved significantly in litter levels after one
household, a repeat offender, completed the move-out process.
In Washington, D.C., curbside bulk collection is permitted when a resident specifically
requests it through 311 because trash crews cannot accommodate for large items on
their weekly pickup routes. The images below of bulk debris in front of a garage were
taken one week apart and show early signs of spillage and the inability of trash crews to
service the cans because of their placement behind a pile of miscellaneous items. The
third image shows mattresses leaning upon a fence, which was noted on the data
collection sheet as being present for four consecutive weeks.
36
-------
Debris associated with construction was also prevalent along the data collection routes
and had an indirect impact on alleyway litter scoring,
The data collection team observed that instances of bulk debris/ illegal dumping or
excessive spillage from curbside cans were often caused by the same households week
after week. In some cases, the impact of these "repeat offenders" on the surrounding
neighborhood was evident because their mismanaged waste spread throughout the
alleyway over time. This is another example of how baseline measurement of litter index
levels and disposal behaviors could help inform more targeted outreach and education.
The impacts of "repeat offenders" on the behaviors of neighbors were not specifically
addressed in this study, but research suggests that litter attracts more litter, meaning
people are more likely to litter in areas they already perceive as unclean, further
exacerbating the problem.
There was repeated dumping in a particular alleyway along the Brightwood data
collection route, which did actually display a District government-provided "No Dumping"
sign, in a different alley, the dumping of hazardous materials like oil and paint was
noted and serviced by DRW soon after being reported.
37
-------
Evidence of rodents, either through visible bite marks and holes in can lids or actual rat
carcasses, was also well cited in data collection notes.
38
-------
A potential solution to reduce the amount of unintentional litter associated with
residential pickup is to encourage using a designated waste can storage area near the
back of each home for use outside of collection day. District staff observed that homes
with space set aside for store cans often exhibited better disposal behaviors and were
kept tidy and clean, likely because the area was perceived as part of an individual's
property. This can be compared to homes that were engineered in a way that almost
required permanent placement of bins in communal areas like an alleyway, for which an
individual may feel less responsible. The District government may explore the validity of
this potential solution further.
39
-------
Focus Group Takeaways
Methodology
After distribution, project partners hosted several focus groups to garner community
feedback on the campaign sticker. Typically, focus groups are held after initial campaign
messaging, and sticker design have been developed before any orders have been
placed. This ensures that feedback can be incorporated into the final product. However,
due to the timing of contractor support and Covid-19, this effort was pushed to March
2021, after the campaign materials were shared with residents in the four target
neighborhoods.
The project team conducted two focus groups to gauge resident feedback on the
educational material and their experiences with the District Government and their
communities. The first focus group included Advisory Neighborhood Commissioners
(ANC) from Single Member Districts within the treatment area. The second focus group
was held with residents recruited from solid waste-related service requests made
through the 311 system between November to December 2020, also residing within the
treatment area. District staff used a random number generator to pull 60 names from
this list. Eight individuals from the list volunteered to participate in a meeting.
Each focus group ran for 60 minutes and covered the following topics: neighborhood
characteristics, alley characteristics, opinions and understanding of trash and recycling,
comments on the educational material, and opinions of and engagement with overall
government services and local waterways. The below discussion outlines key
takeaways within the project's scope. The focus groups were recorded for notetaking
purposes.
Key Findings
Participants in both groups indicated a high engagement level with their neighborhoods
and District government operations, unsurprisingly given the recruitment methods.
Perceptions of Alleys
Participants have mixed feelings about the alleys they live on, exacerbated (both
positively and negatively) by their experiences during the Covid-19 public health
emergency. They described often gathering in their alleyways with their neighbors to
socialize and letting children play. Still, the presence of illegal dumping, rodents, and
human waste gave them reservations about using the space more often:
Commissioner: "My 10-year-old likes to ride his bike back there, but I worry about
disease—I make him take a shower."
40
-------
Resident: "I let my son ride his scooter back there, which I question sometimes.
It's a nice usable space; it would be great if it was a little bit cleaner."
The insight shared by the participants suggests that alley design impacts alley
cleanliness, among other factors. Very narrow alleys correlate with more neighbor
conflict and a sense of disorder, while alleys that have recently been improved through
programs such as "AlleyPalooza" (the District Government's alley repair and
rehabilitation program) become cleaner and more attractive after the upgrade. It was
also suggested that alleys with a mix of single-family rowhouses and apartment
buildings tended to have more overflowing dumpsters and uncollected bulk trash at the
multi-family properties (these residences are serviced by private haulers instead of the
D.C. DPW). Multiple participants commented on spillage and damage to cans and
property from D.C. DPWs collection vehicles.
Understanding of District Disposal Guidelines and Services
All participants correctly knew their trash and recycling collection days, which was
unsurprising given the focus group recruitment method. Most reported splitting the
duties of taking the trash from the house to the can, and the can to the collection point,
with a household member (spouse or child). This takeaway solidifies the importance of
simple and easy to understand sticker verbiage and the use of illustrations to convey
key campaign messages. Despite the District sanitation requirement that cans may only
be in public space for collection after 6:30 pm the night before collection and until 8 pm
on the collection day, most participants reported that it was common practice on their
block to leave cans in the alley at all times. This suggests that in order for more lasting
behavior change around can placement, and enhanced enforcement may be necessary.
Commissioners reported using the District's 311 system for overflowing trash and
sanitation enforcement. Operationally, they use 311 to create a paper trail to
demonstrate ongoing/repeat problems because they find servicing agencies "treat
everything like an isolated event." Residents say they use 311 and follow agency social
media accounts to stay up to date on District policies and programs. This suggests that
a city-wide service system used to submit and manage requests is a valuable
investment for municipalities, as well as distributing pertinent information via social
media and agency newsletters, as demonstrated in this comment:
Commissioner: "Reporting dumping and overflowing cans through 311 on Twitter
has worked wonders."
Perspectives on the Campaign Sticker
41
-------
Participants in both groups expressed confusion about the purpose of the sticker,
indicating that they already knew all the tips and wondering if the garbage can was the
right location to place these messages:
Commissioner: "If I brought my bin in, the rats would be in my backyard. Not sure
how keeping bins in my property solves the rat issue."
Focus group participants were likely to be more knowledgeable about proper disposal
behaviors than residents as a whole because of the focus group recruitment method
used; however, it is likely that many residents were not familiar with all four
recommendations included on the sticker. However, future focus group efforts should
consist of participants with no known prior engagement with waste-related issues in
their city to provide a more representative treatment sample group. Holding focus
groups before sticker design and messaging have been finalized can help reduce future
confusion around the educational product's intention and expected outcomes. Some of
this confusion could have also been alleviated if project partners were able to hold one-
on-one conversations during the distribution process or if they were able to implement a
more comprehensive educational campaign.
A few participants suggested making the recommendations shorter and removing
ancillary information so the product could be smaller:
Commissioner: "I wonder if the programs would allow us to eventually remove all
of the logos and other information from the stickers themselves (but still keep
them on the flyer explaining the program, of course). I wonder if we could make
the stickers rounder, cuter, and easier for folks to like enough that they'll slap it
on their cans."
Several would have preferred a similar product but focused on recycling instead,
particularly related to rinsing items and keeping them loose in the bin:
Commissioner: "The info we get through the mail is very helpful. I didn't see how
these stickers added to it. I know some people who don't recycle. The sticker
isn't going to move them."
Participants also broadly believed that if an individual was motivated enough to apply
the campaign sticker on their trash can, they were likely already following the behaviors
outlined on the sticker. In addition, participants felt that the people who most need this
education are the least likely to change behavior. This doubt around the ability to effect
change in the "worst offenders" is shared among many in the behavior change
42
-------
community, no matter the cause (i.e., environmental, health-related, etc.). However,
strategic steps outlined in Community-Based Social Marketing can be taken to most
effectively reach these groups. A potential next step could be to use a similar approach
to identify repeat offenders and inform more targeted messaging to influence behavior
of those select individuals:
Resident: "I think it would be helpful to my neighbors, but I don't think they would
follow it. Are they not aware, or do they not care? Maybe it will help a few."
Commissioner: "The only people who put the sticker on already bring their bin in.
If you don't bring your bin in, why would you put a sticker on there saying you
should?"
The above findings and comments can help inform improvements to a similar approach
to residential curbside disposal education in other communities.
43
-------
Limitations
As outlined in the Data Collection Methodology and Statistical Analysis sections above,
several nuances may have influenced the findings (i.e., timing and frequency of walk
routes). The most significant potential impact on this project was the Coronavirus
pandemic.
If project partners were able to interact directly with residents when distributing
campaign materials, the percentage of residents who applied a sticker to their trash
cans would likely have been much higher. Unfortunately, the presentation card served
as the only form of communication between project partners and sticker recipients. In
the few instances where District staff and volunteers encountered residents during
canvassing, residents reacted positively when receiving the educational materials. One
of the most effective methods of Community-Based Social Marketing is to educate
trusted and influential individuals who can, in turn, help persuade others to adopt a
specific behavior - this approach also likely would have increased the number of
stickers applied.
For this pilot, project partners considered whether it would be best to avoid contact with
residents entirely by applying the sticker to residential trash cans on their behalf.
Although the cans are District-provided, project partners decided against this approach
to avoid any complications associated with cans located on private property. Project
partners were also unsure how effective the sticker would be in changing behaviors if it
was applied without directly prompting a resident to read or absorb the
recommendations.
The Coronavirus pandemic also had implications on capacity. During the beginning of
the pandemic, District staff were assigned to emergency response teams and had less
bandwidth for carrying out logistics for this project. The pandemic also impacted the
number of volunteers to help distribute stickers to residents in the target neighborhoods.
The full extent of the impacts of Covid-19 on data collection is unknown; however,
project partners assume there was more improperly disposed of bulk waste than
typically found in the four target neighborhoods because Washington, D.C.'s Fort Totten
transfer station - the District's dedicated bulk waste drop-off location - was closed to
residents for several months. The amount of residential waste produced in each
household may have increased during the pandemic because so many more individuals
were working from home than in pre-pandemic times. In addition, news sources,
including The Washington Post, reported on what they referred to as "the great
decluttering of 2020," where stay-at-home orders and social distancing guidelines led to
mass home cleanouts, inundating donation centers and causing temporary closures
44
-------
once warehouses were deemed full. The number and type of 311 service requests
could also be influenced by those at home more likely to witness mismanaged waste
and submit a complaint.
45
-------
REPLICABILITY AND RECOMMENDATIONS
Recommendations
In conducting this pilot project and case study, project partners have identified a number
of recommendations for those interested in launching a similar campaign. The following
suggestions for improvement are related to project partner selection, campaign
messaging and design, effective implementation and advertisement, and thorough
impact measurement. Additional research ideas for consideration are also included
below. The concluding project checklist and cost estimation summary can be used to
adopt and adapt a similar approach to curbside disposal behavior change in other
communities.
Partner Selection
It is critical to have a key project champion to ensure the success of a similar campaign.
Julie Lawson, Director of D.C. MOCC, played an essential role during the early planning
stage of this pilot project and maintained this role throughout the rollout of the campaign
by keeping fellow District government leadership (DPW, DOEE, and the Mayor) and the
Interagency Waste Reduction Working Group informed about project progress. MOCC
staff also attended monthly Advisory Neighborhood Commissioner (ANC) meetings for
each neighborhood included in the project scope. At these meetings, MOCC staff
explained the initiative to ensure commissioners would be able to accurately respond to
any questions about the effort from citizens. MOCC staff dedicated countless hours to
weekly data collection and led both focus group meetings. Having a passionate on-the-
ground partner with the community's best interests at heart can make all the difference
in campaign effectiveness. Any future effort should incorporate perspectives from a
variety of stakeholders to ensure project success, including the solid waste industry,
stormwater, local neighborhood associations, volunteer/advocacy groups, and others.
Sticker Messaging and Design
Conduct baseline data before narrowing in on a list of recommendations. Project
partners developed the four recommendations included in this campaign through
background research and first-hand knowledge. Yet, through observations during the
campaign, it became apparent that another issue that could have been addressed in the
sticker recommendations was illegal dumping and bulk debris.
Time constraints affected the ability to hold a series of focus groups with residents prior
to final decision on sticker design and verbiage. Having a discussion with members of
the intended target audience in advance of the campaign's rollout would have been
extremely helpful to ensure that a sticker was the best form of communication and that
46
-------
the proposed campaign messaging resonated with them. The perspectives and
suggestions shared during a focus group will undoubtedly enhance overall campaign
effectiveness.
Campaign Rollout
As mentioned above, campaign messaging is more effective when distributed in
combination with brief personal contact with a trusted source. Going a step further in
educating residents about proper disposal behaviors by encouraging them to act upon
this information is advisable. The use of "commitments" is recommended in Community-
Based Social Marketing to promote ownership and responsibility in carrying out a
specific action in the future. A commitment could be a verbal agreement; for example, a
resident might agree to the following request: "We are asking residents to commit to
undertake proper curbside disposal behaviors to help keep our waterways and
communities clean; would you be willing to join the growing number of people who have
made a similar pledge and agree to apply this sticker and follow these
recommendations?" Alternatively, a commitment could be physical such as signing a
pledge to follow the four recommended disposal behaviors.
Advertisement & Marketing
Public signage is a must in a more broad-reaching city or county-wide education
campaign. Physical signage in public spaces, social media content, press releases,
news coverage, and other forms of communication can serve as a frequent reminder of
the campaign recommendations even while residents are outside the home. Hosting
community cleanup events or educational presentations with neighborhood groups and
Advisory Neighborhood Commissioners combined with campaign messaging could
increase engagement in the cause.
Impact Measurement
Below are several recommendations related to data collection methodology:
• Collect data consistently either the day before pickup or day after pickup, or
collect data twice a week (keeping in mind the collection schedule for each
neighborhood if it differs).
• Collect data from a larger area (increase the size of both the treatment group
and control group).
• Depending on your community, focus on gathering data from alleyways, not
street fronts.
• Quality control: If there are multiple individuals conducting data collection,
ensure they all have the same understanding and perception of what constitutes
a certain litter index score, overflowing can, etc.
47
-------
• If not limited by capacity restraints, collect data throughout the sticker
distribution period.
• Distribute a follow-up survey or canvass door to door to better understand the
self-reported impact of the sticker on resident behavior.
Part of the data analysis would not be possible without existing municipal data tools on
311 service requests and sanitation enforcement. If your community does not already
collect this information, project partners recommend it as the first step to best target
efforts.
Future Considerations
Additional data analysis to conduct with more time and resources:
• Is the process of waste pickup (operator mismanagement or speed of service) a
significant contributor to the amount of litter in a community?
• Are households observed with a sticker applied to their can more likely to exhibit
proper disposal behaviors (i.e., is there a correlation between sticker and closed
cans, or do the alleyways with more stickers have lower litter index scores)?
• Are cans more likely to be overflowing if they are small (trash cart) versus larger
(supercan)? Or on a weekly or semiweekly pickup schedule?
Project Checklist
~ Determine project partners
~ Conduct a baseline assessment to identify key issues to target
~ Draft of slogan and recommendations
~ Draft of sticker design and ancillary messaging (i.e., presentation card)
~ Hold focus groups for feedback on campaign messaging and design
~ Develop an impact measurement plan
~ Incorporate focus group feedback into the final draft of sticker
~ Announce forthcoming campaign launch to the public
~ Begin data collection
~ Place an order for stickers
~ Receive sticker order
~ Distribute stickers
~ Boost external campaign messaging (social media, news coverage, etc.)
~ End data collection
~ Host focus groups for further feedback
48
-------
~ Conduct data analysis
~ Determine efficacy and next steps
* Remember to set aside enough time for approval from all project partners before each essential step.
Cost Estimation
EPA partnered with the National Environmental Education Foundation (NEEF) to design
and produce the stickers later distributed to target neighborhoods. The total price for the
8,000 stickers ($4,589.81) and die-cut wrappers ($1,533.77) was $6,123.48. This
means individual stickers cost ~ $0.76 each. It is worth noting that these were custom
size, performance grade vinyl stickers to fit the cans appropriately and withstand the
elements of being outdoors. Variations could be even more affordable if future iterations
used different dimensions, material, design, and if bought in greater bulk. The above
budget does not include printing the supplementary presentation cards. District staff and
volunteers carried out weekly data collection and the sticker distribution process.
49
-------
50
-------
CITATIONS
Green Solutions, 2008. "Clark County Curbside Recycling Pilot Program." Clark County
Department of Public Works.
www.clark.wa.gov/recvcle/documents/SWAC/ClarkCoPilotProiectFinalReport2.pdf
"How and Why People Litter." Geelong, AUS.
https://www.geelongaustralia.com.au/litter/article/item/8d0baf9822eb50d.aspx
Keep America Beautiful. "2009 National Visible Litter Survey and Litter Cost Study."
https://www.kab.org/sites/default/files/News%20%26%20lnfo Research 2009 National
VisibleLitterSurvevandCostStudv Final.pdf
Keep America Beautiful. "2020 National Litter Study: Summary Report, May 2021."
https://kab.org/wp-content/uploads/2021/05/Litter-Studv-Summarv-Report-Mav-
2021 final 05172021.pdf
Koncius, Jura. "The great decluttering of 2020: The pandemic has inspired a cleanout of
American homes." The Washington Post. August 5, 2020.
https://www.washingtonpost.com/lifestvle/home/the-big-pandemic-clean-out-clearing-
the-iunk-out-of-vour-home-while-stuck-there/2020/08/04/230d71 d2-c868-11 ea-a99f-
3bbdffb1af38 storv.html
Owens, E.L., Zhang, Q., and Mihelcic, J.R. 2011. "Material flow analysis applied to
household solid waste and marine litter on a small island developing state." J. Environ.
Eng., 137 (10), pp. 937-944. https://ascelibrarv.Org/doi/abs/10.1061/(ASCE)EE.1943-
7870.0000399
Spehr& Curnow. Litterology. 2015. Book.
Visual On-Land Trash Assessment for Stormwater. April 15, 2015. EOA, Inc.
http://scvurppp.org/wp-
content/uploads/2018/05/Updated Visual Trash Assessment Methodology 4 15 201
5.pdf
Wagner, T.P. and Broaddus, N. 2016. "The generation and cost of litter resulting from
the curbside collection of recycling." Waste Management, (50), pp. 3-9.
https://www.sciencedirect.com/science/article/pii/S0956053X1630Q526
51
-------
Wagner, Travis P. "Analysis of the Residential Curbside Collection of Trash and
Recycling and the Generation of Associated Litter in Portland, Maine." November 2015.
https://www.nrcm.org/wp-
content/uploads/2015/12/PortlandMSWStudv twagner Nov2015.pdf
52
-------
Appendix A. Sticker Distribution Materials
APPENDIX
Cleaner communities
and waterways
start here
i
How to apply your sticker:
Clean: Clean the lid of the trash can with a mild cleaner
and allow to dry.
Peel: Carefully peel off the backing from this sticker.
O Flatten: Use a squeegee or credit card to remove
the air bubbles.
Keep lid closed and do not overfill the can.
dU Bag your trash before putting it in the can.
(Do not bag recyclables; place items loose in your blue can.)
{^) Place trash in can outside shortly before pickup.
UCall 311 or visit 311.dc.gov for assistance
with cans needing repair or replacement.
®TrashFrooDC
cloancity.dc.gov unwiw er*»e,e,M
, I H ~.U\; I If BHOF LNERCiY & aBtelSTRICT or CD U4BI.V
zerowaste.dc.gov MCITYL/W i^™e-jv*k)n»«nt OCMimlfi bowser, mayor
In partnership with the U.S. EPA's Trash Free Waters program
www.epa.gov/trash-free-waters
O Apply: Place your decal on the lid of the trash can. Start at
one end, guiding the sticker to slowly fall into position.
53
-------
Hello, Neighbor!
We know you want to keep your community and
local waterways clean, and this free sticker can Kelp.
Please place it on your green trash can lid as a reminder
about how to properly dispose of trash,
Mayor Bowser 's Office of the Clean City has partnered with the
U.S. Environmental Protection Agency's Trash Free Waters Program
to reduce the a mount of trash falling onto our neighborhood streets
and sidewalks. Trash that spillson streets and sidewalks attracts rats
and other pests. It can also be carried by water or wind into our
sewers and ultimately wind up in local rivers and streams.
The attached sticker lists a few simple steps you can take to help
keep your local community and waterwaysfree from trash.
To read more about how you can help keep your community clean,
please visit the Mayors Office of the Clean City at cleancity.dc.gov
or xerowaste.dc.gov, and follow *TrashPreeDC on social media.
54
-------
Appendix B. Data Collection Route and Sticker Distribution Maps
Brightwood
Marietta PI NW
Madison St NW
Madisort
Brightwood
Park r
xungreibw stnw-
Drr^ttiTKr
Kenneqy St NW
Kenned
ingrahatn
Rudolf
Hamilton St
-tJattatnrbl
Park View
55
-------
Rosedale/Kingman Park
Trinidad
56
-------
Brightwood
Park View
Columbia Kd NW
Rosedaie/Kingman Park
Trinidad
Gallaudet
University
Ivy City
The above maps show the pilot's weekly 1-mile data collection routes in the four target
neighborhoods with an extra layer - the blue shaded area denotates estimated blocks
which received the campaign sticker.
57
-------
Appendix C. Additional Images Taken During Data Collection
58
-------
Appendix D. Initial Data Analysis
Litter Index Score
Overflowing + Open Cans
Neighborhood
Magnitude
Direction
Significant
Magnitude
Direction
Significant
Brightwood
-0.395
1
Yes
-1.640
1
Yes
Park View
-0.066
1
No
-3.769
1
Yes
Rosedale
Kingman
0.015
T
No
-1.119
1
No
Trinidad
0.302
T
No
-1.706
1
No
Combined
-0.071
1
No
-2.159
1
Yes
Overflowing Cans
Open Cans
Neighborhood
Magnitude
Direction
Significant
Magnitude
Direction
Significant
Brightwood
-0.500
1
No
-1.286
1
Yes
Park View
-2.616
1
Yes*
-0.031
1
No
Rosedale
Kingman
-1.524
1
No
0.655
T
No
Trinidad
-0.068
1
No
-0.497
1
No
Combined
-1.364
1
Yes
-0.411
1
No
59
------- |