NCEE Working Paper
Customer Participation in Lead
Service Line Replacement
Heather Klemick, Ann Wolverton, Bryan
Parthum, Kristen Epstein, Sandra Kutzing, and
Sarah Armstrong
Working Paper 23-02
March, 2023
U.S. Environmental Protection Agency /if
National Center for Environmental Economics
https://www.epa.gov/environmental-economics env'r o n m e n ta"e con o m i cs
-------
Customer Participation in Lead Service Line Replacement1
Heather Klemick*+, Ann Wolverton+, Bryan Parthum+, Kristin Epstein*, Sandra Kutzing*, Sarah
Armstrong®
ABSTRACT:
Many water systems are developing programs to identify and replace the 6-10 million lead service lines
(LSLs) estimated to supply drinking water to homes in the United States. This study investigates factors
affecting customer participation in a voluntary LSL inspection and replacement program. Using individual
account data from a large public water system, we examined the characteristics of properties and
neighborhoods more likely to have LSLs and to participate in the program. We also used quasi-
experimental and experimental methods to evaluate the causal impacts on LSL program registration,
inspection, and replacement of two programs that offered subsidies to certain residents that covered the
homeowner costs of replacement. LSLs were more prevalent in the urban area of the water system, which
had a higher concentration of older housing stock, Black and Hispanic residents, renters, and poverty than
the suburban portions of the service area where LSLs were less common. We found that renter-occupied
and lower-valued properties were less likely to participate in the LSL replacement program, and properties
receiving more outreach letters were more likely to participate. Results from the two program evaluations
suggest that subsidies to cover homeowner LSL costs can significantly boost participation, but only when
the programs are well publicized and easy to access. Even then, there was still significant non-participation
among properties with confirmed LSLs.
Keywords: Lead Service Lines, Drinking Water, Randomized Controlled Trial, Quasi-experiment,
Environmental Justice
JEL Codes: Q52 I Q53 | Q56
Corresponding author: klemick.lieatliergepa.eov
+ National Center for Environmental Economics, US EPA
t CDM Smith; Kristin Epstein was Assistant Director of Water & Sewer at Trenton Water Works when this study was
initiated.
§ Accenture Federal Services; Sarah Armstrong was Senior Research Manager at the University of Chicago Energy
and Environment Lab when this study was initiated.
1 Proprietary data used in this study were acquired under a data use agreement with Trenton Water Works. The
views expressed in this paper are those of the authors and do not necessarily represent those of the U.S.
Environmental Protection Agency (EPA) or of Trenton Water Works. In addition, although the research described in
this paper may have been funded entirely or in part by the U.S EPA, it has not been subjected to the Agency's
required peer and policy review. No official Agency endorsement should be inferred. The authors thank Trenton
Water Works for their participation in the project; Caitlin Fair, East Trenton Collaborative, and Farrah Gee, City of
Trenton Department of Housing and Economic Development, for providing information about Trenton lead service
line replacement grant programs; Emma Hopkins, EPA ORISE research fellow, for research assistance; Ludovica
Gazze, University of Warwick, and Jackson Reimer, Wharton School at the University of Pennsylvania, for valuable
comments and contributions; Erik Helm, EPA Water Economics Center, and Michael Goldberg, Amina Grant, and
Kory Wait, EPA Office of Ground Water and Drinking Water, for helpful comments on the manuscript. The protocol
for the randomized controlled trial was approved by the University of Chicago Institutional Review Board.
-------
1 Introduction
An estimated 6-10 million lead service lines (LSLs) supply drinking water to homes in the United States
(EPA 2021), and about one third of U.S. water systems report at least some LSLs (Cornwell et al., 2016).
These lines pose an increased risk of lead exposure through drinking water to residents (Brown et al.,
2011; EPA 2021; Triantafyllidou and Edwards, 2012). Lead exposure increases the likelihood of adverse
health outcomes for both adults and children (EPA 2013).2 Adverse impacts on children's cognitive
function and behavioral outcomes are particularly well established. No safe level of lead exposure has
been identified. Pervasive LSLs coupled with ineffective corrosion control in Flint, Michigan, and Newark,
New Jersey, illustrate the risks of elevated lead exposure and adverse health effects from LSL (Pieper et
al. 2017; Dave and Yang 2022).
In response to legal requirements, public pressure, and new federal funding, water systems are
developing programs to identify and replace LSLs. EPA's 2021 revisions to the Lead and Copper Rule
require every water system to develop and release a service line material inventory by October 2024 and
require systems exceeding EPA's action level for water lead concentrations to implement mandatory lead
service line replacement (LSLR) programs. The Biden Administration announced a plan for eventual
replacement of 100 percent of LSLs that leverages regulatory, non-regulatory, and funding mechanisms
(White House 2021). The 2021 Bipartisan Infrastructure Law authorized $15 billion of direct funding for
LSLR, and LSLR is eligible for additional federal funding.3
Utilities are developing or expanding LSLR programs using federal financing and other strategies. These
strategies include rate increases, cost sharing programs, and replacement at no cost to residents. Potential
benefits to residents from LSLR can include reduced probability of lead exposure and reduced household
expenditures on bottled water and water filters purchased to mitigate exposure. The replacement may
also be capitalized into property values (Theising 2019; Christensen et al. 2023). However, the cost of
replacement is typically several thousand dollars and can be prohibitive to many residents (EPA 2021;
AWWA 2022). A paucity of accurate historical records means that an inspection is often required to
determine the service line material, and a resident is unlikely to independently know the lead status of
their service line.
Even when full subsidies are offered, non-financial barriers could impede household participation in both
inspections and replacements, including time and inconvenience costs, lack of trust in public institutions,
and mismatched incentives between landlords and renters. LSLR programs that do not address both
financial and non-financial barriers to participation could exacerbate pre-existing economic and racial
disparities in lead exposure. Black children and children in poverty have persistently higher exposures to
2 Lead exposure has a causal relationship with reduced cognitive function and disorders related to attention,
impulsivity, and hyperactivity in children, as well as hypertension and coronary heart disease, adverse hematological
effects, and adverse reproductive and developmental effects in adults (EPA 2013).
3 The Bipartisan Infrastructure Law authorized an additional $11.7 billion for which LSLRs are eligible, and the 2021
Build Back Better Act authorized a further $85 billion for which LSLRs are eligible.
-------
lead from multiple sources than other children in the United States as indicated by higher blood lead levels
(EPA 2022a).
This study investigates factors affecting customer participation in a voluntary LSL inspection and
replacement program. Using customer-level data from a large water system in New Jersey, we examine
the characteristics of properties and neighborhoods more likely to have LSLs and to participate in the
inspection and replacement program. We also evaluate the causal impact on LSLR participation of two
interventions offering full-cost subsidies to certain residents for replacement. The first intervention is a
community-based program offering grants plus extensive outreach and education about LSLs to residents
of a specific neighborhood in the urban area of the water system with a high percentage of Black and
Hispanic residents and high poverty rates. We estimate the effect of the community grant program using
a synthetic control approach comparing LSLR participation before and after the grant program was
launched among treatment and control groups that were perfectly balanced in terms of observable
characteristics affecting participation rates. In the second intervention, postcards were sent to customers
informing them about a different program run by the city housing department offering grants to low-
income homeowners for urgent home repairs, including LSLRs. Households in the treatment group of
properties were randomly selected from a larger pool of potentially eligible residents.
Results show that LSLs were more prevalent in the urban area of the water system service area, which has
a higher concentration of older housing stock, Black and Hispanic residents, renters, and poverty than the
suburban portions of the service area where LSLs are less common. An observational regression analysis
finds that renter-occupied and lower-valued properties were less likely to participate in LSL program
registration, inspection, and replacement. Results also indicate that properties receiving more outreach
letters were more likely to participate.
The evaluations of the two grant programs show that the community-based program was highly effective
in boosting participation in all stages of the LSLR program, though replacements remained well below 100
percent among properties with confirmed LSLs. However, sending postcards with information about the
housing department grant program did not have statistically significant effects on program participation.
A comparison of the two interventions suggests that programs covering homeowners' LSL costs can
substantially increase participation when they are well publicized and easy to access but are still
insufficient to achieve replacement of 100% of LSLs. Understanding the characteristics of customers who
participate in LSLR and the program characteristics that yield the highest participation rates can help
inform equitable and effective program design.
2 Barriers to Customer Participation
Inspection and replacement of LSLs is challenging due to numerous informational, transactional, and
financial barriers. Informational barriers include potential lack of understanding about the risks of lead
exposure, lack of trust in water systems, and language barriers. Transactional barriers include the time-
cost and inconvenience of scheduling an inspection or replacement and any subsequent disruption to
one's home or yard. Barriers to participation were likely exacerbated during our study period due to the
-------
COVID-19 pandemic, which may have decreased households' willingness to allow in-person inspection
and construction work.
2.1 Service Line Ownership and Transactional Barriers
The ownership structure for most service lines further complicates inspection and replacement (AWWA
2005) (Figure 1). Typically, the utility only owns the lines on the street-side of the curb stop valve or meter.
The portion of the service line that extends from the curb stop or meter into the home is privately owned.
Lead can be present in either or both sides of the line (EPA 2022b). In most cases, this implies that the
utility must obtain homeowner permission to enter the property before it can inspect and then replace
homeowner-side LSLs (Hull and Anderson 2022). Scheduling and participating in these efforts pose time
and inconvenience costs to homeowners. Low-income households may face outsized transactional
barriers due to tradeoffs between time spent ensuring basic needs versus preparing documents or
attending appointments needed for LSLR program participation (Mullainathan and Shafir 2013). Needing
to obtain permission from a landlord can pose further transactional barriers for rental properties. With
lower-income households being more likely to rent, this additional hurdle can further exacerbate pre-
existing economic and racial disparities in lead exposure.
Figure 1: Split Ownership of Water Service Lines
Water
Meter
Property
Boundary
Street
System-owned
Service Line
Customer-owned
Service Line
This typical service line configuration in a suburban area shows split ownership of the line between homeowner
and utility. Public water systems may have records of where lead-containing utility-owned service lines are
located. Inventory on the homeowner-side of the service line is typically less well documented and is a necessary
step in replacing water service lines. Source: EPA 2022b.
2.2 Informational Barriers
The lead status of homeowner-side service lines is often unknown to residents and water systems alike
due to limited availability of reliable records. Having an audit or inspection is a way to address imperfect
or incomplete information prior to making a home improvement that may yield health or environmental
benefits (Holladay, et al. 2016; Gillingham and Tsvetanov 2018). The inspection reveals or confirms the
presence (or absence) of a qualifying condition (e.g., having an LSL) but also involves time and
-------
inconvenience costs. Participation in the inspection will therefore depend on how the costs of acquiring
this information compare to household-level gains. Split incentives between renters and owners and
language barriers both increase the cost of acquiring the information. Gains also likely correlate with
housing and household characteristics (e.g., presence of children in the home; other sources of lead).
Households might also decline to participate in an inspection to avoid potential bad news about their
water quality, exhibiting an "ostrich effect" (Karlsson et al. 2009).
2.3 Financial Barriers
The decision to replace a lead service line once this information has been revealed also depends on how
household costs compare to gains. Water utilities in some states and municipalities are not currently
allowed to use rate revenue for homeowner-side service line improvements. In these cases, the
homeowner is responsible for covering part or all of the homeowner-side replacement costs. EPA (2021)
estimated average homeowner-side LSLR construction costs to be $4,000, and other estimates have found
homeowner-side construction costs ranging up to $10,000 (AWWA 2022).4 The gains from LSLR are more
difficult to quantify since the probability of being exposed to lead and the likelihood of a subsequent
health effect are uncertain.
2.4 Mitigating Barriers
Utilities seeking to encourage LSL inspection and replacement often focus on lowering coordination
failures and reducing the cost of participation (e.g., AWWA 2005; U.S. EPA 2019). This includes the
provision of subsidies or options to spread upfront costs over time, translating materials into languages
other than English, and, in a few cases, not requiring homeowner approval to access a property for
inspection and/or replacement. However, program design varies greatly across programs.5
Some utilities are constrained by local or state laws limiting the use of public funds to make investments
in private property, while others offer partial or full subsidies to replace the homeowner portion of the
service line (EPA 2019). Subsidies can take the form of discounted upfront prices or zero percent interest
loans with payments spread over several years. Some systems, such as Washington, DC, and Providence,
Rhode Island, target free or subsidized replacements to low-income homeowners or properties in
disadvantaged neighborhoods (EPA 2019; Kuffner 2022). The state of Michigan requires most water
systems to finance the full cost of LSLR for all properties, including the homeowner side (EPA 2019). Costs
are recouped through a combination of grants, bond financing, and rate increases spread over all utility
customers. While voluntary participation is typical, some cities are beginning to require mandatory
replacements for confirmed LSLs while fully covering the costs. For example, Newark, New Jersey, adopted
a free and mandatory approach including fines for non-compliance (City of Newark 2019).
4 Construction cost estimates provided in 2022 dollars.
5 Participants in roundtable discussions held in ten Northeastern and Midwestern communities by the US
Environmental Protection Agency (EPA 2021b-2021k) mentioned financial constraints, language barriers, landlord-
renter split incentives, and lack of community trust in water utilities as barriers to LSLR. Participants in these sessions
suggested subsidies, multiple types of outreach approaches (e.g., mailers, in person, online, door hangers),
translation of materials into different languages, and partnerships with organizations viewed as "trusted
messengers" in target communities as potential solutions to these barriers.
-------
The existing literature on barriers to LSLR and ways to address them is minimal. One study investigating
neighborhood characteristics associated with LSLR found that replacement rates in Washington, DC, were
significantly higher in neighborhoods with higher incomes and a lower proportion of Black residents
(Baehler et al. 2022). A discussion of the LSLR program in Flint, Michigan, by the Federal Reserve Bank of
Chicago (2022) posits that the voluntary nature of the program contributes to homeowner non-responses
and refusals to have their lines inspected or replaced.
Researchers have investigated the effectiveness of approaches to encourage testing and treatment of
private wells for a wide range of contaminants. Renaud, et al. (2011) examined two programs to
encourage testing for arsenic in Quebec via quasi-experimental empirical techniques. They found a
significant increase in testing (but from a relatively low baseline) in response to targeted information
supplied to households relative to a mass media campaign alone. However, even with the mass media
campaign, they found evidence that having a neighbor who had previously tested their well had a strong
influence on the uptake of testing. MacDonald and Tippett (2020) mailed letters to a random sample of
New Jersey residents with private wells to examine if information on contamination at a nearby well
(versus general information on contaminants of concern) encouraged testing. They found that nearby well
contamination had a significant, positive influence on the likelihood to test regardless of the contaminant.
However, testing (and treatment) remained low due to lack of concern regarding the health effects of
contamination. A follow-up survey found that many households relied on bottled drinking water.
Respondents to a survey administered in Southern Ontario by Krewtzwiser, et al. (2011) indicated that
even when testing is free, inconvenience is an important barrier. In rural areas, drop-off points were far
away and only open during business hours. Hexemer, et al. (2008) distributed free sampling kits to private
well owners in Southern Ontario and then collected samples the next day for testing. While removing cost
and travel time barriers substantially increased participation, they still found that less than 50 percent of
targeted households returned the test kits. A follow-up survey indicated that non-participants often were
not home when kits were distributed and/or unable to sample their wells for next-day pickup. Household
income and education did not affect participation.
Finally, Severtson, et al. (2006) administered a survey to a random sample of participants in a Wisconsin
program to test for arsenic in private wells. They found that only one-third of those with elevated
contaminant levels treated their water. A lack of observed health effects among long-term residents or
obvious indicators of contamination (odor or taste) were listed as common reasons for failing to treat. In
addition, female respondents to a post-program survey reported higher levels of worry about the
potential health effects of arsenic exposure but also perceived greater financial and nonfinancial barriers
to taking action to reduce exposure, while more educated respondents reported the opposite.
In the context of energy efficiency, several studies also have investigated interventions to increase
household participation in inspections and subsequent adoption of energy saving technologies for the
home. Fowlie, et al. (2015; 2018), Holladay, et al. (2016), and Allcott and Greenstone (2017) implemented
randomized field experiments to offer a combination of subsidies and personalized assistance to
encourage program take-up. Fowlie, et al. (2015) found that relatively few households participated in the
federal Weatherization Assistance Program even when financial costs were fully subsidized. They posited
that this may be due to non-monetary costs such as the time needed to fill out the application and for the
-------
audit itself. Likewise, Fowlie et al. (2018) found that extensive outreach and assistance in filling out
applications to presumptively eligible households in Michigan - in combination with free audits and low
interest rate loans to cover the cost of recommended energy-efficient investments - did not substantively
increase program participation. Holladay, et al. (2016) found that individualized letters comparing
households' electricity use with that of their neighbors increased the likelihood that a household
scheduled an audit by two times the very low baseline rate. The offer of gift cards did not affect take-up
of the audit, even when fully subsidized. Offers of substantial rebates for upgrades based on audit
recommendations did not affect audit likelihood or subsequent purchases. Using a combination of letters
and offers of subsidies to households in Wisconsin, Allcott and Greenstone (2017) found a small,
marginally significant increase in the number of energy audits performed, but the content of the letter
did not matter. Offering a gift card to encourage audits had no effect. Interestingly, households that were
induced to be audited via the experiment were much less likely to make a subsequent investment than
those that voluntarily signed up for the program.
Gillingham and Tsvetanov (2018) and Wallander, et al. (2017) implemented randomized field experiments
that focused on relatively "low touch" forms of intervention to encourage pro-environmental behavior.
Gillingham and Tsvetanov (2018) sent reminder postcards to households in Connecticut that had signed
up for but not yet received an energy audit, noting a high attrition rate (60%) between initial sign up and
the audit itself. Sending a reminder notecard resulted in a small (1.1 percentage points) but significant
increase in the probability of following through with the audit for the average household. The method of
communicating that message (e.g., basic reminder vs. use of social norms) did not matter. However, they
found evidence of differential treatment effects: higher income and non-White households were more
responsive. Wallander, et al. (2017) sent reminder letters to farmers with expiring Conservation Reserve
Program contracts to encourage re-enrollment in the program. Like Gillingham and Tsvetanov (2018), they
found a small increase in sign-ups (1.4 to 1.9 percentage points) in response to the letter but no evidence
that the form of the message mattered.
3 Case Study: LSLR in a large New Jersey water system
3.1 Trenton-Area LSLR Program
Our case study involves a large public water system in New Jersey serving more than 60,000 properties in
Trenton and four neighboring suburban municipalities. Water system records indicated that LSLs were
commonly installed in homes built before I960.6 Roughly half of service lines connecting individual
properties to water mains were originally estimated to contain lead. The water system launched an LSLR
program in 2019 in response to exceedances of EPA's action level for drinking water lead concentrations
in 2017 and 2018. By the end of our study period in July 2022, the system had conducted more than 17,000
service line inspections and replaced more than 9,000 lines found to contain lead, including 2,500
homeowner-side lines.
6 Most service lines classified as "lead" by the water system are actually lead-lined galvanized steel lines that pose
similar risks to water quality as solid lead service lines. The State of New Jersey also considers galvanized service
lines as LSLs (httpsi//www,ni,gov/dep/lead/notices,html, accessed Jan. 17,2023). We follow this definition and refer
to lead-lined galvanized steel service lines as LSLs.
-------
The water system offered replacement of homeowner-side LSLs for a subsidized cost of $1,000 payable
over five years without interest.7 Customer participation in the LSLR program was voluntary and offered
to all residents in Trenton and three suburban areas served by the water system with LSLs.8 In addition to
this partial subsidy offered directly by the water system, there were two additional grant programs during
our study period that covered the remaining cost of homeowner-side LSLRs to certain, qualifying Trenton
residents. These programs are described below.
The water system conducted various forms of outreach to encourage customers to register for the
program, including mailings, community meetings, and door-to-door visits. The water system sent two
targeted letters in 2018 and 2019 to the addresses of properties suspected of having LSLs to encourage
program registration.9 Program registration did not entail a formal commitment by the homeowner to
pay the $1,000 cost and proceed with replacement (if the line was confirmed to be lead), nor was
registration required for an inspection or eventual replacement - it served simply as a signal of interest.
The water system prioritized properties that registered for the program prior to 2020 for inspection and
replacement efforts. It also distributed contractor work throughout the service area to ensure that
inspections and replacements occurred in all four municipalities with LSLs. LSLR construction work started
in February 2020, paused during March 2020 due to the COVID-19 pandemic, then resumed in July 2020.
As of July 2022, it had conducted some inspections and replacements in almost all neighborhoods in the
service area.10
The homeowner-side service line material was often unknown when customers registered for the
program. Consequently, customer cooperation was necessary for material identification or verification,
typically with residents needing to allow a water system contractor or employee inside the home to
inspect the pipe. Alternatives to an interior contractor inspection included a self-inspection photo sent by
the homeowner to the utility (an option introduced by the utility in 2020 in response to the COVID-19
pandemic), or an external excavation that required no cooperation from the homeowner. If an LSL was
confirmed by the inspection, the homeowner was asked to sign a contract and right-of-entry agreement
to replace the line. A resident was required to be present during the replacement, which typically took
about half a day to complete. Construction included patching of walls or floors and restoration of
pavement or sod damaged during the replacement. For properties confirmed to have lead on the utility-
side of the line and a non-lead material on the homeowner-side, the water system could conduct a
replacement of the utility side without accessing the property, so further homeowner participation was
not needed.
7 Replacements of utility-side LSLs were completed at no cost to the homeowner.
8 Water system records indicate that there are no lead service lines in one of the suburban municipalities of the
water system. We exclude this municipality from the analysis.
9 Because mailings were sent to property addresses, renters rather than landlords received them at non-owner-
occupied properties.
10 An interior inspection and/or replacement occurred in at least one home in 95% of Census block groups in the
study area by the end of the study period. These block groups included over 99% of accounts in our final analysis
sample.
-------
3.2 Community-Based Grant Program
In March 2021, a trusted community organization active in the East Trenton neighborhood for over a
decade launched a grant program to cover 100% of the $1,000 LSLR cost for owner-occupied homes and
50% of the cost for renter-occupied homes (with landlords covering the remainder) in that neighborhood.
The grants involved minimal paperwork and no eligibility requirements besides location of the property
in the community organization's target neighborhood. This neighborhood, which is comprised of four U.S.
Census block groups, was home to a higher share of Black and Hispanic residents and those living below
the poverty level than the remainder of the city. The community organization conducted extensive
outreach through in-person and virtual events, door-to-door and neighborhood canvassing, and social
media to notify residents about the program. We subsequently refer to this subsidy program as the
"community-based grant program."
3.3 Housing Department Grant Program
In addition, Trenton's Department of Housing and Economic Development offered an existing program
funded by a U.S. Department of Housing and Urban Development Community Development Block Grant
to subsidize urgent home repairs for low- and moderate-income homeowners. LSLR was added as a
qualifying home repair in 2021, meaning that the entire $1,000 cost to customers would be covered for
successful applicants. Applicants were required to provide extensive documentation, such as W-2s and
tax returns, to prove their eligibility. Only owner-occupied properties in Trenton were eligible. Information
about the urgent repair grant program was publicly available on the city government website but was not
otherwise advertised, and the program had not been widely utilized.11 We subsequently refer to this
subsidy program as the "housing department grant program."
3.4 Study Objectives
Based on observations from the existing literature, we test the following hypotheses within the context
of the Trenton LSLR program:
Hypothesis 1: Renters, properties with lower assessed value, and customers living in
neighborhoods with higher poverty, lower educational attainment, and other indicators of
economic disadvantage are less likely to participate in the LSLR program, absent additional
interventions, due to greater financial, informational, and transactional barriers.
Hypothesis 2: Households receiving outreach letters to register for the LSLR program participate
at greater rates due to reduced informational barriers.
Hypothesis 3: Subsidies to cover the $1,000 cost-share will mitigate financial barriers for low-
income customers, leading to higher program participation among customers offered a subsidy
through either the community-based grant program or the housing department grant program.
Hypothesis 4: Low-income households are less likely to participate in the LSLR program unless
both financial and non-financial barriers are addressed. Thus, we expect a larger response to the
11 A housing department representative indicated that there are fewer than 50 applicants for urgent rehabilitation
grants in a typical year (personal communication, Farrah Gee, City of Trenton Department of Housing and Economic
Development).
-------
community-based grant program than to postcards providing a single notification about the
housing department grant program, which had greater transactional barriers.
4 Data
We assembled the dataset by linking individual water system account records with property tax
assessment data and Census neighborhood characteristics. We obtained data from the water system on
62,529 customer accounts reflecting LSLR program status as of July 2020, May 2021, and July 2022.12 The
data included property address, property type (residential or non-residential), account status (active or
inactive), utility-side and homeowner-side service line material prior to any replacement work (if known),
LSLR program registration and date of response, and date of utility-side and/or homeowner-side
replacement (if applicable). We also obtained data from the water system on which properties were sent
letters and postcards about the LSLR program and which street segments in the service area were under
a moratorium for non-emergency water line construction due to recent street paving.
Using street address, city, and zip code, we linked 91% of water system accounts to county property tax
assessment records for tax year 2020.13 Tax assessment records included data on property type, number
of dwellings, year constructed, assessed value, location address, and property owner address. We
compared location address and owner address to identify owner-occupied properties.14 We excluded
duplicate accounts, as well as accounts that either tax assessment or water system records identified as
non-residential properties, or apartment buildings with more than four dwellings (7% of accounts).15 As
noted above, we also excluded all properties located in one suburban municipality where the water
system determined that LSLs were never installed based on the dates of water main and service
connection installations (3% of accounts).
We incorporated data on neighborhood sociodemographic characteristics at the block group level from
the 2015-2019 Census American Community Survey (ACS) 5-year estimates. We geocoded property
addresses and identified Census block groups using the Census Geocoder batch address processing tool.16
We excluded accounts in one block group that ACS data indicated had no occupied housing units (less
than 1% of accounts). The organization implementing the community-based grant program provided
neighborhood boundaries, which corresponded to four Census block groups. Our final analysis dataset
12 The research team also obtained weekly water system data pulls from September 2021-April 2022 that were not
used in this analysis.
13 Observations across the two datasets were linked based on an exact string match of street address and city name,
and when, available in the assessor data, zip code. We did extensive data cleaning to fix typos and to increase
consistency in street type abbreviations across datasets to attain a high merge rate. Water accounts that were
unable to be linked may have remaining differences in spellings, abbreviations, or other characteristics that we were
unable to identify.
14 We categorized properties as owner-occupied if the first seven characters of the property address exactly matched
the first seven characters of the owner address. This indicator is extremely similar to a variable denoting a match of
the entire character string of the property and owner addresses (rho = 0.99) but allows for flexibility due to spelling
mistakes or differences in the way apartment numbers are recorded across the datasets.
15 We excluded residential buildings with more than four dwellings from the analysis because LSLs typically serve
single-family or small multi-family residences due to their relatively small diameter; larger buildings require larger
diameter service lines that were rarely made of lead (EPA 2022b).
16 https://geocoding.geo.census.gov/geocoder/geographjes/addressbatch7form [Accessed Jan. 22, 2022]
-------
consisted of 55,917 water system accounts; 91% of these were residential properties of four families or
less as indicated by property assessment records, while 9% were not able to be linked to the assessment
data.
5 Methods
We estimated the parameters of a series of regression models, first to examine the characteristics of
households willing to participate in the LSLR program, and then to evaluate the impact of two specific
programs intended to subsidize the cost of LSLR for certain eligible properties.
5.1 Observational Analysis
Our first analysis used the full-sample dataset to examine the factors associated with three outcome
variables reflecting different aspects of program participation: registration for the LSLR program,
participation in an interior home inspection conducted by a water system contractor or employee, and
completion of an LSLR conditional on having an LSL. We regressed each participation variable on a set of
property and neighborhood characteristics in a linear probability model17 as follows:
yij = Po + Ptxi + Pizj + £ij (i)
In equation (1), ytj is one of the three participation outcomes observed for property i in neighborhood j
as of July 2022, the end of our study period. The vector Xt represents individual characteristics of the
property. These characteristics include whether the account was inactive at any point during the study
period (denoting a vacant property),18 the number of letters the property was sent to encourage
registration in 2018 or 2019 (prior to the start of the LSLR program), and whether the property was located
on a street segment where non-emergency service line repairs were under moratorium during 2020 and
2021 due to recent street paving.19 We also included a variable denoting whether the utility-side of the
service line contained lead; contractors had incentivizes to seek out these properties because they would
be able to conduct a replacement even if the homeowner-side of the line was found to be lead free.
Furthermore, if the homeowner side was found to be an LSL, they would be paid more for the work
because both sides of the line required replacement. For properties linked to tax assessment data, we
included the year built (pre-1951 and 1951-1960, after which LSLs were uncommon in the study area), the
natural logarithm of assessed value, and indicators for owner-occupied and multi-unit (2-4 family)
properties. We included a dummy variable denoting missing assessment data. We imputed missing
assessment data values using the average value for linked residential properties in the same
municipality.20 The vector Zy represents neighborhood characteristics at the Census block group level.
17 While other estimators could be used in the case of a binary outcome variable, such as probit or logistic, the linear
probability model is attractive in our observational setting that includes a mix of continuous and categorical variables
while also offering easily interpretable marginal effects (Angrist and Pischke 2008).
18 Accounts are considered inactive by the water system when a property is abandoned or when a property owner
requests that the water be shut off due to vacancy. Failure to pay water bills did not trigger inactive status or water
shutoff in the study area during the study period.
19 We considered a property to be affected by a street paving moratorium if it was located within 50 meters of a
street segment under moratorium.
20 The results of the analysis are not substantively different if we instead exclude observations unable to be linked
to the assessment data.
-------
These characteristics include percent of residents that are Black, percent of residents of Hispanic ethnicity,
percent living below the poverty level, percent that are below age 5, percent above age 64, percent of
residents age 25 and over with a college degree, and percent of the occupied housing stock occupied by
renters. The /? terms are coefficients to be estimated, and Eij is heteroskedasticity-robust error term. We
estimated equation (1) separately for the urban and suburban properties in our analysis to allow for
heterogeneity in the association of property and neighborhood characteristics with LSLR program
participation.
For two of the three outcomes in our analysis—registrations and inspections—we included the full
samples of urban and suburban properties regardless of service line material in the regression models,
since material was typically unknown prior to inspection. The third outcome, replacement, is only relevant
for properties found to have a homeowner-side LSL, which customers can then decide to leave as-is or
replace. Therefore, we estimated the LSLR outcome equation restricting the sample to properties whose
homeowner-side service line was confirmed to contain lead, excluding those whose line was of unknown
material or was confirmed to not contain lead. Focusing on this smaller sample allows us to isolate factors
affecting the decision of whether to proceed with LSLR conditional on having an LSL. Conditioning the
sample in this way provides the most relevant interpretation for water systems and policymakers
interested in understanding the willingness to participate in LSLR among the most relevant customer
population. We included the same property and neighborhood characteristics as explanatory variables,
except for housing age because 98% of properties with a confirmed LSL were built before 1961.
5.2 Program Evaluations
Next, we evaluated the impacts on willingness to participate in the LSLR program of two programs offered
to certain residents in the urban core of the service area during the study period: a community-based
grant program for residents of a specific neighborhood and a housing department grant program for low-
and moderate-income homeowners.
5.2.1 Synthetic control to evaluate impact of community-based grant program
We conducted a quasi-experimental evaluation of the community-based grant program on LSLR program
participation in the water system's urban municipality.21 We estimated a difference-in-difference (DiD)
linear probability model:
yijt = Yo+ YiTreatj + Y2^ftert + STreatj * Aftert + (2)
In equation (2), represents each of the same three participation outcomes already discussed but
includes subscript fto denote time. The model includes Treat to denote location in one of the four Census
21 The sample used for the quasi-experimental community-based grant program evaluation consisted of all urban
properties in our final analysis sample except for 1,338 properties that received postcards about the housing
department grant program as part of the field experiment, 201 properties receiving other special outreach mailings
from the water system after the launch of the community-based grant program, 80 properties whose program
registration and LSLR dates could not be determined from the water utility account data, and 22 properties in
neighborhoods that were not visited by water system contractors during both the pre- and post-grant periods. The
22 properties in neighborhoods never visited by a contractor were not in the target neighborhood; however, 13 of
these properties did receive at least one outreach letter encouraging program registration in 2018 or 2019.
-------
block groups that comprise the neighborhood eligible for the grant program and After to represent the
period after the grant program's launch. The coefficients y0, ylt and y2 represent city-wide LSLR
participation pre-launch, the difference in LSLR participation in the target neighborhood from the rest of
the city pre-launch, and the city-wide increase in LSLR participation post-launch, respectively. The key
coefficient is S, which represents the impact of the grant program on participation in the target
neighborhood. £ is a cluster-robust error term that is clustered at the level of the block group to match
the spatial scale of the grant intervention (Abadie et al. 2022).
Because location in the treatment neighborhood is not random and may be correlated with factors
affecting LSLR program participation, we used a synthetic control approach to estimate the causal effect
of the community-based grant program. A synthetic control is a weighted average of observations in the
untreated sample that serves as a counterfactual for the treatment group (Abadie 2021). In settings with
many treated units (as is the case in this study), the weights balance the mean values (or other moments)
of a set of observed characteristics across the treated and synthetic control groups (Hainmueller 2012;
Robbins, Saunders, and Kilmer 2017). To implement the synthetic control approach, we estimated the DiD
model as a weighted regression.22
We used entropy balancing to derive the weights for our synthetic control group. Entropy balancing is a
matching method that identifies a set of non-negative weights that satisfy a set of balance constraints (in
our case, equality of means of several variables), that sum to the number of observations in the treatment
group and are as close as possible to uniform (Hainmueller 2012). We constrained the treatment and
synthetic control groups to have equal mean values across all three pre-intervention outcomes, all
property-level characteristics, and the neighborhood characteristics found to have a statistically
significant association with LSLR program participation in the urban core.23 Because the weights exactly
balance the characteristics of all variables associated with LSLR program participation across the
treatment and synthetic control groups, it is not necessary to include control variables in equation (2) to
obtain an unbiased estimate of S.24
22 In the appendix, we also present results of a standard unweighted DiD model that controls for property
characteristics and block group fixed effects, as well as results using an alternative set of weights derived using
coarsened exact matching (lacus et al. 2012).
23 Note that we examined the LSLR completion outcome using a restricted sample of properties confirmed to have
a homeowner-side LSL and the other outcomes using the broader sample of properties regardless of having a
confirmed LSL. Consequently, we derived two sets of weights using entropy balancing corresponding to these two
samples. The weights used with the broader sample to examine the likelihood of registration and inspection were
derived using all three pre-treatment outcomes, all individual property characteristics, and all neighborhood
characteristics except for share of the population under age 5 because this variable was not a significant predictor
of any of the outcomes in the urban area (Table 2, columns 1 and 2). The weights used when examining LSLR
conditional on having an LSL were derived using all three pre-treatment outcomes, all property characteristics, and
the share of occupied housing that is renter occupied because this is the only neighborhood characteristic that is
significantly associated with LSLR uptake in this sample (Table 2, column 3). The two sets of weights are highly
correlated (rho = 0.82).
24 In the appendix, we also present results of a model that included property characteristics and block group fixed
effects.
-------
5.2.2 Experimental design to evaluate impact of housing department grant program
The second evaluation is a field experiment to assess the impact of postcards informing residents about a
program run by the city's housing department to cover homeowner-side LSLR costs (and other urgent
home repairs) for low- and moderate-income homeowners. The research team worked with the water
system to identify a sample of 3,100 properties located in the urban core that were potentially eligible for
the housing department grant program but outside of the neighborhood targeted by the community grant
program.25 The sample of 3,100 properties was stratified by block group and then split into equally sized
treatment and control groups. Rerandomization (Morgan and Rubin 2012) was used to ensure balance on
pre-intervention LSLR program registration, participation in an inspection, and, for properties linked to
assessor data, the inflation-adjusted home sales price and an indicator for pre-1951 construction.
The water system sent English-language postcards to the treatment group providing information on the
availability of the grant program to cover the costs of LSLR, eligibility criteria, information on how to apply,
and a brief reminder about the health benefits of LSLR (Appendix Figure A3). Properties in the control
group did not receive an LSLR mailing at that time. For 15% of the treatment group, either the mailing
address was deemed invalid by the firm contracted to print and mail the postcards, or the postcard was
returned to the water system by the postal service, so this portion of the treatment group did not receive
the mailing.
To evaluate the impact of notifying customers about the availability of the housing department grant
program on participation in the LSLR program, we estimated the following equation:
Vijt = Po + YiTreati + y2Aftert + STreatt * Aftert + Etj (3)
Equation (3) is similar to equation (2), except that the treatment, Treat—being sent a postcard with
information about the housing department grant program to cover the cost of LSLR—was assigned at the
individual rather than neighborhood level. It is again unnecessary to include control variables or block
group fixed effects in this equation to estimate S, the impact of notification about the urgent repair grant
program, because the intervention was randomized within block groups and balanced on key observable
characteristics related to LSLR program participation.26 The heteroskedasticity-robust error term, s, is not
clustered at the neighborhood level, consistent with random assignment of individual properties to the
treatment after stratifying by block group.
Because 15% of the treatment group did not receive the postcard, equation (3) is more appropriately
characterized as an "intent to treat" model. To address the treatment contamination resulting from non-
25 To obtain the sample for the randomized controlled trial, we excluded properties that met at least one of the
following criteria prior to the intervention: location outside of the urban municipality; location inside the target
neighborhood for the community-based grant program; the homeowner-side service line was confirmed to not
contain lead; a LSLR already occurred; the water system account was inactive, vacant, or a non-valid address
according to water system records; the property was not owner-occupied according to assessor data or the
municipality's registry of rental properties; occupants had previously refused water system staff or contractors
access to the property; the property was located in a Census block in which no properties had previously registered
for the LSLR program.
26 In the appendix, we also present results of a model that included property characteristics and block group fixed
effects.
-------
delivered postcards, we estimated (3) using an instrumental variables (IV) model with receipt of the
postcard as the treatment variable and assignment to the treatment group as the instrument (Sussman
and Hayward 2010).
6 Results
6.1 Characteristics of LSL Incidence and LSLR Participation
Table 1 summarizes the characteristics of all properties in our final analysis sample. It disaggregates by
homeowner-side service line material (prior to the completion of an LSLR, if applicable) to shed light on
whether properties with a confirmed LSL systematically differed from those with a confirmed lead-free
service line or a service line of unknown material.
Forty percent of properties with confirmed homeowner-side lead lines registered for the LSLR program,
in contrast to 27% of properties with confirmed non-lead lines (who presumably registered before
learning their line did not contain lead), and 2% of properties with unknown material. About two-thirds
of properties had their homeowner-side material confirmed as either lead or non-lead through an interior
contractor inspection (as opposed to an external excavation or a self-inspection photo sent by the
homeowner to the utility).27 Forty-three percent of confirmed homeowner-side LSLs were replaced by the
end of the study period. Most, but not all, customers who had a homeowner-side LSLR had registered for
the program; while registration was encouraged, it was not a required step.
Most properties with confirmed homeowner-side LSLs were sent at least one letter encouraging
registration prior to the launch of the LSLR program (out of a maximum of two letters). Properties with
non-lead and unknown material were less likely to receive these outreach letters. This is unsurprising
given that the utility targeted letters to properties with suspected homeowner-side LSL based on age of
the home and the utility-side service line material. Properties with confirmed homeowner-side LSLs were
also somewhat less likely to have inactive accounts during the study period.
Fifty-nine percent of properties with a homeowner-side LSL also had lead on the utility-side of the line
(prior to any replacement work). The percentage was similar for properties with a non-lead homeowner-
side line. This similarity is surprising, though it could be due to the fact that the water system targeted
properties with utility-side LSLs for inspections and outreach because they suspected these properties
were also more likely to have lead on the homeowner side. Utility-side LSLs were much less common for
properties with unknown homeowner-side material. The data on utility-side service line material are
mostly comprised of historic utility records, with only 20% verified by a contractor inspection, so it could
contain inaccuracies.
27 We consider homeowner-side service line material to be "confirmed" if it was inspected by a utility employee or
contractor (interior visual inspection or exterior excavation) or a homeowner visual inspection documented in a
photo submitted to the utility. We do not consider historic utility records to be sufficient for service line material
confirmation.
-------
While most homes in the study area were built before 1951, properties with confirmed homeowner-side
LSLs were especially likely to fall into this category. Most of the remaining homes with homeowner-side
LSLs were constructed by I960.28
Table 1. Full-sample summary statistics disaggregated by pre-construction homeowner-side service line
material
Confirmed lead Confirmed non-lead Material unknown
Observations 5,769 15,761 34,387
Mean
Std. Dev.
Mean
Std. Dev.
Mean
Std. Dev.
LSLR participation outcomes
Registered for LSLR program
40%
0.49
27%***
0.44
2%* * *
0.13
Interior contractor inspection
66%
0.48
67%
0.47
0%***
0.02
Homeowner-side LSLR completed
43%
0.50
0%***
0.00
0%***
0.00
Property characteristics
Lead on utility side of line
59%
0.49
58%
0.49
28%***
0.45
Number of LSLR letters0
1.15
0.80
1 09***
0.82
0.60***
0.74
Account inactive
8%
0.27
7%
0.26
6%***
0.25
Street paving moratorium
10%
0.30
8%***
0.27
12%***
0.33
Suburban location
46%
0.50
59%***
0.49
68%***
0.47
Assessor data unlinked
4%
0.20
6%***
0.24
11%***
0.31
Property built before 1951f
87%
0.33
59%***
0.49
55%***
0.50
Property built 1951-1960f
10%
0.30
23%***
0.42
16%***
0.37
Owner-occupied property!
66%
0.47
71%***
0.46
71%***
0.45
Multi-family property!
5%
0.21
4%***
0.19
5%
0.21
Assessed value ($100,000)!
1.11
0.68
2 39***
0.78
1.58***
0.93
Block group characteristics
Share Black
36%
0.27
31%***
0.27
29%* * *
0.27
Share Hispanic
31%
0.24
23%***
0.23
22%***
0.22
Share under 5
7%
0.05
6%***
0.04
6%***
0.04
Share over 64
11%
0.06
13%***
0.07
13%***
0.08
Share rental
45%
0.23
41%***
0.23
39%***
0.24
Share college graduate
19%
0.13
25%***
0.16
25%***
0.17
Share below poverty
18%
0.14
15%***
0.14
15%***
0.13
¦fData presented only for properties linked to assessor data. OLSLR letters refers to mailings sent by the utility in
2018 and 2019 to properties suspected of having an LSL to encourage program registration rather than outreach
conducted through the community-based grant program or the housing department grant program. *** p<0.01, **
p<0.05, * p<0.1 denotes a statistically significant difference compared to properties with confirmed homeowner-
side lead lines.
More than half of properties with confirmed homeowner-side LSLs were located in the urban core, while
the majority of properties with non-lead or unknown material were located in the suburbs. Properties
28 While not presented in the table, we note that, among properties confirmed not to have lead on the homeowner
side, those with a LSL on the utility side were more likely to have been built pre-1951 than those without lead on the
utility side.
-------
with confirmed LSLs had lower rates of owner occupancy, lower assessed values, and were in
neighborhoods with a higher share of Black and Hispanic residents, rental housing, young children, and
poverty and a lower share of college graduates and elderly residents. These results suggest that LSLs were
more prevalent in economically disadvantaged neighborhoods and could pose an environmental justice
concern in the water system service area. The results are similar to an analysis of block group
characteristics in four cities that found that likelihood of having an LSL was associated with a higher share
of older homes, multi-unit buildings, and poverty in all four cities and a higher share of minority residents,
renters, and adults without a college degree in three out of the four cities (GAO 2020).
When we further disaggregate the summary statistics by urban/suburban location as well as service line
material (Tables Ala and Alb in the Appendix), we find that properties in the urban core were somewhat
more likely to register for the program and have a contractor inspection than those in the suburbs, while
the two locations had similar rates of replacement of confirmed lead lines. Irrespective of service line
material, urban properties were more likely to be renter-occupied, multi-family buildings constructed
before 1951 with lower assessed values and to be located in neighborhoods with higher rates of Black and
Hispanic residents, people living below the poverty level, and lower rates of college graduation compared
to properties in the suburbs.
Table 2 presents the estimated coefficients from the regression models exploring the characteristics
associated with LSLR program participation, estimated separately for properties in urban (i.e., Trenton)
and suburban municipalities. We present results from an alternative model in the Appendix (Table Ale)
that includes urban and suburban properties in the same regression and interacts all predictor variables
with an indicator for suburban to test for heterogeneity in characteristics affecting LSLR program
participation by location. The results are not sensitive to this alternate specification but show that the
magnitudes of the coefficients for several property and neighborhood characteristics are statistically
different across the urban and suburban samples, even though the coefficients for the property
characteristics often have the same sign across the two areas.
Table 2 reveals several commonalities in the characteristics of properties that participated in the LSLR
program across urban and suburban areas.29 Properties in both locations that registered and had interior
inspections of their service lines were significantly more likely to be built pre-1951 or 1951-61 and to have
lead on the utility side of the service line. Neither result is surprising; pre-1961 homes are more likely to
have homeowner-side LSLs, and contractors had incentives to encourage registration and conduct
inspections at homes with utility-side LSLs, where they would be guaranteed some construction work and
would receive a higher payment if a full line replacement was needed. In addition, properties that received
outreach letters from the water system about the LSLR program were much more likely to register and
have an inspection than properties not sent letters, even after controlling for home age and utility-side
service line material (characteristics used by the water system to target properties for LSLR outreach).
29 Figure A1 in the Appendix provides an illustration and comparison for a subset of these coefficients.
-------
Table 2: Regression analysis of determinants of LSLR program participation
Urban municipality Suburban municipalities
(1)
(2)
(3)
(4)
(5)
(6)
Registration
Inspection
Replacement
Registration
Inspection
Replacement
of confirmed
of confirmed
LSL
LSL
Number of LSLR
0.027***
0.052***
0.068***
0.099***
0.148***
0.114***
letters
(0.005)
(0.006)
(0.014)
(0.004)
(0.004)
(0.017)
Account inactive
-0.024***
0.002
-0.027
-0.005
-0.017**
-0.010
(0.008)
(0.010)
(0.028)
(0.007)
(0.008)
(0.033)
Lead on utility
0 179***
0.359***
0.350***
0.037***
0.171***
0.353***
side
(0.007)
(0.008)
(0.021)
(0.006)
(0.007)
(0.029)
Street paving
-0.001
-0.063***
-0.009
-0.015***
-0.126***
-0.128***
moratorium
(0.008)
(0.009)
(0.023)
(0.005)
(0.005)
(0.029)
Owner-occupied
0.072***
0.037***
0.212***
0.023***
0.043***
0.076***
property
(0.006)
(0.007)
(0.017)
(0.004)
(0.006)
(0.023)
Multi-unit
-0.064***
-0.137***
0.005
0.002
-0.095***
-0.101
property
(0.010)
(0.011)
(0.033)
(0.018)
(0.020)
(0.086)
Log assessed
0.065***
0.046***
0.046**
0.014***
0.032***
-0.049*
value
(0.008)
(0.009)
(0.022)
(0.005)
(0.006)
(0.030)
Property built
0.100***
0.139***
0.043***
0.068***
before 1951
(0.009)
(0.011)
(0.004)
(0.005)
Property built
0.092***
0.209***
0.016***
0.075***
1951-1960
(0.020)
(0.022)
(0.004)
(0.005)
Assessor data
-0.088***
-0.136***
-0.073
-0.008
-0.044***
-0.032
unlinked
(0.011)
(0.013)
(0.053)
(0.005)
(0.007)
(0.037)
Share Black
0.159***
0.069**
-0.101
-0.006
-0.113***
-0.290***
(0.024)
(0.027)
(0.070)
(0.010)
(0.012)
(0.053)
Share Hispanic
0.068**
0.046
-0.058
0.065***
0.032*
-0.115*
(0.028)
(0.032)
(0.084)
(0.016)
(0.019)
(0.069)
Share under 5
-0.088
0.075
-0.120
0.007
-0.333***
0.376*
(0.064)
(0.072)
(0.195)
(0.042)
(0.052)
(0.205)
Share over 64
0.012
0.289***
0.072
-0.016
-0.060**
0.240*
(0.049)
(0.056)
(0.165)
(0.022)
(0.028)
(0.135)
Share rental
-0.105***
-0.089***
-0.109**
-0.001
0.054***
0.165***
housing
(0.017)
(0.020)
(0.050)
(0.010)
(0.012)
(0.051)
Share college
0.186***
0.241***
-0.059
0.053***
0.022
-0.031
graduate
(0.044)
(0.049)
(0.128)
(0.013)
(0.016)
(0.077)
Share below
0.164***
0.079***
0.092
-0.036
-0.069**
-0.150
poverty
(0.025)
(0.029)
(0.074)
(0.023)
(0.028)
(0.132)
Constant
-0.111***
-0.043
0.182**
-0.043***
-0.006
0.081
(0.028)
(0.032)
(0.083)
(0.011)
(0.014)
(0.053)
Observations
20,529
20,529
3,098
35,388
35,388
2,671
R-squared
0.103
0.211
0.237
0.116
0.270
0.313
Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1
-------
Single-unit, owner-occupied properties with higher assessed values were also more likely to register and
have an inspection compared to non-participating properties in both urban and suburban areas. These
properties were also less likely to be on a block with a street construction moratorium, where water
system contractors were not supposed to conduct LSLRs. Urban properties with an inactive water system
account were significantly less likely to register, and suburban properties with an inactive account (where
there might have been no one in residence to let in a contractor) were significantly less likely to have an
inspection. Participation among properties that we were unable to link with property assessor records
was lower than those we successfully merged. We note that unlinked data could include some non-
residential properties because the assessor data was our primary data source for determining property
type; the water system included some data on property type, but it was sparse.
Urban properties in neighborhoods with higher shares of Black residents, college graduates, and residents
below the poverty line and lower shares of rental properties were more likely to register and have an
inspection. Registration rates were also higher in urban neighborhoods with higher shares of Hispanic
residents. Urban properties that had inspections also tended to be in neighborhoods with higher shares
of elderly residents, which could indicate that retirees had fewer constraints when scheduling an
inspection. A higher share of young children in the neighborhood was not significantly associated with
LSLR program participation in the urban core.
In suburban municipalities, location in a neighborhood with a higher share of Hispanic residents and
college graduates was associated with higher registration rates, similar to results from the urban core.
However, unlike the urban results, being located in a neighborhood with a higher share of Black residents,
young children, elderly residents, and people below the poverty level and a lower share of rental
properties was associated with lower inspection rates in the suburbs.
Columns 3 and 6 show the factors associated with participation in the LSLR program—the most important
outcome in our study—in urban and suburban areas, respectively. The coefficient estimates show that
properties receiving more outreach letters and those with lead on the utility side of the line were more
likely to have a homeowner-side LSLR. Contractors may have targeted these homes since they were paid
a higher rate by the utility for replacements involving both sides of the service line. Owner-occupied
properties had significantly higher replacement rates. These findings are all consistent across urban and
suburban properties.
In the urban core, LSLR was more common among properties with higher assessed values and in
neighborhoods with a lower share of rental housing, but other neighborhood characteristics were not
significantly associated with LSLR in the urban area. Suburban properties with confirmed higher assessed
valued were slightly less likely to have an LSLR than those with lower assessed value, which is surprising,
but this result is only marginally significant. Properties in suburban neighborhoods with higher shares of
Black or Hispanic residents were less likely to participate in a replacement, and those with a higher share
of young children and elderly residents were more likely to have a replacement, though the results for
percent Hispanic, young children, and elderly residents are only marginally significant. Suburban
properties with confirmed homeowner-side LSLs located in neighborhoods with a higher share of rental
properties were also more likely to have a LSLR. The positive association between replacements and
neighborhood-level rental housing is counterintuitive, but recall that our regression controls for whether
the property itself is owner-occupied. The Census rental variable therefore captures any additional effect
-------
associated with living in a neighborhood with a high concentration of rental properties, regardless of an
individual property's tenancy status.30
These results characterize participation in the program for the entire service area, where the cost to
replace a confirmed LSL was $1,000 for all suburban and most urban customers. The higher rates of
program registration and inspection (and, in the urban core, replacement of confirmed LSL) among
properties with higher assessed values suggest that households living in lower valued properties (who
likely also had lower incomes) faced greater barriers to participation. This analysis did not assess whether
programs that subsidize the cost of LSLR can reduce these barriers and whether these barriers were solely
financial or were also non-financial. We address these questions next in our evaluations of two programs
offered to certain urban residents during the study period.
6.2 Effect of subsidies on LSLR participation
Figure 2 presents the results of the quasi-experimental analysis evaluating the impact of the community-
based grant program on LSLR participation in the urban core. Recall that the community-based grant
program offered to cover the entirety of the $1,000 LSLR cost for homeowners and half of the cost for
landlords in a specific low-income neighborhood in Trenton with almost no paperwork or additional
eligibility requirements. The organization also conducted outreach in the target community using a variety
of methods, including social media, virtual community meetings, and in-person canvassing. After the
introduction of the grant program, LSLR participation was substantially higher in the target community
than in a synthetic control group of urban properties with similar characteristics. After the launch of the
grant program, registration rates were twice as high, and inspection and replacement rates were more
than 50 percent higher in the treatment group than in the synthetic control group. The increases in
participation among the treatment group relative to the synthetic control for all three outcomes are all
statistically significant (though the increase in inspection rates in the treated group is only significant at
the 10 percent level using a two-tailed test). Most notably, replacement rates among properties with
confirmed homeowner-side LSLs increased by 17 percentage points more in the target community than
in the synthetic control group. Note that completion of homeowner-side LSLRs also increased in the
synthetic control group in the period after the grant launch, consistent with the uptick in overall LSLR
program activity in the study area throughout 2021 and 2022.
30 We also note that in suburban properties with confirmed homeowner-side LSLs, percent rentals in the census
block group is highly correlated with percent Black population (rho = 0.48) and percent of the population below the
poverty level (rho = 0.49). When these two variables are excluded from the regression, the coefficient on percent
rental is much smaller and is no longer statistically significant. Thus, the positive and significant association with
percent rental is driven in part by multicollinearity with other neighborhood characteristics.
-------
Figure 2. Impact of community-based grant program on LSLR registrations, inspections, and completions
50%
40%
30%
20%
10%
0%
Lead Service Line Replacement Program Participation:
Properties in target community vs. synthetic control
**
16%
22%
8%
7%
Program registration (18,879
properties)
10%
Participation in contractor Replacement of confirmed
inspection to verify line material howeowner-side lead line (2,742
(18,879 properties) properties)
Synthetic control, before grant program
Target community, before grant program
Synthetic control, after grant program
Target community, after grant program
Asterisks denote statistical significance of the treatment effect (*** p<0.01, ** p<0.05, * p<0.1).
The synthetic control approach ensures that pre-treatment participation rates are equal across the
treatment and synthetic control groups. The estimated difference-in-difference coefficients representing
the impact of the community grant program are identical regardless of whether property characteristics
and block group fixed effects are included in the regressions using the synthetic control approach (Tables
A2b and A2c in the Appendix). The estimated impacts are also similar when using an unweighted
difference-in-difference model including property characteristics and block group fixed effects in lieu of a
synthetic control approach, indicating that the results are robust to alternative ways of constructing the
control group (Table A2d in the Appendix). However, pre-treatment inspection and replacement rates
were notably higher without the synthetic control approach (Figure A2 in the Appendix), suggesting that
the synthetic control provides a more appropriate counterfactual for the target community.31
Next, we present the results of the field experiment providing a sample of urban residents with
information about the housing department grant-program offered to low-income homeowners. Recall
that while the program would cover the entire $1,000 cost of LSLR, applicants were required to provide
extensive documentation to prove their eligibility. Prior to randomly assigning addresses to the treatment
and control groups, power calculations were estimated using a sample size of 1,500 treatment and 1,500
control with an assumed baseline take-up rate of 10%, resulting in a minimum detectable effect of 3.2%.
31 Results were also qualitatively similar using coarsened exact matching (CEM) as an alternative matching approach
(Appendix Table A2e). As with the unweighted data, pre-treatment participation outcomes were significantly
different across the treatment and control groups when using CEM, confirming that the synthetic control approach
provides a more appropriate control group.
-------
Figure 3 shows participation rates before and after the postcard mailing in the treatment and control
groups. Program registration, inspection, and replacement rates were similar across the control and
treatment groups before the intervention.32 After the postcards were sent, registration rates stayed
roughly constant, while inspection and replacement rates increased significantly among both groups,
consistent with the uptick in LSLR program activity throughout the service area in the latter half of the
study period. There were slightly larger increases in registration and inspection rates in the treatment
group than the control group. The treatment effects for registration and inspections are 1.7% and 2.8%,
respectively. Given post-intervention participation rates among the control group of 12% for registration
and 16% for inspection (which were higher than the 10% originally assumed), these effects are below the
minimum detectable effect size and are not statistically significant.
Figure 3. Impact of postcard experiment on LSLR registration, inspections, and completions
30%
10%
0%
Lead Service Line Replacement Program Participation:
Properties receiving postcard vs. control group
21% 21%
20% 16% ™
12% 12% 12%
7 10%
6%
Program registration (3,100 Participation in contractor Replacement of confirmed
properties) inspection to verify line material howeowner-side lead line (526
(3,100 properties) properties)
No postcard group, before mailing
Postcard group, before mailing
H No postcard group, after mailing
¦ Postcard group, after mailing
There were no statistically significant treatment effects in this analysis.
The increase in replacement rates among properties with confirmed homeowner-side LSLs was nearly
identical across the two groups. The effect size of 0.8% is near zero in a qualitative sense as well as not
being statistically distinguishable from zero. These results show that the informational treatment about a
grant opportunity was largely ineffective in boosting LSLR program participation.33 Housing department
staff also confirmed that they received no applications for LSLR under the grant program after the
32 Baseline registration and inspection were included as features in the rerandomization strategy implemented at
assignment of households to treatment and control groups.
33 Because the postcards only included English-language information, they could have been less effective in areas
with a higher concentration of non-English speakers. We tested this hypothesis by interacting the treatment variable
with the share of Hispanic residents in the block group (Table A3d). The impact of receiving a postcard interacted
with the share of Hispanic residents is not statistically significant in any of the three LSLR program participation
outcomes.
-------
postcards were sent (personal communication, Farrah Gee, City of Trenton Department of Housing and
Economic Development).
Even if these effects were shown to be statistically significant in a more highly powered study, the
magnitudes of the effects may not be substantial enough to make this type of intervention attractive to
water systems, particularly given the negligible effect on the ultimate outcome of interest—replacement
of confirmed homeowner-side LSLs. While the cost of the postcard intervention is low, it still entails
administrative and time costs to water systems to coordinate with the organization providing the grants
and to design the outreach.
The results in Figure 3 were derived using the coefficient estimates from the instrumental variables
difference-in-difference regression (Table A3b in the Appendix). Table A3c presents the coefficient
estimates for the intent-to-treat difference-in-difference model that does not account for the fact that
15% of observations in the assigned treatment group did not receive the postcards. The intent-to-treat
and instrumental variables approaches yield similar results.34
Results from the two program evaluations are not entirely consistent with the hypothesis that households
offered subsidies are more likely to participate in the LSLR program, since the postcards about the housing
department grant program had no effect on LSLR participation. The differing results across the two grant
programs is, however, consistent with the hypothesis that subsidies are more effective when combined
with outreach and trusted community messengers that can address non-financial barriers to program
participation.
7 Discussion and Conclusions
To our knowledge, this study is the first to examine individual property and neighborhood characteristics
associated with occurrence of LSL and participation in a voluntary LSLR program. LSLs were more prevalent
in the urban portion of the water system service area, which has a higher concentration of pre-1950
housing stock, Black and Hispanic residents, renters, and poverty than the suburban portions of the
service area. Properties in the urban area were somewhat more likely to register for the program and
participate in a service line inspection than those in the suburbs. Replacement of confirmed LSLs occurred
at similar rates across the two location types.
Notable results from the observational regression analyses of the full urban and suburban samples include
the importance of tenancy, property values, and outreach. Replacement rates among urban properties
with confirmed LSLs were 21 percentage points higher in owner-occupied than in renter-occupied
properties. In suburban areas, this association was less pronounced but still statistically significant, with
owner-occupied properties 7 percentage points more likely to have a replacement than renter-occupied
properties. Registration and inspection rates were also significantly higher in owner-occupied than renter-
occupied properties in both urban and suburban areas. These results support the hypothesis that split
incentives between landlords and renters pose a substantial barrier to LSLR.
Properties with higher assessed value were also significantly more likely to register and to have an
inspection in both urban and suburban municipalities, and significantly more likely to replace a confirmed
34 Table A3e presents the coefficients for the intent-to-treat difference-in-difference model adding property-level
control variables and block group fixed effects. The estimated treatment effects are identical to those in the intent-
to-treat model that does not include the control variables and fixed effects.
-------
LSL in the urban municipality. This result suggests that barriers to participation are greatest among lower
income households, though we cannot identify the relative importance of financial and non-financial
barriers contributing to this outcome since low-income households may face both types of barriers
disproportionately.
Another consistent finding across urban and suburban areas is the importance of water system outreach.
The number of letters sent to encourage registration before program implementation was significantly
associated with all three participation outcomes in both urban and suburban areas. In the urban
municipality, replacement conditional on having a confirmed LSL was 7 percentage points higher per letter
sent, while in suburban areas, the effect was even larger at 11 percentage points per letter. Since letters
were sent to the properties rather than owners, the higher rental rate in the city could be a reason for the
smaller magnitude of the coefficient on outreach letters. The water system intentionally targeted the
letters to properties suspected to have homeowner-side LSLs, so we cannot make a strong causal claim
about the impact of the letters per se because they are likely to be correlated with additional forms of
outreach that were not tracked in our data, such as contractor and water system staff door-to-door visits.
In addition, homeowners who suspected they had an LSL might have been more likely to register and sign
up for an inspection even in the absence of outreach letters, though our analysis controlled for housing
age, the primary determinant of LSL occurrence. Despite these limitations, the results suggest that
individualized property-level outreach is a useful strategy, and that multiple outreach attempts are more
effective than single attempts in eliciting a customer response.
At the neighborhood level, registration rates were generally higher in areas with more Black and Hispanic
residents, suggesting that lack of information, mistrust, and language barriers were not particular barriers
for minority residents. However, participation in replacement of confirmed LSL—the ultimate outcome of
interest—was lower in these neighborhoods, which could reflect additional financial constraints not
reflected in tenancy and property values. The inconsistent findings about other neighborhood
characteristics associated with LSLR program participation across urban and suburban areas, such as
poverty and presence of young children and elderly residents, make it difficult to generalize about
additional barriers to participation.
One limitation of our study is that we were not able to examine the impact of the COVID-19 pandemic on
customer participation. In-person outreach, inspection, and construction work paused from March
through July 2020, but customers after that time could still have been reluctant to allow contractors inside
of their homes due to concerns about disease transmission, dampening program participation.
Using quasi-experimental and experimental approaches, we evaluated two interventions designed to
address the financial barriers to participation by providing full-cost subsidies for certain urban residents.
The results yield insights about approaches that do and do not motivate participation among residents
facing financial barriers. We found that the community-based grant program was highly effective in
encouraging participation in the LSLR program. In contrast, the postcard informing residents about the
housing department grant program was not. The two programs differed substantially in terms of both
mode of outreach and transactional barriers, so we cannot parse the relative contribution of these factors
to the divergent outcomes. The community organization conducted extensive in-person and virtual
outreach and education about LSLR, while the field experiment involved a single postcard mailing about
the housing department grant program, which was not otherwise well publicized. In addition, the housing
department grant program required an application and review process including formal documentation
-------
of income to prove eligibility, while the community program had no income requirements. The results
suggest that programs to cover homeowner-side LSL costs can greatly boost customer participation when
the programs are well publicized and easy to access.
It is worth noting that participation rates in the neighborhood targeted by the community grant program,
while significantly higher than the control group, topped out at 54% of properties with confirmed LSLs
during study period. Despite the promising results from this program, participation was well under 100%
even when the financial barrier to participation was removed for homeowners and substantially reduced
for landlords. A representative from the community organization confirmed that the program could have
funded more LSLRs had they received more applications, indicating that the shortfall in full participation
was due to lack of customer demand rather than insufficient funding (personal communication, Caitlin
Fair, East Trenton Collaborative). Our results confirm that water systems designing programs to identify
and replace LSLs will need to address both financial and non-financial barriers to achieve the goal of full
and equitable replacement of LSLs.
-------
References
Abadie, A., S. Athey, G. Imbens, and J. Wooldridge. 2022. When Should You Adjust Standard Errors for
Clustering? Quarterly Journal of Economics, qjac038.
Abadie, A. 2021. Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects.
Journal of Economics Literature 59(2): 391-425.
Allcott, H., and Greenstone, M. 2017. Measuring the welfare effects of residential energy efficiency
programs. NBER Working paper #23386. National Bureau of Economic Research.
American Water Works Association, 2005. Strategies to obtain customer acceptance of complete lead
service line replacement. White paper.
Angrist, J. D., and Pischke, J.S. 2008. Mostly Harmless Econometrics: An Empiricist's Companion (1st ed.).
Princeton University Press.
American Water Works Association (AWWA). 2022. Considerations when costing lead service line
identification and replacement. Report prepared by CDM Smith, Nov. 2022.
Baehler et al. 2022. Full Lead Service Line Replacement: A Case Study of Equity in Environmental
Remediation. Sustainability 14(1): 352.
Brown et al. 2011. Association Between Children's Blood Lead Levels, Lead Service Lines, and Water
Disinfection, Washington, DC, 1998-2006. Environmental Research 111(1): 67-74.
Christensen et al. 2021. Economic Effects of Environmental Crises: Evidence from Flint, Michigan. SSRN.
https://ssrn.com/abstract=3420526 [accessed Nov. 10, 2022]
City of Newark. 2019. Mandatory Replacement of Lead Service Lines, Sept. 10. 2019:
https://staticl.sauarespace.eom/static/5ad5e03312bl3f2c50381204/t/5daf07f6e298021e8365cf82/157
1751926771/LSLR+Ordinance+20190910.pdf [accessed Nov. 14, 2022]
Cornwell et al., 2016. National Survey of Lead Service Line Occurrence. Journal AWWA 108(4): E182-E191.
Dave and Yang. 2020. Lead in Drinking Water and Birth Outcomes: ATale of Two Water Treatment Plants.
Journal of Health Economics 84: 102644.
Federal Reserve Bank of Chicago. 2022. Flint, Ml: Water crisis spurs new focus and funding. May 2022.
https://www.chicagofed.org/research/lead/flint-mi [accessed Jan. 17, 2023]
Fowlie, M., Greenstone, M., and Wolfram, C. 2015. Are the non-monetary costs of energy efficiency
investments large? Understanding low take-up of a free energy efficiency program. American Economic
Review, Papers and Proceedings 105(5): 201-204.
-------
Fowlie, M., Greenstone, M., and Wolfram, C. 2018. Do energy efficiency investments deliver? Evidence
from the Weatherization Assistance Program. Quarterly Journal of Economics 133(3): 1597-1644.
GAO, 2020. Drinking Water: EPA Could Use Available Data to Better Identify Neighborhoods at Risk of Lead
Exposure. GAO Report to Congressional Requesters, Dec. 2020. GAO-21-78.
Gillingham, G. and Tsvetanov, T. 2018. Nudging energy efficiency audits: Evidence from a field
experiment. Journal of Environmental Economics and Management 90: 303-316.
Hainmueller, J. 2012. Entropy Balancing for Causal Effects: A Multivariate Reweighting Methods to
Produce Balanced Samples in Observational Studies. Political Analysis 20: 25-46.
Hexemer, A., Pintar, K., Bird, T., Zentner, S., Garcia, H., and Pollari, F. 2008. An investigation of
bacteriological and chemical water quality and the barriers to private well water sampling in a
Southwestern Ontario Community. Journal of Water and Health 6(4): 521-525.
Holladay, J., LaRiviere, J., Novgorodsky, D., and Price, M. 2016. Asymmetric effects of non-pecuniary
signals on search and purchase behavior for energy-efficient durable goods. NBER Working Paper
#22939, National Bureau of Economic Research.
Hull, C., and Anderson, N. 2022. What will it take to remove all lead service lines? Common barriers to
getting the lead out of drinking water. Federal Reserve Board of Chicago. May.
lacus, S., King, G., Porro, G., 2012. Causal inference without balance checking: coarsened exact
matching. Political Anal. 20 (1), le24.
Karlsson, N., G. Loewenstein, and D. Seppi. 2009. The ostrich effect: selective attention to information.
Journal of Risk and Uncertainty 38: 95-115.
Kreutzwiser, R., de Loe, R., Imgrund, K., Conboy, M. J., Simpson, H., and Plummer, R. 2011.
Understanding stewardship behaviour: Factors facilitating and constraining private water well
stewardship. Journal of Environmental Management 92(4): 1104-1114.
Kuffner, Alex. Providence Water gets $3.3M more in federal grant money to replace lead service lines.
The Providence Journal, Aug. 16, 2022.
https://www.providenceioumal.com/storv/news/2022/08/16/providence-water-federal-grant-replace-
lead-pipes-drinking-water/10329899002./ [accessed Dec. 8, 2022]
MacDonald, K., and Tippett, M. 2020. Reducing public exposure to common, harmful well water
contaminants through targeted outreach. Journal of Water and Health 18(4): 522-532.
Morgan, K.L and D.B. Rubin. 2012. Rerandomization to improve covariate balance in experiments. Ann.
Statist. 40(2): 1263-1282.
Mullainathan, S., and Shafir, E. 2013. Scarcity: Why Having Too Little Means So Much. Times Books.
-------
Pieper et al. 2017. Flint water crisis caused by interrupted corrosion control: Investigating "ground zero"
home. Environ Sci Tech 51(4): 2007-2014.
Renaud, J., Gagnon, F., Michaud, C. and Boivin, S. 2011. Evaluation of the effectiveness of arsenic
screening promotion in private wells: a quasi-experimental study. Health Promotion International 26(4):
465-475.
Robbins, M., J. Saunders, and B. Kilmer. 2017. A Framework for Synthetic Control Methods with High-
Dimensional, Micro-Level Data: Evaluating a Neighborhood-Specific Crime Intervention. Journal of the
American Statistical Association 112(517): 109-126.
Severtson, D., Baumann, L, and Brown, R. 2006. Applying a health behavior theory to explore the
influence of information and experience on arsenic risk representations, policy beliefs, and protective
behavior. Risk Analysis 26(2): 353-368.
Sussman, J. and R. Hayward. 2010. An IV for the RCT: using instrumental variables to adjust for treatment
contamination in randomised controlled trials. BMJ. May 4, 2010;340:c2073.
Theising, 2019. Lead Pipes, Prescriptive Policy and Property Values. Environmental and Resource
Economics 74: 1355-1382.
Triantafyllidou and Edwards, 2012. Lead (Pb) in Tap Water and in Blood: Implications for Lead Exposure in
the United States. Critical Reviews in Environmental Science and Technology 42(13): 1297-1352.
U.S. EPA, 2013. Integrated Science Assessment for Lead. EPA Office of Research and Development,
National Center for Environmental Assessment, Research Triangle Park, NC. EPA/600/R-10/075F.
U.S. EPA. 2019. Strategies to Achieve Full Lead Service Line Replacement. October. EPA 810-R-19-003.
https://www.epa.gov/sites/default/files/2Q19-
10/documents/strategies to achieve full lead service line replacement 10 09 19.pdf [accessed Dec.
16, 2022]
U.S. EPA, 2021a. Economic Analysis for the Final Lead and Copper Rule Revisions. Office of Water, EPA
816-R-20-008.
U.S. EPA, 2021b. LCRR Community Roundtable Summary - Newark NJ. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-QW-2021-0255-0273 [accessed Dec. 8, 2022]
U.S. EPA, 2021c. LCRR Community Roundtable Summary - Newburgh NY. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0275 [accessed Dec. 8, 2022]
U.S. EPA, 2021d. LCRR Community Roundtable Summary - Milwaukee Wl. Posted Sept. 20, 2021.
https://www.regulations.gov/document/EPA-HQ-QW-2021-0255-0273 [accessed Dec. 8, 2022]
-------
U.S. EPA, 2021e. LCRR Community Roundtable Summary - Maiden MA. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0274 [accessed Dec. 8, 2022]
U.S. EPA, 2021f. LCRR Community Roundtable Summary - Washington DC. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0278 [accessed Dec. 8, 2022]
U.S. EPA, 2021g. LCRR Community Roundtable Summary - Pittsburgh PA. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0087 [accessed Dec. 8, 2022]
U.S. EPA, 2021h. LCRR Community Roundtable Summary - Chicago. Posted Sept. 20, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0281 [accessed Dec. 8, 2022]
U.S. EPA, 202li. LCRR Community Roundtable Summary - Flint and Detroit Ml. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0284 [accessed Dec. 8, 2022]
U.S. EPA, 2021j. LCRR Community Roundtable Summary - Benton Harbor and Highland Park Ml. Posted
Aug. 23, 2021. https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0277 [accessed Dec. 8,
2022]
U.S. EPA, 2021k. LCRR Community Roundtable Summary - Memphis TN. Posted Aug. 23, 2021.
https://www.regulations.gov/document/EPA-HQ-OW-2021-0255-0279 [accessed Dec. 8, 2022]
U.S. EPA, 2022a. Biomonitoring - Lead. American's Children and the Environment.
https://www.epa.gov/americaschildrenenvironment/biomonitoring-lead [accessed Nov. 10, 2022]
U.S. EPA, 2022b. Guidance for Developing and Maintaining a Service Line Inventory. Office of Water
(4606M), EPA 816-B-22-001, August 2022. https://www.epa.gov/svstem/files/documents/2022-
08/lnventorv%20Guidance August%202022 50 [accessed Nov. 1, 2022]
Wallander, S., Ferraro, P., and Higgins, N. 2017. Addressing Participant Inattention in Federal Programs: A
Field experiment with the Conservation Reserve Program. American Journal of Agricultural Economics
99(4): 914-931.
White House, 2021. Fact Sheet: The Biden-Harris Lead Pipe and Paint Action Plan. December 16, 2021.
https://www.whitehouse.gov/briefing-room/statements-releases/2021/12/16/fact-sheet-the-biden-
harris-lead-pipe-and-paint-action-plan/ [accessed November 10, 2022]
-------
Appendix: Additional Tables and Figures
Al. Urban and suburban full-sample analyses
Table Ala. Summary statistics disaggregated by location and service line material: Urban municipality
Observations
Confirmed lead
3,098
Confirmed non-lead
6,429
Material unknown
11,002
Mean
Std. Dev.
Mean
Std. Dev.
Mean
Std. Dev.
LSLR participation outcomes
Registered for LSLR program
41%
0.49
36%***
0.48
2%* * *
0.12
Interior contractor inspection
70%
0.46
80%***
0.40
0%***
0.03
Homeowner-side LSLR completed
44%
0.50
0%***
0.00
0%***
0.00
Property characteristics
Lead on utility side of line
60%
0.49
65%***
0.48
32%***
0.47
Number of LSLR letters sent
1.32
0.72
1.30
0.72
Q gy***
0.69
Account inactive
9%
0.29
10%
0.30
10%
0.30
Street paving moratorium
12%
0.33
11%
0.32
13%
0.34
Suburban location
0%
0.00
0%
0.00
0%
0.00
Assessor data missing
3%
0.17
6%***
0.23
10%***
0.30
Property built before 1951^
99%
0.12
92%* * *
0.27
90%***
0.30
Property built 1951-1960t
1%
0.09
5%***
0.21
2%* * *
0.14
Owner-occupied propertyt
51%
0.50
49%**
0.50
43%***
0.50
Multi-family propertyt
8%
0.27
8%
0.28
12%***
0.33
Assessed value ($100,000) t
0.64
0.35
0.64
0.37
0.61***
0.31
Block group characteristics
Share Black
47%
0.28
49%***
0.29
49%***
0.29
Share Hispanic
40%
0.27
37%***
0.27
38%***
0.26
Share under 5
7%
0.05
7%
0.05
7%* * *
0.05
Share over 64
9%
0.06
9%* * *
0.06
9%
0.06
Share rental
57%
0.20
58%***
0.20
60%***
0.19
Share college graduate
12%
0.09
13%***
0.10
12%***
0.08
Share below poverty
25%
0.13
27%***
0.13
28%***
0.14
¦f Data presented only for properties linked to assessor data. *** p<0.01, ** p<0.05, * p<0.1 denotes a statistically
significant difference compared to properties with confirmed homeowner-side lead lines.
-------
Table Alb. Summary statistics disaggregated by location and service line material: Suburban
municipalities
Confirmed lead Confirmed non-lead Material unknown
Observations 2,671 9,332 23,385
Mean
Std. Dev.
Mean
Std. Dev.
Mean
Std. Dev.
LSLR participation outcomes
Registered for LSLR program
39%
0.49
20%***
0.40
2%* * *
0.14
Interior contractor inspection
61%
0.49
57%***
0.49
0%***
0.02
Homeowner-side LSLR completed
42%
0.49
0%***
0.00
0%***
0.00
Property characteristics
Lead on utility side of line
57%
0.50
53%***
0.50
26%***
0.44
Number of LSLR letters sent
0.96
0.85
0.95
0.85
0.43***
0.69
Account inactive
6%
0.24
5%*
0.22
5%**
0.22
Street paving moratorium
8%
0.27
6%***
0.23
12%***
0.33
Suburban location
100%
0.00
100%
0.00
100%
0.00
Assessor data missing
6%
0.24
6%
0.24
11%***
0.31
Property built before 1951f
74%
0.44
37%*
0.48
38%*
0.49
Property built 1951-1960f
22%
0.41
35%*
0.48
23%
0.42
Owner-occupied property!
84%
0.37
86%*
0.35
85%
0.36
Multi-family property!
1%
0.10
1%
0.08
1%
0.11
Assessed value ($100,000) f
1.68
0.54
1.91*
0.53
2.04*
0.75
Block group characteristics
Share Black
24%
0.19
18%***
0.17
20%***
0.20
Share Hispanic
20%
0.16
13%***
0.12
14%***
0.14
Share under 5
7%
0.05
6%***
0.04
6%***
0.04
Share over 64
13%
0.06
16%***
0.06
15%***
0.08
Share rental
31%
0.19
29%* * *
0.17
29%* * *
0.18
Share college graduate
26%
0.14
33%***
0.14
32%***
0.16
Share below poverty
9%
0.08
8%***
0.07
9%*
0.08
¦f Data presented only for properties linked to assessor data. *** p<0.01, ** p<0.05, * p<0.1 denotes a statistically
significant difference compared to properties with confirmed homeowner-side lead lines.
-------
Table Ale: Full-sample regression analysis of determinants of LSLR program participation
(1) (2) (3)
Registration Inspection Replacement of
confirmed LSL
* * *
* * *
**
Number of letters sent 0.027*** 0.052*** 0.068**
(0.005) (0.006) (0.014)
Account inactive -0.024*** 0.001 -0.027
(0.008) (0.010) (0.028)
Lead on utility side 0.179*** 0.359*** 0.350
(0.007) (0.008) (0.021)
Street paving moratorium -0.001 -0.063*** -0.009
(0.008) (0.009) (0.023)
Owner-occupied property 0.072*** 0.037*** 0.212
(0.006) (0.007) (0.017)
Multi-unit property -0.065*** -0.137*** 0.005
(0.010) (0.011) (0.033)
Ln assessed value 0.065*** 0.047*** 0.046
(0.008) (0.009) (0.022)
Property built before 1951 0.100*** 0.139
(0.009) (0.011)
Property built 1951-1960 0.092*** 0.209
(0.020) (0.022)
Assessor data unlinked -0.088*** -0.136*** -0.073
(0.011) (0.013) (0.053)
Share Black 0.159*** 0.069** -0.101
(0.024) (0.027) (0.070)
Share Hispanic 0.068** 0.046 -0.058
(0.028) (0.032) (0.084)
Share under 5 -0.088 0.075 -0.120
(0.064) (0.072) (0.195)
Share over 64 0.011 0.288*** 0.072
(0.049) (0.056) (0.165)
Share rental housing -0.105*** -0.089*** -0.109**
(0.017) (0.020) (0.050)
Share college graduate 0.186*** 0.241*** -0.059
(0.044) (0.049) (0.128)
Share below poverty 0.164*** 0.079*** 0.092
(0.025) (0.029) (0.074)
Suburb 0.068** 0.037 -0.101
(0.030) (0.035) (0.099)
Number of letters sent 0.071*** 0.096*** 0.046**
~suburb (0.006) (0.007) (0.022)
Account inactive 0.019* -0.019 0.018
~suburb (0.011) (0.013) (0.043)
Lead on utility side -0.142*** -0.189*** 0.003
~suburb (0.009) (0.010) (0.036)
-------
Street paving moratorium -0.014 -0.063*** -0.119***
~suburb (0.009) (0.010) (0.038)
Owner-occupied property -0.049*** 0.007 -0.136***
~suburb (0.007) (0.009) (0.028)
Multi-unit property 0.067*** 0.041* -0.106
~suburb (0.021) (0.023) (0.092)
Ln assessed value -0.051*** -0.015 -0.096***
~suburb (0.009) (0.011) (0.037)
Property built before 1951 -0.057*** -0.072***
~suburb (0.010) (0.012)
Property built 1951-1960 -0.076*** -0.134***
~suburb (0.021) (0.023)
Assessor data unlinked 0.079*** 0.092*** 0.041
~suburb (0.012) (0.015) (0.065)
Share Black -0.165*** -0.182*** -0.189**
~suburb (0.025) (0.030) (0.088)
Share Hispanic -0.003 -0.014 -0.057
~suburb (0.032) (0.037) (0.109)
Share under 5 0.096 -0.407*** 0.497*
~suburb (0.076) (0.089) (0.283)
Share over 64 -0.027 -0.349*** 0.169
~suburb (0.054) (0.062) (0.214)
Share rental housing 0.104*** 0.143*** 0.274***
~suburb (0.020) (0.023) (0.072)
Share college graduate -0.132*** -0.218*** 0.027
~suburb (0.046) (0.051) (0.150)
Share below poverty -0.199*** -0.148*** -0.243
~suburb (0.034) (0.040) (0.152)
Constant -0.111*** -0.043 0.182**
(0.028) (0.032) (0.083)
Observations 55,917 55,917 5,769
R-squared 0.124 0.267 0.272
Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1
-------
Figure Al. Regression analysis of select determinants of LSLR program participation
¦ Program Registration A Service Line Inspection ¦
Urban
Property Characteristics
3
Replacement of
Confirmed LSL
! 0.4
'u
E
ro
Neighborhood Characteristics
0.2
o
&
3 0.0
o
I—
Q_
QJ
i
1 :
T ^
U
,QJ
-0.2
S3 -0.4
C
o
u
SJ=
-0.2
¦0.4
Neighborhood Characteristics
I; I1
Property Characteristics
.1
4
.iiS9, ^ ^ ;/ J* J*
X- so
©5^ dP •&' <&' <9
- ^ ,xv° 0° ,*° AOX X
/" J? <$*> vd
^ ^
^ cP xf
-------
A2. Quasi-experimental evaluation of community grant program
Table A2a. Summary statistics for treated, untreated, and synthetic control groups
Target community Untreated urban Synthetic control}
properties
Observations 1,010 17,869 17,869
Mean
Std. Dev.
Mean
Std. Dev.
Mean
Std. Dev.
LSLR participation outcomes
Registered for LSLR program
24%
0.43
17%***
0.38
14%***
0.35
Interior contractor inspection
34%
0.47
36%
0.48
26%***
0.44
Homeowner-side LSLR completed
6%
0.24
7%
0.25
4%***
0.20
Property characteristics
Lead on utility side of line
44%
0.50
46%
0.50
44%
0.50
Number of LSLR letters sent
1.07
0.73
1.12**
0.72
1.07
0.74
Account inactive
13%
0.34
10%***
0.30
13%
0.34
Street paving moratorium
25%
0.43
12%***
0.32
25%
0.43
Suburban location
0%
0.00
0%
0.00
0%
0.00
Assessor data missing
11%
0.32
7%* * *
0.26
11%
0.32
Property built before 1951f
91%
0.28
92%
0.28
91%
0.28
Property built 1951-1960f
0%
0.00
3%***
0.16
0%
0.02
Owner-occupied property!
41%
0.49
45%**
0.50
41%
0.49
Multi-family property!
4%
0.19
11%***
0.31
4%
0.19
Assessed value ($100,000)!
0.32
0.13
0.64***
0.33
0.32
0.11
Block group characteristics
Share Black
50%
0.13
48%
0.29
50%
0.31
Share Hispanic
43%
0.14
38%***
0.27
43%
0.27
Share under 5
6%
0.02
7%* * *
0.05
8%***
0.06
Share over 64
5%
0.04
9%* * *
0.06
5%
0.03
Share rental
64%
0.10
59%***
0.20
64%
0.16
Share college graduate
6%
0.03
12%***
0.09
6%
0.05
Share below poverty
34%
0.15
27%***
0.13
34%
0.16
¦fData presented only for properties linked to assessor data. *** p<0.01, ** p<0.05, * p<0.1 denotes a statistically
significant difference compared to properties with confirmed homeowner-side lead lines. {Synthetic control
summary statistics are calculated using entropy balancing weights.
-------
Figure A2. Impact of community-based grant program on LSLR registrations, inspections, and
completions using an unweighted average of all untreated properties instead of a synthetic control
50%
40%
30%
20%
10%
0%
Lead Service Line Replacement Program Participation:
Properties in target community vs. untreated properties
**
16%
10%
8%
8%
¦ I
Program registration (18,955
properties)
22%
* * *
22%
12%
¦
14% I
ll
14%
9%
Participation in contractor Replacement of confirmed
inspection to verify line material howeowner-side lead line (2,742
(18,955 properties) properties)
All untreated, before grant program
Target community, before grant program
All untreated, after grant program
Target community, after grant program
Table A2b. Synthetic control difference-in-difference regression estimates of impact of community-based
grant program on LSLR program participation
(I) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Target community
After
Target community*
After
Constant
Property characteristics
BG fixed effects
0.000
(0.012)
-0.019
(0.014)
0.095**
(0.041)
0.084***
(0.007)
No
No
-0.000
(0.018)
0.008
(0.025)
0.086*
(0.048)
0.124***
(0.014)
No
No
-0.000
(0.053)
0.169***
(0.047)
0.175**
(0.078)
0.095***
(0.031)
No
No
Observations
R-squared
37,758
0.015
37,758
0.012
5,484
0.117
Regressions are weighted using entropy balancing weights. Standard errors clustered by Census block group are in
parentheses. ***p < 0.01, **p < 0.05, *p < 0.1
-------
Table A2c. Synthetic control difference-in-difference regression estimates of impact of community-based
grant program on LSLR program participation including block group fixed effects and property control
variables
(I) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Target community
After
-0.019
0.008
0.169***
(0.014)
(0.025)
(0.048)
Target community*
0.095**
0.086*
0.175**
After
(0.041)
(0.048)
(0.078)
Constant
0.001
-0.038*
-0.076
(0.016)
(0.020)
(0.074)
Property characteristics
Yes
Yes
Yes
BG fixed effects
Yes
Yes
Yes
Observations
37,758
37,758
5,484
R-squared
0.084
0.103
0.269
Regressions are weighted using entropy balancing weights. Standard errors clustered by Census block group are in
parentheses. ***p < 0.01, **p < 0.05, *p < 0.1
Table A2d. Unweighted difference-in-difference regression estimates of impact of community-based
grant program on LSLR program participation including block group fixed effects and property control
variables
(I) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Target community
After
-0.025**
-0.077***
0.170***
(0.011)
(0.025)
(0.048)
Target community*
0.101**
0.171***
0.175**
After
(0.040)
(0.048)
(0.078)
Constant
-0.001
0.043**
-0.053*
(0.010)
(0.017)
(0.027)
Property characteristics
Yes
Yes
Yes
BG fixed effects
Yes
Yes
Yes
Observations
R-squared
37,758
0.056
37,758
0.102
5,484
0.149
-------
Standard errors clustered by Census block group are in parentheses. ***p < 0.01, **p < 0.05, *p < 0.1
-------
Table A2e. CEM (coarsened exact matching) weighted difference-in-difference regression estimates of
impact of community-based grant program on LSLR program participation including block group fixed
effects
(I) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Target community
After
Target community*
After
Constant
Property characteristics
BG fixed effects
-0.007
(0.011)
0.084**
(0.040)
0.084***
(0.005)
No
Yes
-0.049**
(0.023)
0.144***
(0.046)
0.183***
(0.011)
No
Yes
0.203***
(0.055)
0.148*
(0.086)
0129***
(0.026)
No
Yes
Observations 32,612 32,612 4,060
R-squared 0.014 0.022 0.102
Regressions are weighted using CEM weights (lacus et al. 2012). CEM weights were derived by matching the
treatment and comparison groups on all nine property-level covariates: lead on the utility-side of the line, number
of LSLR letters sent, account inactive, street paving moratorium, assessor data missing, property built before 1951,
property built 1951-1960, owner-occupied property, and assessed value. An exact match was used for all variables
except assessed value, which was coarsened into five categories denoting $l-$25,000, $25,000-$50,000, $50,000-
$75,000, $75,000-$100,000, and $100,000-$ 125,000. Census block group characteristics were not used as match
variables; instead, the regressions include block group fixed effects. Standard errors clustered by Census block group
are in parentheses. ***p < 0.01, **p < 0.05, *p < 0.1
-------
A3. Experimental evaluation of postcards with information about city
grant program
Figure A3. Postcard sent to experimental sample
Take advantage of this special opportunity and
REPLACE YOUR LEAD PIPES!
14/ty should I replace My leod service line?
Exposure to lead in drinking water may cause serious health problems including damage to
the kidneys and brain. Infants, children, and those who are pregnant are especially at risk.
CITY OF TRENTON INCOME ELIGIBLE HOMEOWNERS MAY QUALIFY FOR GRANT ASSISTANCE
Qualifying homeowners can have their entire lead service line (including the portion they own)
replaced at NO COST through the Trenton Urgent Rehabilitation Program (TURP)l
HOW TO APPLY:
Check if your home has lead pipes by visiting Trenton Water Works (TWW) at
www.twwleadprogram.com and clicking "check your line," or by contacting TWW's Lead
Service Line Replacement Program: (609) 989-3600 or twwleadprogram@trentonnj.org
Submit an application to TURP for financial assistance. Applications and eligibility criteria are
available at www.trentonnj.org/turp or at City Hall, 319 East State Street, Trenton NJ. Select
"Lead Service Line Replacement" on the application. The City will provide an award letter if you
are eligible for the grant.
A contractor will then contact you to schedule an appointment for free lead pipe replacement.
JAS Group is the contractor hired by TWW to conduct replacements in this special program.
Property owners are responsible for maintenance of their water service line from the curb to
inside their house. Trenton Water Works is responsible for the water service from the main in
the street to the curb. Both sides can contain lead.
-------
Table A3a. Summary statistics for postcard treatment and control groups
Observations
Assigned treatment group
3,100
Assigned control group
3,100
Mean
Std. Dev.
Mean
Std. Dev.
LSLR participation outcomes
Registered for LSLR program
Interior contractor inspection
Homeowner-side LSLR completed
Property characteristics
Postcard received
Lead on homeowner side of the line
Lead on utility side of line
Number of LSLR letters sent
Account inactive
Street paving moratorium
Assessor data missing
Property built before 1951f
Property built 1951-1960f
Owner-occupied property!
Multi-family property!
Assessed value ($100,000) f
Block group characteristics
Share Black
Share Hispanic
Share under 5
Share over 64
Share rental
Share college graduate
Share below poverty
22%
27%
4%
85%
47%
51%
1.09
4%
13%
8%
94%
4%
67%
7%
0.85
52%
36%
7%
9%
61%
12%
27%
0.41
0.44
0.2
0.35
0.5
0.5
0.73
0.2
0.33
0.28
0.25
0.19
0.47
0.25
4.02
0.3
0.28
0.05
0.06
0.19
0.09
0.13
22%
27%
5%
0%***
51%
53%
1.13
5%
14%
9%
94%
4%
65%
8%
0.80
52%
36%
7%
9%
61%
13%
27%
0.41
0.44
0.21
0.06
0.50
0.50
0.73
0.23
0.34
0.29
0.23
0.19
0.48
0.26
1.52
0.30
0.28
0.05
0.06
0.19
0.09
0.13
"fData presented only for properties linked to assessor data. *** p<0.01, ** p<0.05, * p<0.1 denotes a statistically
significant difference compared to properties with confirmed homeowner-side lead lines.
-------
Table A3b. Instrumental variables difference-in-difference regression estimates of impact of postcard on
LSLR program participation
(1) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Received postcard
-0.013
-0.012
-0.006
(0.013)
(0.013)
(0.024)
After
0.001
0.042***
0i44**:
(0.012)
(0.012)
(0.029)
Received postcard X
0.017
0.028
0.008
After
(0.019)
(0.020)
(0.047)
Constant
0.116***
0.114***
0.065**:
(0.008)
(0.008)
(0.015)
Observations
6,200
6,200
1,056
R-squared
0.000
0.006
0.046
Robust standard errors in
parentheses. ***p<0.01,
**p < 0.05, *p<0.1
Table A3c. Intent-to-treat difference-in-difference regression estimates of impact of postcard on LSLR
program participation
(I) (2) (3)
Registration Inspection Replacement
(conditional on
confirmed lead)
Received postcard
-0.011
-0.010
-0.005
(0.011)
(0.011)
(0.021)
After
0.001
0.042***
0 i44**:
(0.011)
(0.012)
(0.029)
Received postcard*
0.015
0.024
0.007
After
(0.016)
(0.017)
(0.041)
Constant
0.115***
0.114***
0.065**:
(0.008)
(0.008)
(0.015)
Observations
6,200
6,200
1,056
R-squared
0.000
0.007
0.046
Robust standard errors
in parentheses. ***p < 0.01,
**p < 0.05, *p<0.1
-------
Table A3d. Intent-to-treat difference-in-difference regression estimates of impact of postcard interacted
with percent Hispanic on LSLR program participation
(1)
(2)
(3)
Registration
Inspection
Replacement
(conditional on
confirmed lead)
Received postcard
-0.030
-0.022
-0.036
(0.020)
(0.019)
(0.032)
After
-0.012
0.063***
0.110**
(0.021)
(0.021)
(0.045)
Received postcard*
0.022
0.037
0.079
After
(0.029)
(0.030)
(0.065)
Share Hispanic
-0.158***
-0.059**
-0.069
(0.027)
(0.030)
(0.043)
Share Hispanic*After
0.037
-0.060
0.105
(0.040)
(0.044)
(0.099)
Share Hispanic*
0.054
0.032
0.090
Received postcard
(0.038)
(0.040)
(0.064)
Share Hispanic*After*
-0.019
-0.036
-0.209
Received postcard
(0.056)
(0.062)
(0.140)
Constant
0.171***
0.135***
0.088***
(0.015)
(0.014)
(0.024)
Observations
6,200
6,200
1,056
R-squared
0.011
0.012
0.049
Robust standard errors in parentheses. ***p < 0.01,
**p < 0.05, *p<0.1
Table A3d. Intent-to-treat difference-in-difference regression estimates of impact of postcard on LSLR
program participation including block group fixed effects and property control variables
(1)
(2)
(3)
Registration
Inspection
Replacement
(conditional on
confirmed lead)
Received postcard
-0.009
-0.009
-0.008
(0.011)
(0.011)
(0.024)
After
0.001
0.042***
0.144***
(0.011)
(0.012)
(0.027)
Received postcard*
0.015
0.024
0.007
After
(0.016)
(0.017)
(0.039)
Constant
0.055
0.040
-0.350***
(0.043)
(0.044)
(0.125)
Observations
6,196
6,196
1,056
R-squared
0.080
0.098
0.194
Robust standard errors in parentheses. ***p < 0.01,
**p < 0.05, *p<0.1
------- |