Xs, EPA
United States
Environmental
Protection Agency
EPA 600/R-23/058 | March 2023 | www.epa.gov/research
Kl
Using Human-Centered Design
to Create a Hazardous Waste
Management App for use in
Rural Alaska
i
KA
Office of Research and Development
Center for Environmental Measurement and Modeling
Atlantic Coastal Environmental Sciences Division
-------
EPA/600/R-23/058
March 2023
Using Human-Centered Design to Create
a Hazardous Waste Management App
for use in Rural Alaska
By
Jesse S. Sayles1, Ryan P. Fury2, Marilyn R. Buchholtz ten Brink3,
Gabriela Carvalho4, Blair Crossman2, Lynn Zender6, Reilly Kosinski5,
Simone Sebalo5, Paul Lemieux6, Timothy Boe6,
Sherry Davis7, Kay la Krauss4
1. Oak Ridge Institute for Science and Education Fellow appointed with the U.S. Environmental Protection Agency,
Office of Research and Development, Center for Environmental Management and Modelling,
Atlantic Coastal Environmental Sciences Division. Narragansett, Rl 02882
2. Oak Ridge Associated Universities contracted to the U.S. Environmental Protection Agency,
Office of Research and Development, Center for Environmental Management and Modelling,
Atlantic Coastal Environmental Sciences Division. Narragansett, Rl 02882
3. U.S. Environmental Protection Agency, Office of Research and Development, Center for Environmental Management
and Modelling, Atlantic Coastal Environmental Sciences Division. Narragansett, Rl 02882
4. U.S. Environmental Protection Agency, Region 10, Land, Chemicals, and Redevelopment Division,
Pollution Prevention and Communities Branch, Tribal Solid and Hazardous Waste Program. Seattle, WA 98101
5. Zender Environmental Health and Research Group, Anchorage, AK 99501
6. U.S. Environmental Protection Agency, Office of Research and Development, Center for Environmental Solutions and
Emergency Response, Homeland Security and Materials Management Division, Research Triangle Park, 27711
7. U.S. Environmental Protection Agency, EPA Region 10, Regional Administrator's Division,
Tribal Trust and Assistance Branch, Alaska Operations Office, Anchorage, AK 99513
Office of Research and Development
Center for Environmental Measurement and Modeling
Atlantic Coastal Environmental Sciences Division
Narragansett, Rl 02882
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Notice and Disclaimer
This research project was funded through EPA's Regional-State-Tribal Innovation Projects (RSTIP) which is
administered by the EPA Office of Research and Development's (ORD) Regional Science Program.
Additionally, Sayles was supported by an appointment to the U.S. Environmental Protection Agency (EPA)
Research Participation Program administered by the Oak Ridge Institute for Science and Education (ORISE)
through an interagency agreement between the U.S. Department of Energy (DOE) and the EPA. ORISE is
managed by ORAU (Oak Ridge Associated Universities) under DOE contract number DE-SC0014664.
Crossman and Furey were supported under EPA contract number 68HERH20D0003 to ORAU. The research
described in this report has been conducted by the U.S. Environmental Protection Agency (EPA) with
collaboration of Zender Environmental Health and Research Group, under Memorandum of Understanding
#EPA MOU 173-23.
This document has been reviewed by the U.S. Environmental Protection Agency, Office of Research and
Development, and approved for publication. Approval does not signify that the contents reflect the views of
the Agency, nor does mention of trade names, manufacturers or products imply an endorsement by the
United States Government or the U.S. Environmental Protection Agency. EPA and its employees do not
endorse any commercial products, services, or enterprises. Links to websites outside the EPA website are
provided for the convenience of the user. Inclusion of information about a website, an organization, a
product, or a service does not represent endorsement or approval by EPA, nor does it represent EPA opinion,
policy or guidance unless specifically indicated. EPA does not exercise any editorial control over the
information that may be found at any non-EPA website. This is a work of the U.S. Government and is not
subject to copyright protection in the United States.
This is a contribution to the EPA ORD Center for Environmental Measurement and Modeling.
The citation for this report is: Sayles, J.S., R.P. Fury, M.R. Buchholtzten Brink, G. Carvalho, B. Crossman, L. Zender,
R. Kosinski, S. Sebalo, P. Lemieux, T. Boe, S. Davis, and K. Krauss. 2023. Using Human-Centered Design to Create
a Hazardous Waste Management App for use in Rural Alaska. U.S. Environmental Protection Agency, Atlantic
Coastal Environmental Sciences Division, Narragansett, Rl, EPA/600/R-23/058.
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Acknowledgments
Many people and groups contributed important information to this Regional-State-Tribal Innovation Project
(RTSIP) in one or more ways. This includes participating in conversations, workshops, and presentations;
developing the pilot app; and evaluating and providing feedback on interim research products. We are
especially appreciative of the many individuals with busy schedules working on solid waste removal locally,
regionally, and state-wide in Alaska that set aside time to participate. Their help grounded the project in the
lived experiences of those working to safely remove, manage, and recycle solid waste in Alaska. Several
reviewers also provided voluble comments that helped improve this manuscript. Thank you to everyone who
contributed to, or reviewed, this project:
Aaron Lestenkof (Aleut Community of St. Paul Island), Adam Jenkins (Esri, Inc.), Augusta Edmund (Alakanuk
Traditional Council), Allen Brookes (EPA ORD), Anahma Shannon (Kawerak, Inc), Andrew Fleagle (Native
Village of Port Heiden), Andy Wall (Kodiak Area Native Association), Billy Rivers Jr. (Municipality of Scammon
Bay), Brady Ross (Esri), Bruce Robson (Community and Ecology Resources, LLC), Catherine Keske (University
of California-Merced), Chandra Poe (Qawalangin Tribe of Unalaska), Colin MacArthur (Canadian Digital
Services), Dan Gillis (University of Guelph), Dave Thomas (60hertz Energy), Denise Roy (EPA OLEM), Derrick
Spoelman (Waste Management National Services, Inc.), Desiree Mack (Alaska Native Tribal Health
Consortium), Use Modene (60Hertz Energy), Jim Penor (EPA), Joel Forbes (60Hertz Energy), Jose Zambrana
(EPA ORD), Karen Chu (EPA ORD/OSAPE), Kim Katronica (EPA OLEM), Lauren Divine (Aleut Community of St.
Paul Island), Mary Fisher (Alaskans for Litter Prevention and Recycling), Melanie Eakin (Bristol Bay Area
Health Corporation), Melanie Savala (Green Star of Interior Alaska), Mike Brooke (Alaska Native Tribal Health
Consortium), Muhammad (Taha) Karimi (ORAU) at EPA ORD), Niles Friedman (U.S. Digital Response), Piper
Wilder (60hertz Energy), Ryan Bahnfleth (Esri, Inc.), Santina Gay (U.S. EPA), Scott Anderson (Native Village
of Port Heiden), Stephanie Mason (Zender Environmental Health and Research Group), Tonya James
(60Hertz Energy), Trisha Bower (Alaska Department of Environmental Conservation), and Vanessa Tahbone
(Kawerak, Inc).
The list (above) excludes project participants who are also named authors of this report.
ii
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
ions
ADEC Alaska Department of Environmental Conservation
ADFG Alaska Department of Fish and Game
AIS Automatic Identification System
AK State of Alaska
ANTHC Alaska Native Tribal Health Consortium
App Digital application, especially a software application designed for a mobile device
ATV All-Terrain Vehicle
Backhaul Cargo carried on a return journey; to carry freight on a return journey
CIFAR Canadian Institute for Advanced Research
CONEX a type of shipping container: Container Express (a large, steel-reinforced reusable
container for shipping or storage).
CT Control Tower
DOT U.S. Department of Transportation
EPA U.S. Environmental Protection Agency
ESRI Environmental Systems Research Institute; suite of mapping tools by Esri Inc.
e-waste Electronic waste
GIS Geographic Information System
GPS Global-Positioning System
GIF a file in Graphics Interchange Format
HHW Household Hazardous Waste
IARPC Interagency Arctic Research Policy Committee
ID Identification number
IRB Institutional Review Board
LEO Local Environmental Observer; LEO Network members share environmental data
MB Megabyte
N/A Not Applicable
OLEM EPA Office of Land and Emergency Management
ORD EPA Office of Research and Development
OS Software operating system
PCB Polychlorinated Biphenyl
PPE Personal Protective Equipment
QAQCs Quality Assurances and Quality Controls
RIO EPA Region 10
RC Regional Coordinator
RCRA The Resource Conservation and Recovery Act
RSTIP EPA Regional State Tribal Innovation Project
SC Statewide Coordinator
SWAT Solid Waste Alaska Taskforce
USB Universal Serial Bus, a standard for connecting digital devices to each other
VC Village Coordinator
VHF Very High Frequency Radio
WA State of Washington
Wi-Fi Wireless networking technology that uses radio waves to provide Internet access
iii
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Abstr
Roughly 200 rural Alaska communities, with mostly Alaska Native populations, operate Class III landfills that
are not lined (due to environmental constraints). In such landfills household hazardous waste (HHW, e.g.,
batteries, electronics, and light bulbs) can leach chemicals into the environment which, through various
pathways, impacts human health and subsistence resources that provide economic and nutritional value for
many community members and are important culturally and for other reasons. Waste burning without
emissions treatment or temperature control is also carried out at many of these landfills to reduce waste
volume; presenting another potential point for toxins to enter the environment. Additionally, reactive and
ignitable HHW materials can explode or catch fire under certain circumstances. To avoid potential toxin
release, many communities employ "backhaul" as much as possible, which involves shipping out waste on
barges and planes that might otherwise return south empty after bringing materials into communities.
Communities receive discounted shipping rates; however, the practice is still expensive, coordination is
challenging, and training is needed to ensure waste is packed properly and safely.
A program called Backhaul Alaska was created to address these challenges on a statewide scale. After
developing successful approaches in several pilot locations, Backhaul Alaska identified a need to develop a
digital application, or app, to support and improve backhaul training, waste inventory, tracking, and
feedback. In response, this United States Environmental Protection Agency (EPA) Regional-State-Tribal
Innovation Project (RSTIP) evaluates how an app can best support Backhaul Alaska and how different
technology options might address these needs. We took a human-centered design approach that considers
how the design of a technology goes beyond target users and looks at the full socio-political context in which
a product is developed and deployed. We conducted 45 conversations, presentations, and/or workshops
with 38 individuals internal and external to Backhaul Alaska and analyzed video footage of a pilot app being
used by the program. Through qualitative analysis, our research identifies and describes critical themes and
other program considerations that app development must address. We evaluate different users' app needs
and how users relate to and work with digital technology. Critically important for app development, we also
outline existing knowledge gaps about users' needs and how these unknowns should be taken into
consideration when developing an app. We also evaluate the pros and cons of four app development options
and how they suit Backhaul Alaska. While there is no perfect solution, our work provides a blueprint for
developing one or more options to support Backhaul Alaska and provides important lessons learned for
other programs operating in similar contexts that might wish to develop apps.
iv
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Executive Summary
There are roughly 200 communities in rural Alaska that operate Class III landfills. Class III landfills in Alaska
are unlined (due to environmental constraints), which means that leachate cannot be collected and treated
as it typically would in a lined landfill; and waste is often burned, without emissions control, to reduce
volume. This is allowed under Alaska Administrative Code 18 AAC 60 (Alaska DEC 2022), because of a lack of
infrastructure in rural Alaska. Leaching and burning of household hazardous wastes (e.g., lead acid batteries,
electronics, and light bulbs) releases contaminants, creating human exposure risks. Contaminants also enter
local food chains and are detrimental to cultural and subsistence hunting, fishing, and gathering activities
that are important to residents, the majority of whom are Alaska Native. One option to deal with hazardous
wastes is to "backhaul" them, a process where waste is shipped out on planes or barges that might otherwise
return south empty after bringing materials into communities. Even at discounted backhaul rates, the
practice is expensive, coordination is challenging, and training is needed to ensure waste is packed properly
and safely.
To support solid and hazardous waste removal, a program called Backhaul Alaska was developed to
coordinate statewide recycling logistics and provide training and assistance to community staff to ensure
waste is safely and properly packed for transport (SWAT 2017, 2023, Backhaul Alaska 2017, 2023,
https://backhaulalaska.org). As Backhaul Alaska expands past its successful pilot phase to eventually serve
all rural communities statewide, program administrators identified a need to use digital applications (apps)
to improve communication, data inventory, program management, and participant training.
In response, this United States Environmental Protection Agency (EPA) Regional-State-Tribal Innovation
Project (RSTIP) aims to: (1) Understand the goals, practices, and preferences of Backhaul Alaska participants
to ensure that app development meets their needs; (2) Identify areas where an app or similar technology
can help build capacity for Backhaul Alaska and its constituency; and (3) Communicate the research process
and outcomes to other programs operating in similar contexts. The project builds on a longstanding working
relationship among EPA Region 10, EPA Office of Research and Development (ORD), and local partners
working on backhaul and tribal solid waste management challenges in Alaska. This report is informative to
Backhaul Alaska, its tribal participants, other groups with connections to Backhaul Alaska, and technology
developers who might create future content for the program.
We took a human-centered design approach that considers how the design of a technology goes beyond
target users and the immediate work environment where it will be used to understand how it will impact
and be impacted by the larger social and environmental context in which it operates. Accordingly, we: (1)
sought to learn from existing environmental or resource management apps being used in the region; (2) had
conversations (interviews) with regional waste management experts (working external to Backhaul Alaska),
technology experts, and Backhaul Alaska pilot participants and staff; and (3) analyzed video footage that was
collected by pilot program participants and staff as they conducted backhaul activities and training, and used
a pilot app in the field. Research activities were synthesized at several stages and communicated with
Backhaul Alaska pilot participants and administrators to discuss project findings, explore outstanding
questions, and co-interpret the work, resulting in 45 conversations, presentations, and/or workshops with
38 individuals.
v
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Through qualitative analysis, our research details the working conditions in which people do backhaul work,
as well as their views and attitudes about how they can be supported by an app. We identified and analyzed
eight important themes: (1) working offline and in remote locations, (2) communication and coordination,
(3) creating community profiles, (4) capacity building for people and programs doing the work, (5)
inventorying and sharing information about equipment and materials, (6) opportunities for synergies, (7)
addressing liability and safety concerns, and (8) addressing data sovereignty and culturally specific
considerations for Alaska Native communities. We also evaluated how the program will grow over time, data
management needs, and geospatial needs. Our analysis of these program dynamics provides a blueprint for
developing a Backhaul Alaska app.
We also created user profiles detailing what different users will need from the app and how the app must
account for their specific work environments and personal backgrounds. Critically important for app
development, we also outline existing knowledge gaps about users' needs and how these unknowns should
be taken into consideration when developing an app. Some things about users and their needs are unknown
because of the limitations of this study. For example, we were unable to speak directly with all users.
Unknowns and uncertainties in this category may warrant more specific user-based research. Other items
remain unknown because Backhaul Alaska is continuing to evolve around how best to remove hazardous
waste from rural Alaska communities, build local and regional capacities, support job creation, and improve
human well-being. Therefore, app development must be able to adapt to program changes.
Finally, we assess the pros and cons of four app development options and how they suit Backhaul Alaska.
These include custom app development, using third party apps (often chosen over custom development to
reduce cost and long-term maintenance of the technology), web apps, and websites. We note tradeoffs in
functionality and customizability vs. cost, which were key considerations for program administrators.
Importantly, there is no one best solution. We also document Backhaul Alaska's experience trying two third
party apps. This mini case study highlights the importance of financial costs when making decisions, and how
previous investments in learning to work with a specific tool can be an obstacle to adapting something new.
We also compare record collection with a digital app and paper-based documentation, verifying that digital
technology was preferred.
Our work highlights important and transferable lessons. First, do not develop a tool based on an assumed
problem. The initial scope of the project was to understand how to design a custom app. A fundamental
principle of human-centered design, however, is to validate the problem statement through research. Once
we understood the interests of most of the potential users, and learned from working with the pilot apps,
we ended up shifting the focus from custom app development to more affordable alternatives. We chose
not to develop or test specific user interfaces, which would take valuable researcher and participant time
for something that may not have been needed. Rather, we continued to build foundational knowledge about
the program's app development needs to inform app development options. We did, however, document
important user interface improvements based on evaluation of one of the tested third-party apps.
Second, human-centered design can be a long road. Even after we formally closed the research phase of our
project, EPA researchers and Backhaul Alaska administrators continued to work together to optimize
Backhaul Alaska implementation, e.g., discuss third-party app options in order to assess product suitability
for Backhaul Alaska, evaluate prices for different licensing options, and decide on a near-term app solution.
vi
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Building trusting and long-term working relationships is vitally important when designing from a human-
centered perspective, perhaps even more so when the work depends on trust and relationships from
community participation. We encourage other programs doing similar work to commit fully to developing
these relationships, even if it means pushing back deadlines or engaging in extra tasks.
vii
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Table of Contents
Notice and Disclaimer i
Acknowledgments ii
Acronyms and Definitions iii
Abstract iv
Executive Summary v
Table of Contents viii
List of Boxes, Figures and Tables x
List of Figures and Tables in Appendices xi
Section 1. Introduction 1
1.1 Household Hazardous Waste in Rural Alaska and the Need for an App 1
1.2 Statement of Authorship and Voice 5
1.3 Project Objectives 5
1.4 Report Audience 5
1.5 Report Structure 7
1.6 Research Context 7
1.7 Human-Centered Design in Research and Development 8
1.8 Methods and Sources of Information 9
Section 2. The Backhaul Alaska System 13
2.1 The Backhaul Alaska Program 13
2.2 Critical Implementation Themes and Relevancy to App Development 16
2.2.1 Offline and Remote 16
2.2.2 Communication and Coordination 16
2.2.3 Community Profiles 21
2.2.4 Capacity of Staff and Personnel 22
2.2.5 Equipment and Materials 23
2.2.6 Opportunities for Synergies 23
2.2.7 Liability and Safety Concerns 24
2.2.8 Data Sovereignty and Cultural Considerations 25
2.3 Data Management Needs 26
2.4 Geospatial Needs 29
2.5 Section Summary 30
Section 3. Users of the Backhaul Program App 32
3.1 Introduction 32
3.2 User Profiles 32
3.2.1 The State Coordinator (SC) 33
3.2.2 The Control Tower (CT) 34
3.2.3 Regional Coordinators (RCs) 35
3.2.4 Village Coordinators (VCs) and Supporting Staff 36
3.2.5 Shippers (Small and Large) 37
3.2.6 Vendors (focusing on receiving and not supplying) 38
3.2.7 Other Users 39
3.3 Summary of Generalized App Needs by User Type 40
Section 4. App Development Options 41
4.1 Introduction 41
4.2 Development Options 41
viii
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
4.3 Pilot App Case Studies 44
4.4 Future Development Options for Backhaul Alaska 46
4.5 What About Paper? 47
4.6 Section Summary 49
Section 5. Conclusion and Recommendations 50
5.1 Report Summary 50
5.2 Looking to the Future 51
5.3 Transferable Lessons Learned 53
Section 6. Literature Cited 55
Appendix A. Surveyl23 User Feedback 59
A.l Objective 59
A.2 Background 60
A.3 Data and Methods 60
A.4 Participant Feedback 61
A.4.1 Surveyl23, Positive Sentiments 61
A.4.2 Surveyl23 Improvements 62
A.5 Possible Expansions 66
Appendix B. Details of Materials and Methods 68
B.l Research Approach and Materials 68
B.2 Project Timeline 71
B.3 Details of Conversations, Participant Recruitment, and Coding 72
B.4 Details of Photo and Video Interpretation 74
Appendix C. Thematic Code Book 75
Appendix D. Story board Provided to Professional Videographer and RCs for Collecting Video Footage of
Backhaul Activities 78
D.l Storyboard Provided to Professional Videographer: 78
D.2 Storyboard Provided to RCs: 80
Appendix E. App Requirements Document Used for Communicating with Developers During the Project
81
E.l Backhaul Alaska Software Application Functional Specification Document 81
E.1.0 Background 81
E.l.l Objectives & Outcomes 82
E.l.2 Product Context 82
E.2. Requirements 83
E.2.1 Functionality Table 83
E.2.2 Usability Requirements 84
E.2.3 Technical Constraints 84
E.2.4 Device Specifications 84
E.2.5 Data Requirements 84
E.2.6 Access Requirements 84
E.2.7 Security Requirements 84
Appendix F. Select Results from the App Specific Systems Workshop 88
Appendix G. App Wish List Developed Before Commencement of RSTIP 92
Appendix H. Statement of Human Subject Exemption 94
ix
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Boxes
Box 2.1 A nightmare scenario
Box 2.2 What is geospatial data?
Box 4.1 What is an app?
Figures
Figure 1.1 Backhaul loading
Figure 1.2. Location of communities participating in the Backhaul Alaska Pilot: Study area map
Figure 1.3 Overall project timeline
Figure 2.1 Illustration of core actors in the Backhaul Alaska Program
Figure 2.2 Annual cycle of Backhaul Program activities and how an app might support individual stages
Figure 2.3. (A, B) Backhaul implementation overtime
Figure 2.4 (A, B) Communication pathways among actors in Backhaul Alaska
Figure 2.5 (A, B) Process flow diagrams for the loading and shipping phase of the backhaul process as
documented on paper and in practice
Figure 2.6 (A, B) Proposed general process for formal notification and verification of tasks in Backhaul
Alaska
Figure 3.1 SC assisting with local village backhaul and tracking program training with a spreadsheet
Figure 3.2 Database software used by the CT for creating shipping manifests
Figure 3.3 RC training village staff in an office
Figure 3.4 Laborer packing e-waste amidst 12 feet of snow
Figure 3.5 Properly packaged waste received in Seattle, WA
Figure 4.1 Comparison of app development options considering customizability and functionality vs. costs
Figure 4.2. The challenge of trying to do Backhaul inventory using paper
Tables
Table 1.1 The number of participant engagements by type
Table 1.2 List of participant engagements by participant type
Table 2.1 Communication types and their app development implications
Table 2.2 Possible scenario of generation and access for records among Backhaul Alaska Program actors
Table 3.1. Generalized App needs for Backhaul Alaska user, by user type.
Table 4.1 Comparison of several different development options
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
List of Figures and Tables in Appendices
Figures in Appendices
Figure A.l Backhaul Alaska staff using Surveyl23 in the field to inventory waste in a shipping container
Figure A.2 Local village staff completing a survey on the app
Figure B.l Activity timeline of this RSTIP
Figure E.l Backhaul Process Flow Diagram
Figure E. 1.1 The Problem. Improperly discarded hazardous waste in unlined landfills of rural Alaska
Figure E.2 Shipment Sub-process Flow Diagram
Figure E.3 Inventory Management Data Flow Outline
Figure F.l System diagram of Backhaul Alaska operations pertinent to use of an app
Tables in Appendices
Table B.l List of conversations by participant type.
Table B.2 Example conversation agenda used with representatives of SWAT, RCs, VCs, and their assistants
during the early stages of the project
Table B.3 Example conversation agenda used with technology experts during early stages of the project
Table B.4 Thematic codes for binary tagging video content
Table C.l Thematic code book
Table D.l Guidelines for professional videographer
Table D.2 Guidelines for RCs
Table E.l Functionality Table
Table F.l Selection of key actors and system dynamics generated during the workshop discussion.
Table G.l Pre-project "Wish list" of items for a custom developed Backhaul Alaska Program App
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Sec
1.1 Household Hazardous Waste in Rural Alaska and the Need for an App
During several cold spring days, working in the wind-swept Alaska tundra, local community members in a
small, mostly Alaska Native fishing community, along with a state-wide nonprofit, gathered and safely
packed 3,334 pounds of electronic waste, 12,558 pounds of lead acid batteries, and 283 pounds of UV lamps
(Zender Group 2022). The material would later be transported south to Seattle, Washington where it could
be safely disposed of or recycled in an appropriate facility. Had the waste not been shipped out, it would
continue to accumulate indefinitely and leach into the environment over time, or worse, have been burned
in an uncontrolled manner, releasing toxins and other contaminants.
The majority of communities in rural Alaska, roughly 190, have no road access to the State Highway System
or to other communities (Barnes 2020), and are accessible only by boat or plane. Removal of waste and
transporting it to an appropriate end-destination at a larger city recycling or disposal facility is both costly
and difficult. Even for those communities that are connected to the road system, transport is challenging
because of harsh weather or poor, often seasonal, roads. Due in part to their remoteness, communities in
rural Alaska lack the infrastructure to safely dispose of hazardous waste locally. Because of complex
geohydrological (land and water systems), economic, and logistical factors, most small communities in rural
Alaska operate Class III landfills that, in compliance with Alaska Administrative Code 18 AAC 60 (Alaska DEC
2022), are exempt from several protective elements of the federal Resource Conservation and Recovery Act
(RCRA) municipal landfill regulations, which outline minimum design and operation requirements. Class III
landfills are unlined, which means that leachate cannot be collected and treated, as it would in a typical lined
landfill. As part of the exemptions, burning of waste in steel tanks or cages lacking emissions treatment as
well as any temperature and air controls may also occur because it is an inexpensive means of volume
reduction and landfill litter control.
Toxic contaminants, via waste degradation and subsequent leaching or waste burning, are released into the
surrounding soil, water, and air from Class III landfills. Of particular concern are electronic waste (e-waste)
and lead acid batteries which contribute an estimated 95 percent by weight of toxic heavy metals in
municipal waste streams (EPA 2021). Because community landfills in rural Alaska are often proximate to
homes, human activities, water bodies used for drinking water, marine transportation, subsistence
harvesting, and other purposes (Zender Group 2017, Kawerak 2021), there are many potential exposure
pathways that can negatively impact human health and the environment (Samuelson 1998, Eisted and
Christensen 2013, Keske et al. 2018).
One option to deal with these hazardous materials in rural Alaska communities is to ship them out by a series
of plane and/or barge transportation legs to recycling processing facilities. This process is called "backhaul,"
which gets its name from the utilization of vessels, aircraft, and vehicles used in transportation that
otherwise would have returned empty to hub locations from rural communities. There are no hazardous
waste processors in Alaska, so materials must be transported to Vancouver or Seattle before being routed
to its final destination. Arranging shipment is complicated and requires an iterative series of steps (Zender
Group 2017, Kawerak 2021).
1
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Figure 1.1. Backhaul Loading. Photo of an improperly packed load of waste (A, on left) and the same load repacked
following the Backhaul Alaska training (B, on right).
In 2014, the Solid Waste Alaska Taskforce (SWAT) was formed as a multi-institution taskforce to collaboratively
tackle rural Alaska solid/hazardous waste challenges (Zender Group 2017, SWAT 2023). SWAT is comprised of
the Alaska Department of Environmental Conservation (ADEC), the Alaska Native Tribal Health Consortium
(ANTHC), the non-profit corporation Kawerak, Inc., and the community-based non-profit Zender Environmental
Health and Research Group (Zender Group 2017, Kawerak 2021). At that time, the activities and interests of
several groups and initiatives came together through meetings and workshops as outlined in Zender Group
(2017) and the Backhaul Alaska website 1 in order to develop a state-wide plan to address Alaska
solid/hazardous waste challenges (SWAT, 2023). The groups and initiatives included work by the Alaska
Senator's office, a statewide backhaul panel, meetings among Regional Tribal representatives and local
communities, local, regional, statewide, Tribal, and federal governments and agencies, transporters, recyclers,
and other groups with connecting interests. In 2017, after being vetted and refined, this plan resulted in the
Intermittent backhauling by a handful of individual communities and organizations started in rural Alaska in
the early 2000s. Many barge and plane transporters supported this effort and even provided free-of-charge
or discounted shipping. By the 2010s, backhaul became more widespread throughout rural Alaska. Around
this time, transporters began seeing not just increased volume, but poorly packaged materials, mislabeled
hazardous waste, leaking batteries, and more (Figure 1.1). Shipping of hazardous waste falls under strict
federal United States Department of Transportation (DOT) packaging regulations and requires safety training
for all personnel that handle or transport the waste. At that point in time, recyclers were receiving materials
that they had not agreed to take and for which communities did not pay. As a result, companies started to
refuse any backhaul materials from rural Alaska, while others would not serve many of the smaller
communities, and backhaul costs began to increase (Personal observations by authors LZ, RK, and SS who
helped develop the Backhaul Alaska Program; Zender Group 2017, Kawerak 2021).
Photo credit: Backhaul Alaska
1 https://backhaulalaska.org/overview/
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
creation of the Backhaul Alaska Program, commonly called Backhaul Alaska and referred to as such throughout
this document.
The goal of Backhaul Alaska is to coordinate the safe removal of hazardous and potentially harmful waste
from rural communities, with a focus on bolstering local community capacity and creating local jobs. The
program ensures that every participating community has trained staff that knows how to inventory, stage,
and safely package waste for transport. Because it is economically and logistically infeasible for most
communities to independently manage this process, Backhaul Alaska also helps to scale up these efforts by
coordinating with vendors to ship waste material to recycling facilities (Zender Group 2017).
Starting in 2018, SWAT began to pilot Backhaul Alaska in order to develop, evaluate, and improve: (1) staff
training, (2) waste and equipment inventory methods, and (3) transportation and shipping logistics (Kawerak
2021). It was essential for the program to consider how it would incorporate a range of unique needs
throughout rural Alaska, which is culturally, environmentally, and geographically diverse (Figure 1.2). Alaska
stretches from North of the Arctic circle, where the landscape is arctic tundra, 1,400 miles south to
temperate rainforest along the south-central and southeast costs. Areas in between include boreal forest in
the interior of the state and maritime tundra along the coast and islands (ADFG 2015). Rural Alaska includes
roughly 200 small communities, typically with a few hundred residents, the majority of which are Alaska
Natives living on ancestral lands (ADFG 2015). In total, there are 229 federally-recognized Alaska Native
Tribes (ADFG 2015), using roughly 20 different languages; the number of speakers and levels of fluency vary
among languages and communities.2
To establish a set of pilot communities, Backhaul Alaska, advertised the program and accepted applications
from self-identifying communities, and also consulted with various regional and statewide peers to establish
a pool of communities. These included communities with some backhaul experience, spanning Alaska's
diverse geography, a range of population sizes, and airplane and barge access. Backhaul Alaska was able to
include all 27 nominated communities within the phased three-year pilot (Personal observations by author
LZ; Figure 1.2).
The Backhaul Alaska pilot paved the way for what is now a growing and successful hazardous waste removal
and recycling system for small, remote communities (Kawerak 2021). At this stage, the pilot relied heavily
on manual data entry, paper forms, email, and direct phone calling. SWAT identified a need to develop
automated efficiencies in its coordination, communication, and verification processes as it scaled up over
the next ten years to include approximately 160 to 200 communities3 across rural Alaska (Kawerak 2021).
2 Alaska Native Language Preservation and Advisory Council. Accessed online (22 Febuary 2023) at
https://www.commerce.alaska.gov/web/dcra/AKNativeLanguagePreservationAdvisoryCouncil/Languages.
3 Kawerak (2021) uses an estimate of 160 communities for full program capacity which we continue to use here. The exact number
of communities in the program may be more or less than this number depending on a variety of factors, and other backhaul
planning and communication documents sometimes report a different target number.
3
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Sea
Location of Communities Participating in Backhaul Alaska Pilot
Chukchi
8
° v>
— Q- _
°OQ ALASKA o N
° uni9ijd
J*®® stmt
' A *° •
iJnalaaeet
9°
p 0
0
Arctic
' Village
0 /
O
Vsiette O
0 /
—
-0 -4
— O O
Baring
S#a
O
Saint
: a ~
Aialtanuai'V _
V%°
HcoperBJi'^. 3^0,33, o ©
Q Gfc? OieS^lP o
U Jk. OK»»gKjanak
"~o Koiganen ^hain
KsHgillnga* M gj ..\
O E"q ®NE?Sltyawi
=•• •*>< "r jk
P«frpfeiasi ¦ o C0^<
bST* «
f
\
Backhaul Alaska Pilot
Communities by Region
•
Aleutians
¦
Bristol Bay/Lake & Perm
O
Interior
A
Kodiafc
•
North Slope
v
Northwest Arctic
0
Norton Sound
Western
-
o.
Larsen B§
j| Cfilgnii
OO Lagcor
0%) ©
o
OIO Hartoof
O Alaska Communities
Major Roads
— — Arctic Circle
GCI Mobile Coverage
o
o o
ib
250
I
500
J
Figure 1.2. Location of communities participating in the Backhaul Alaska Pilot. Study area map showing the 27 rural
Alaska communities that participated in the Backhaul Alaska pilot (blue symbols and gray names), with region
indicated by the symbol shape. Other communities and roads are depicted for context. Most Backhaul Alaska pilot
communities are not connected to the road network and are only accessible by boat or plane. Areas with mobile Wi-
Fi access are shown in green shading. Data Sources: Pilot Communities https://backhaulalaska.org/pilot-program
(Backhaul Alaska 2023) and Mobile coverage www.gci.com/.
As Backhaul Alaska expanded, SWAT and EPA Region 10 staff began to explore whether using a mobile app
(i.e., digital application) for program participants could help address some of the logistical and coordination
challenges and support additional program goals. Removal, recycling, source reduction, and cleanup
activities associated with hazardous waste backhaul have potential for creating jobs, cost savings, improved
health, and clean-up of culturally important areas. Day-to-day operational needs include methods to manage
inventory, package, and track shipments, submit observations that impact operations, connect with peers,
and receive program support. To help address these needs, EPA Region 10 and EPA Office of Research and
Development (ORD) staff applied and received Regional-State-Tribal Innovation Project (RSTIP)4funds to
develop criteria and evaluate options for the design of a mobile app, or similar technology solution, that
would serve the user-base of Backhaul Alaska (including but not limited to regional and village backhaul
4 The RSTIP program is an EPA ORD funding program to support EPA regions in pursuit of "innovative projects to address state and
tribal science priorities".
4
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
coordinators and staff) and the wide-ranging data needs for operating the program. The project, which is
the focus of this report, was a collaboration among EPA regional program staff, EPA research scientists, and
the community-based non-profit, Zender Group, that is the lead for developing and administering the
Backhaul Alaska Program on behalf of SWAT. Because of this partnership approach, the project drew from a
variety of community-based research and human-centered design approaches as described below (Sections
1.6 and 1.7).
1.2 Statement of Authorship and Voice
As a collaboration among EPA regional program staff, EPA research scientists, and the community-based
non-profit Zender Group, this report contains multiple voices. Zender Group and EPA Region 10 staff have
been working for years on solid waste and related environmental issues in rural Alaska and are experts in
this regard. Throughout the text, we have attempted to clarify who the active speaker is when warranted.
This report presents a variety of research findings. Some findings are novel and illuminating to all team
members. Other findings represent information well-known to the on-the-ground practitioners on the team
and were analyzed or reinterpreted using specific methods and frames by the EPA research team for case
specific or more generalizable objectives and audiences.
1.3 Project Objectives
Our guiding question was: What are the best design options for a Backhaul Alaska Program data management
application, or similar technology, that addresses different users' needs for supporting program operations
and decision-making?
The project aims to:
1) Understand the unique goals, practices, behaviors, and beliefs of Backhaul Alaska participants to
ensure that app development meets their needs.
2) Identify areas where an app or similar technology can help build capacity for Backhaul Alaska and its
constituency, including but not limited to training, program operations, and data management.
3) Communicate our process and outcomes so that other programs operating in similar contexts can
learn from them.
The primary short-term outcome is to provide guidance for program administrators and participants that
supports adaptive and responsive program operations, decision-making, and expansion. Long-term, the
research presented in this report will help develop one or more information management tools that assist
communities as they work to build their capacity to implement waste management best practices.
1.4 Report Audience
This report may be of interest to several audiences:
• For Backhaul Alaska: This report documents and analyzes important Backhaul Alaska operations, as
well as data management and resource needs, among program participants. While this report is not
an all-encompassing analysis of Backhaul Alaska, it does investigate important issues related to
coordination and communication, information sharing and access, data collection, storage, and
5
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
visualization, and overall program function in relation to developing an app, or similar technology,
that can support Backhaul Alaska. It synthesizes and analyzes important insights from individuals
involved in Backhaul Alaska, which can identify what is working and where opportunities exist for
improvements.
• For Programs of Similar Context: This report outlines an approach to app development for a
program meant to serve predominantly Alaska Native communities that are remote, with limited
internet and technology infrastructure. The experiences and lessons learned that are presented in
this report will be informative to similar programs and contexts (e.g., rural or island communities)
throughout the United States and elsewhere.
• For App Developers: This report is designed to provide a foundation for developing an app, or similar
technology solution, for Backhaul Alaska. It documents and analyzes the relevant program
operations, users, and considerations such as costs and time investments that need to be included
in an app by a technology developer. A single solution is not proposed; instead, the advantages and
disadvantages of several development options are presented. Because choices about app user
interfaces and features will be influenced by the specific app developed and its underlying software
(Teacher et al. 2013, Jabangwe et al. 2018, Gerlitz et al. 2019), this report focuses on identifying
users' needs and a holistic understanding of Backhaul Alaska that can assist with the selection of
user interfaces and features, independent of the development approach. Where relevant, specific
technology features and user interface recommendations are made.
• For Communities in Rural Alaska: This report documents important user needs and program
logistics and analyzes them in a way that can be used to build tools to enhance local capacity to
safely and effectively carry out backhaul activities. This report also supports activities intended to
protect subsistence resources, create jobs, and remove toxins from the land; these values were
considered when developing the mobile app technology for Backhaul Alaska.
6
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
1.5 Report Structure
This report consists of five sections, each concluding with a section summary. Additional and supplemental
information are provided in seven appendices, labeled A through H.
Section one introduces Backhaul Alaska, and explains the context, research approach, methods, and sources
of information used in this research project. A more detailed and technical methodology is provided in
Appendix B.
Section two looks at how Backhaul Alaska operates and discusses what various characteristics of the program
mean for app development. The chapter starts with an analysis of seasonal and interannual program
activities and the opportunities for an app, or similar technology, to support various stages of the program.
Section two outlines eight critical themes to consider when developing an app or similar technology. It also
outlines the program's data management and geospatial data needs.
Section three focuses on likely app user groups within Backhaul Alaska, outlining their roles and
responsibilities, working conditions, backgrounds, key challenges related to Backhaul Alaska, and what they
are likely to do with a Backhaul Alaska app. Importantly, section three also outlines any unknowns and
unanswered questions for different use types, and items that might warrant future research.
Section four explores the suitability of several different options for developing a Backhaul Alaska app,
considering factors such as customizability, functionality, and cost. Section four also details a case study
about Backhaul Alaska's trial use of two different apps during the 2020 - 2021 program pilot session.
(Detailed user feedback is provided in Appendix A). The case study illustrates important decision points,
historical dependencies, and long-term thinking that factor into Backhaul Alaska's app development choices
and next steps. Finally, given the remote, offline environment, and harsh weather conditions of rural Alaska,
which are challenges for many digital devices, section four discusses strengths and weaknesses of a digital
vs. paper solution for Backhaul Alaska.
Section five provides a short summary of the report and discusses the findings. Section five also considers
how anticipated environmental, technological, and economic changes might impact future design
considerations for a Backhaul Alaska app and possible steps to consider in addressing such changes. Finally,
section five provides a set of lessons learned that can be transferred to other programs interested in using
human-centered design approaches for similar problems.
1.6 Research Context
Mobile apps are increasingly being used for environmental data collection, citizen monitoring, and gathering
of public health information (Teacher et al. 2013, Andrachuk et al. 2019, Holeman and Kane 2019). Apps
might be custom made, or developed using off-the-shelf software that allows selection of specific data fields,
workflows, content, and possibly appearance and user interface elements (Andrachuk et al. 2019). Ideally,
end users are consulted during development to ensure that apps meet program end goals and can be used
effectively by target audiences (Teacher et al. 2013). All too often though, end users are not adequately
involved in the development of digital applications. For example, in a review of 71 studies using apps for
environmental conservation by Andrachuk et al. (2019), only 15% confirmed end user consultation during
app development, 69% confirmed no user consultation, and 15% could not be determined. Similarly, out of
7
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
62 papers about environmental citizen science apps reviewed by Skarlatidou et al. (2019), only 29% involved
direct user evaluation.
Consulting with end users to better understand the context in which an app will be used is particularly
important in communities with extreme environmental conditions and a lack of digital infrastructure (Kouril
et al. 2015, Kipp et al. 2019). Without a full understanding of local environmental, social, and technological
characteristics of communities, developed apps may fail to meet program goals. As a research approach for
product development or app design, community engagement can take several forms, ranging from
consultation, where researchers discuss ideas with community members, to full co-development, in which
researchers and community members co-develop questions, research methods, and work together on data
analysis and interpretation (Reed et al. 2014, 2018). No single form of community engagement fits all
situations, and the level of engagement should respect both researchers' and community participants'
capacities, the scope of the research, and the research objectives (Jasney et al. 2021). Nonetheless, taking
engagement seriously can help ensure that research will meet the needs of the communities involved (Kouril
et al. 2015, Ford et al. 2016, IARPC 2018, Kipp et al. 2019).
When community-based work involves co-collecting data, or community members collecting data that is
shared with research teams, data ownership and privacy are important issues (Kouril et al. 2015, Ford et al.
2016). Assumptions about data storage and sharing can lead to unintended consequences; for example, even
well-meaning open-access data standards may serve to reinforce existing inequalities when those
communities that wish to use and safeguard their own data lack the resources or expertise to do so (Lewis
2020). Understanding cultural and local socio-political contexts around technology and data are therefore
vital to ensure that an app is developed successfully and mindfully.
1.7 Human-Centered Design in Research and Development
To ensure that this research provided a holistic understanding about how to develop an app, or similar
technology, for Backhaul Alaska, we used human-centered design. Human-centered design is an approach
to developing technology products that "puts people at the center ... prioritizing [their] aspirations and
ordinary experiences when imagining and implementing complex systems, services, or products" (Holeman
and Kane 2019:488). Human-centered design is a dynamic field of practice, influenced by design, technology
studies, human-computer interaction, anthropology, and public health (Bannon 2011, Holeman and Kane
2019). It shares some similarities with user-centered design, which emphasizes the experiences of target
users (Harte et al. 2017). Human-centered design, however, is broader and less strict in scope, considering
how the design of a technology goes beyond just target users, and looks at the full socio-political context in
which a designed product is developed and deployed. Holeman and Kane (2019:496) further describe this
idea:
8
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
"... designers who build technologies or solve technical problems can hardly claim that their
work is human-centered if they systematically ignore human rights or humanitarian
concerns that are part of daily life for the people they purport to 'design with' or serve. ...
this ... involves looking beyond the design of discrete technologies to reimagine services, the
organization of health systems, and broader social arrangements that pattern who receives
equitable care and who does not."
Human-centered design intentionally blurs the separation of design and implementation (Holeman and Kane
2019), which we interpret to also extend to research and implementation. It is an intentionally iterative
process, through the phases of development, application, research, and reflection (Holeman and Kane 2019).
While private sector development using human-centered design often targets short research and
development cycles (Chen et al. 2020), tools for public health (an analog for Backhaul Alaska) might be
developed and refined over several years (Holeman and Kane 2019).
Many of the human-centered design principles that are outlined above align tightly with community-based
approaches that are essential when working with Indigenous communities (Ford et al. 2016, IARPC 2018). As
reviewed by Chen et al. (2020), there are several differences worth noting. Human-centered design
prioritizes empathy and creativity, focuses on understanding individual users at extreme ends of the user
spectrum, often works under shorter time frames, and seeks to generate scalable solutions. In contrast,
community-based research emphasizes relationship and trust building, often working with community
members and other interested groups over long periods to develop local capacities and arrive at localized
and context specific outcomes.
Because Backhaul Alaska focuses on serving Alaska Native communities, we integrate many of these
community-based research principles with our human-centered design approach. Specifically, we pursue
relationship and trust building with Backhaul Alaska participants and others with connections to Backhaul
Alaska, as well as community capacity building and context specific solutions for rural Alaska waste
management needs. Because Backhaul Alaska is statewide in scope and is expected to eventually encompass
roughly 160 to 200 communities, we are also cognizant that the work done for this specific report cannot be
fully community-based. Depth and breadth require a certain tradeoff, and our approach here might best be
thought of as research and design that is locally informed and regionally focused.
1.8 Methods and Sources of Information
Figure 1.3 depicts the project timeline, which centered around three activities that built on one another
through iterative rounds of research synthesis and participant engagement: (1) We held conversations with
representatives from other programs in the region that had developed environmental or resource
management apps in order to learn from their experiences. (2) We held conversations with regional waste
management experts (working external to the Backhaul Alaska), technology experts, and Backhaul Alaska
participants and staff (roles listed below). (3) We analyzed video footage, collected by Backhaul Alaska
participants, of backhaul activities, training, and pilot app use in the field. The Backhaul Alaska staff and
participants included the following: A Statewide Coordinator (SC) who oversees overall program
implementation and day-to-day operations; a Control Tower (CT) that coordinates shipping and receiving of
waste at final destinations; and Regional Coordinators (RCs) who work with, and coordinate among,
9
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
individual Village Coordinators (VCs) and their assistants in each community of a given region (Kawerak
2021).
These research activities allowed us to develop both a foundational understanding of the users and variables
affecting app design and use, and subsequent evaluative research to understand the strengths, limitations,
and needed refinements of specific ideas or prototypes (Abras et al. 2004, Daae and Boks 2015, Chipchase
2017). All research was done remotely due to the Covid-19 pandemic when travel was restricted. Further
details about methods are provided in Appendix B.
Project
Funding
timeline
Project
start
Planed Extended completion date to
completion Covid-19 & participants' requests
September 2020
September 2021 December 2021
Background: previous work
between SWAT & US EPA,
app "wishlist," survey 123
prototype
Selection
of key
events
Research activities: learning from other programs,
participant and expert conversations, and video
observations.
Pilot app testing
Continued collaboration:
meetings with developers, data
migration, communication &
outreach
Presentation
to RCs Co-interpretation
Workshop workshop: EPA research
App focused with RCs team & Backhaul Alaska
systems workshop administrators
Final evaluation
of two pilot apps
Figure 1.3. Overall project timeline. Timeline indicates project start and ends dates, background work that informed
the project, and continued collaboration after the project. Several events are outlined in the lower part of the timeline
for context. A more detailed timeline of specific research activities and intermediary synthesis documents is presented
in Appendix B, Figure B.l.
Several important background research components and activities also informed the project. These included
engagement with various experts related to Backhaul Alaska, consideration of prior work by EPA on the
Backhaul Alaska system, on experiences of Zender Group in designing and beginning to implement the pilot
program, and experience of ORD in development and use of other apps. Discussions during this background
phase of work resulted in an itemized list of needed and desirable app features (known as the "app wish
list"; Figure 1.3). An important learning opportunity for the project also involved observing and evaluating
(through conversations and video documentation) a pilot app being used by Backhaul Alaska. This app, called
Surveyl23, was an off-the-shelf, third-party app that EPA ORD researchers began to customize for Backhaul
Alaska during the project planning stage. EPA had been working with Zender Group to identify multiple
technology-based solutions to help with backhaul (e.g., shipping route optimization). Several EPA scientists
(authors TB and PL) who were involved in the 'wish list' conversations recognized that Surveyl23 could be
used for Backhaul Alaska's inventory needs, given their experiences using Surveyl23 for other projects
(Section 4.0). This account illustrates how the current project was informed and supported by a long-term,
ongoing relationship between EPA ORD, Region 10, and SWAT. The team's collaborations continued after
the official end of the project (Figure 1.3), for which the implications on our own project and those looking
to replicate similar efforts elsewhere are discussed in the conclusion (Section 5.0).
10
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Participant engagement is a cornerstone of this project. Research activities were synthesized by the EPA
research team at several stages (Appendix B, Figure B.l) to form the basis of three workshops and a
presentation with Backhaul Alaska participants and administrators to discuss project findings, explore
outstanding questions, and co-interpret the work (Figure 1.3, Table 1.1; details in Appendix B). One of these
engagements was a presentation by the EPA research team to RCs (with the SC also in attendance) to explain
the project, the planned research, and to discuss any questions and ideas from the RCs. RC insights were
essential since RCs occupied a key role in the program, facilitating communication between villages in each
region and statewide staff, allowing them to reflect on program operations at multiple levels. During the
presentation, RCs requested that the project timeline be extended from September 2021 to late December
2021 to allow more time to learn from their upcoming summer backhaul experience and use of the
Surveyl23 app. Extending the project end date also made sense due to research and administrative
slowdowns caused by the ongoing Covid-19 pandemic (Figure 1.3). Thus, the project benefited from an
adaptive and flexible approach that accounted for participants' needs and changing conditions.
Extending the project deadline also allowed everyone involved to learn from piloting a second app in
addition to Surveyl23 (refer to Section 4.0). Piloting this second app led to a series of app development
discussions and comparison work (details in Appendix B and Figure B.l) between Backhaul Alaska
administrators and EPA scientists, and a deeper reflection on Backhaul Alaska's short vs. long-term app
development needs (refer to Section 4.0).
In total, the conversations, workshops, presentations, and pilot app comparison work resulted in 45
engagements with individuals and programs, which were often represented by more than one individual
(Table 1.2). In total, the project received participation and insights from 38 individuals (Table 1.2). While
diverse in terms of geographic scope and subject matter, this participation may not be representative of all
users and regions. In each subsequent chapter, we point out possible research limitations when
documenting and analyzing how an app can support Backhaul Alaska (Section 2.0), its users' needs (Section
3.0), and development options (Section 4.0).
11
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Table 1.1. The Number of Participant Engagements by Type.
Engagement Type
Count
Description
Conversations
33
Unstructured conversations with:
• representatives of other programs using similar environmental
monitoring or management apps,
• technology and regional waste management experts (external
to the Backhaul Program)
• backhaul program participants and staff
Presentation and workshops
• Organized by project team
• Attended by project team*
4
1
Presentations and discussions with RCs, CTs, and program
administrators to get feedback about the project, discuss
outstanding questions, and co-interpret results, and/or learn
about technology considerations in the region.
Pilot app comparison work
7
A series of meetings among EPA ORD researchers, Backhaul Alaska
administrators, and representatives from one or more app
development companies to discuss, demo, and pilot two different
apps for the program; and internal discussions between EPA ORD
researchers and Backhaul Alaska administrators about different
software options.
Total
45
*1 workshop was organized by Northern Connection and EPA ORD and Zender team members participated
Table 1.2. List of Participant Engagements by Participant Type. The number of engagements is listed as a total and
with unique entities, i.e., programs or individuals, as we had repeated conversations and interactions with several
individuals and/or programs. Several programs and companies were represented by more than one person; thus, the
total number of unique programs or individuals (n = 27) is less than the total number of participants that we spoke with
(n = 38). Total participants are not listed for each participant type to avoid double counting, since several participants
occupied multiple roles (e.g., the same person may have participated as part of a group for a regional program using a
similar app and individually as a regional expert).
Participant Type
Total
Engagements
(number)
Unique
Entities
(programs or
individuals)
Total
Participants
Representatives of SWAT
3
3
Individual
breakdown
not shown
because
several
individuals
occupied
multiple
roles
Control Tower (CT) and State Coordinator (SC) operators
3
3
Regional Coordinators (RCs)
6
4*
Village Coordinators (VCs) and their assistants
5
4
Regional programs using similar apps
10
3
Regional waste management experts (external to the program)
2
2
Technology experts
10
6
Workshops and presentations with RCs (SC also in attendance)
2
--
Workshops and pilot app planning with program administrators
3
1
Northern Connection Workshop
1
1
--
Total
45
27
38
*A fifth RCparticipated through the RC workshop
12
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Section 2. The Backhaul Alaska System
This section provides a holistic understanding of the Backhaul Alaska Program. The section starts with an
analysis of seasonal and interannual program activities (as they exist in the Backhaul Alaska pilot and initial
implementation) and the opportunities for an app, or similar technology, to support current and future
stages of the program (Section 2.1). The section then outlines eight critical themes to consider when
developing an app (Section 2.2) and discusses Backhaul Alaska's data management (Section 2.3) and
geospatial data needs (Section 2.4). It concludes with a bulleted summary (Section 2.5).
2.1 The Backhaul Alaska Program
To build each community's capacity to remove hazardous waste, Backhaul Alaska operates through a
network of participants from the local to statewide level. These include a Statewide Coordinator (SC) who
oversees overall program implementation and day-to-day operations; a Control Tower (CT) that coordinates
shipping and receiving of waste at final destinations; and Regional Coordinators (RCs), who work with and
coordinate among individual Village Coordinators (VCs) in each community of a given region (Kawerak 2021).
These actors (SC, CT, RCs, and VCs) form the core of the Backhaul Alaska implementation team and are
principal app users. During our research, however, we also discussed several other user groups for a
Backhaul Alaska app. Greater detail about these potential app users will be addressed in other sections of
this report (in particular, refer to Section 3: User Profiles). Backhaul Alaska is overseen by an Executive
Committee that provides guidance and oversight (Figure 2.1., Kawerak 2021).
j Executive Committee
*
I
A
Figure 2.1. Illustration of core actors in the Backhaul Alaska Program. Each of the core actors that implement the
Backhaul Alaska Program are shown: Control Tower (CT), Statewide Coordinator(s) (SC), Regional Coordinators (RC),
and Village Coordinators (VC). Arrows show coordination between groups. Organization in this figure is based on the
General Backhaul Process Flow diagram, prepared by Backhaul Alaska (Appendix E). Alternative graphics (e.g., in
Kawerak 2021) depict a more hierarchical program structure.
At the community level, Backhaul Alaska revolves around a seasonal suite of activities that are designed to
build local capacity and facilitate the removal of hazardous waste from rural communities (villages). The SC
and RCs enroll new communities in Backhaul Alaska and train VCs and their staff on how to inventory and
safely package waste. At the end of the summer backhaul season, the waste is shipped and then tracked by
the CT (Figure 2.2). Each stage in program implementation has unique needs from an app, which could
13
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
include deploying checklists and training information, multiple kinds of inventory surveys, visualizing
information in dashboards, in-app communication, verification of different activities, and waste shipment
tracking (Figure 2.2). Some features are more critical than others and inclusion of specific features to be
incorporated in a Backhaul Alaska app may depend on how its production is implemented, e.g., whether the
program uses existing apps versus custom builds (as explained in Section 4.0).
At the state level, and working within a multi-year timeline, Backhaul Alaska seeks to enhance regional and
statewide coordination of waste removal, staff training, and financing to reduce costs and to build capacity
across Alaska (refer to key themes in Sections 2.2.1 through 2.2.5). The program expects to grow in the next
ten years from 27 pilot communities to a statewide program serving approximately 160 communities5
(Figure 2.3a) (Kawerak 2021). Assuming three users per community, plus regional and administrative actors,
an app for Backhaul Alaska must be able to support program growth and data storage needs for roughly 450
to 650 users (refer to Section 2.3 and Figure 2.3a).
Spring Summer F£[] ^
SITE VISITS OCCUR
ENROLLMENT
PLANNING
TRAINING
INVENTORY
PACKING/STAGING/
LOADING
SHIPPING
VENDOR /
RECONCILIATION
• Identify
interested and
eligible
communities
• Establish village
capacity and
needs
• Deploy village
checklist survey
• Onboard new
communities,
RCs and VCs
• Identify
capacity needs
of specific
villages
• Uniform Backhaul
Training
• Community
onsite training
• Regional
Coordinator
training
• VCs and RCs
conduct inventory
of general sites and
materials
• Produce data on
waste type,
amount, and
location
• VCs work with RCs to
accumulate, package,
stage, and load waste
for transport
• Complete/packaged
waste submitted into
BHA Inventory
• Packaged
materials are
transported
from source
or hub to
vendor
• Vendors received
packaged materials
from shipper (or
from WM staging
area)
• Price settled for
cost of waste and
reimbursement for
certain materials
• App deploys
village checklist
onboarding
suvey
• App provides
dashboard to
manage and
analyze
community
profiles
• App/website
provides access to
training for remote
learning
• App provides
guidance and
quick-start material
for specific
processes
• VCs and RCs enter
inventory records
• In app functionality
to approve, revise,
and comment on
records
• Sharing inventory
records to outside
program users
• Records are
generated that enable
RCs, SCs, and the CT
to see status and
location of packaged
materials
• SC and CT notified of
completion
• Shipper
location is
shared with
VC, RC, SC,
and CT via
AIS data
(possible)
• Vendor
communicates
feedback about
received waste
through the app
Figure 2.2. Annual Cycle of Backhaul Program Activities and how an app might support individual stages based on
the research presented in this report.
5 Kawerak (2021) uses an estimate of 160 communities for full program capacity for Backhaul Alaska, which we continue to use
here. The exact number of communities in the program may be more or less than this number, depending on a variety of factors,
and other Backhaul Alaska planning and communication documents sometimes report a different target number.
14
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
A Number of Villages in Program per year B Number of Backhauls per year
z
50
150-
o-
z
50
150
0
Communities are randomly
assigned to backhaul every:
3, 4, or 5 yrs
1, 2, or 3 yrs
1, 2, 3, 4, or 5 yrs
2021
2025
Year
2030
2020
2030
2040
2050
Year
Figure 2.3. Backhaul implementation overtime, showing (A) Program expansion (defined as number of villages) over
time and (B) number of backhauls per year under different assumptions about how often villages do backhauls (every
one to three years (upper black lines), one to five years (middle dark grey lines), and three to five years (lower light grey
lines)). Because a specific frequency per village was randomly assigned, we run and plotted these scenarios three times
to highlight possible variability, which is why there are replications for each set of grayscale lines in panel B. This
variability does not change the overall picture since the three runs all follow a similar pattern. While a simplification of
the program's dynamics (see details below), the number of backhauls occurring within a given year (B) are well below
the total number of community programs enrolled in Backhaul Alaska. The Program expansion rate (A) was derived
from Kawerak (2021, fig 1.2). Backhaul frequencies (B) are based on estimates provided by people during conversations
for this research project (some people place their next backhaul at even longer time periods, such as five to ten years),
and Zender Group (2017, table 5-4), which describes prioritizing backhaul in communities that have not had a backhaul
within three years. The rate of actual backhauls may differ and there are likely regional variations. The figure does not
incorporate a hub and spoke model that is being piloted in some regions, where waste from small communities is
transported and consolidated to a larger, central location for backhaul. The numbers of backhauls (B) were calculated
in the R computational language environment (R Core Team, 2022) using the "sample" function with replacements.
15
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
2.2 Critical Implementation Themes and Relevancy to App Development
2.2.1 Offline and Remote
"There are just obstacles here in Alaska that the rest of the country wouldn't understand or have to deal with.
... Between the weather and the remoteness and the logistics, it's daunting."
- Backhaul Program administrative staff member
Backhaul Alaska operates in remote areas of Alaska, creating a set of unique challenges for app development.
First and foremost, an app or technology solution must be able to work offline and where telecommunications
services are limited, with unreliable service or low bandwidth. RCs and regional experts that participated in
research activities for this project noted that community level staff prefer to use Zoom for video
communication calls because it requires less bandwidth than many other video conferencing platforms; in
some areas, many villages could not even use Zoom. As an extreme example, the community of Diomede
gets internet service only about one week per month. Indeed, the SC and several RCs were unable to upload
data in small communities when testing the Surveyl23 app. Consequently, waste inventories were stored on
their phones for several days until they returned to larger communities with better telecommunications and
could upload data. Even in communities with good internet connections, access may be limited to municipal
or tribal government offices.
The type and availability of computer hardware and software also varies among communities. It was not
uncommon for RCs, the SC, and regional experts to visit communities with limited, or outdated computer
equipment and software. Most village level workers did have personal cell phones, according to RCs that
participated in this project, which would allow use of a mobile app.
Backhaul Alaska participants also work in rugged conditions that challenge mobile app use. For example, and
as seen in the video footage, participants inventorying waste in the rain and snow often had to take off a
glove to use the Surveyl23 app. One village staff member described working in dusty, dirty conditions: "you
... take out your phone," they said, "dust is built up on it." Dropping and breaking phones is always a risk
when inventorying materials. Indeed, this village staff member crushed their phone, which was in their pants
pocket, when lifting a heavy object to pack for backhaul. Ruggedized devices and/or protective cell phone
cases could be appropriate hardware for Backhaul Alaska; adoption would depend on user preferences (see
section 3.0) and funding.
2.2.2 Communication and Coordination
"Comm unica tion, communica tion; it is just so key with this program."
- Backhaul Alaska administrative staff member
Communication and coordination (henceforth treated as a subset of communication) are central to how an
app can enhance Backhaul Alaska. Training staff to safely pack materials, sharing tricks of the trade from
more experienced participants, and organizing sequential backhaul events among villages to reduce shipping
16
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
costs, are all examples of communication and coordination needs. Table 2.1 describes five categories of
communication within Backhaul Alaska and how they relate to app development.
Table 2.1: Backhaul Alaska Communication Types and their app development implications. All communications need
to be resilient to availability of IT infrastructure and often may need to be performed in the absence of internet services
and uploaded to relevant servers once internet connectivity is re-established.
Type of
Communication
Description and Example
Implications for App Development
Training and
education
Training is a cornerstone of Backhaul Alaska. The
SC and RCs train village staff in the field at the
start of the season (Figure 2.2) but may need to
train new staff mid-season if there is staff
turnover. Other education opportunities might
include packing tutorials or cheat-sheets that all
staff (VCs and RCs) could access in the field as
refreshers.
Communication is primarily one-way (i.e., app
users access static information). RCs expressed
desires to have training checklists in the app.
Training and education materials could be
included as text, images, or videos in the app.
Material for download must account for low
bandwidths in villages.
Verification and
confirmation
Certified packing requires several checks that
materials are inventoried and packed correctly.
Program participants require notification that
corrections are needed. Backhaul Alaska also
needs confirmation that materials are received by
vendors.
Communications are primarily two-way (i.e.,
app users interact). Tasks assignments and
confirmations (with feedback and notes) could
facilitate packing and correction notifications.
Surveys or task assignments could confirm
receipt of materials.
Event dependent
Several communications might be triggered after
an event or time period. For example, certain
waste streams can only be held in storage for a
year, communities are restricted to shipping a
maximum of 16 pallets via barge, and VCs and
RCs may want to know when a shipping vessel is
within a given distance or transit time from a
village. Event-dependent communications might
also occur when prioritizing backhaul in
communities that have not backhauled for
several years, or if supplies, such as personal
protective equipment (PPE), are low.
Communication is primarily one-way, where
the app sends a message. An app might send
notifications when waste storage is
approaching a threshold, such as a time limit,
or maximum number of pallets, when vessels
are within specified distances or arrival
windows, or which communities have not been
served or need supplies.
Information
sharing and
coordination
Actors, both internal and external to Backhaul
Alaska, regularly share information about
program operations. This might include notifying
colleagues about waste and equipment
conditions or needs in villages that they visit and
coordinating specific program activities.
Communication is primarily two-way (i.e., app
users interact). Communications could occur
through an in-app chat with a communication
log (however, offline use of the app may
complicate chat usage). Informal
communications (email, phone, text) could be
documented after the fact in dedicated space
within the app. Features might also format and
export communication records to share with
other users (e.g., photos and description of
needed equipment repair).
Advice seeking/
giving
VCs and RCs often seek or share advice on issues
including best practices and how to address
challenging issues. New staff want to learn from
the experiences of seasoned practitioners and
need insider advice about working in new
locations.
Communications can be one- or two-way.
Users might access static help forums (with
regular updates). Message boards or question
and answer forums could host more dynamic
user exchange. Direct messaging and chat
could also facilitate advice seeking and sharing.
17
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Communication occurs among a network of individuals and entities involved with Backhaul Alaska (Figure
2.4). CT, SC, RCs, and VCs work within Backhaul Alaska and interact with external actors at the state and
regional level. Shippers transport waste to southern locations, where the waste is received by recycling
vendors. Backhaul Alaska also coordinates to receive materials from supply vendors. Participants share
records, photos, and information about waste and equipment needs with external federal, state, and private
sector partners who are involved with solid waste management. For example, during a site visit, the SC
documented transformers filled with polychlorinated biphenyls (PCBs), which Backhaul Alaska is not
equipped to deal with at this time. The transformers were documented, and the information sent to
appropriate parties to be managed externally to Backhaul Alaska. Another time, the SC was able to send
information about damaged equipment to repair personnel who were already working in the region on
another project. In a third example, several regional experts discussed that school groups, governing
councils, or other interested groups could help with backhaul inventorying and packing, after appropriate
training (Section 3.0).
A. Internal Backhaul Program B. External
Figure 2.4. Communication pathways among actors in Backhaul Alaska. Internal actors are shown in panel A and
external actors are in panel B. Solid black arrows indicate documented communication pathways and can account for
multiple kinds of formal and informal communications. Dashed black arrows illustrate possible communication
pathways, or those that might occur only in specific contexts, between various actors (circles) in Backhaul Alaska
implementation. The heavy solid blue line from RC and VC to Shippers represents informal communication when
identifying small local transport options, which are then communicated back to the CT through an RC. The red dashed
line) from Vendors to VC represents a desired communication. While the goal is that VCs receive feedback from
Vendors, communication could be through direct in-app communication, or conveyed through another pathway in the
network, such as CT to RC to VC.
18
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Some communication pathways within Backhaul Alaska are fixed. For example, the CT schedules and tracks
shipping and communication between the CT and local communities, a process that is always mediated
through RCs or the SC (Figs. 2.1 and 2.4). Other communications paths are less fixed. For example, the red
dashed line in Figure 2.4 represents a request by VCs that vendors provide feedback about the quality of
packing when they receive shipped materials. While the goal is that VCs receive feedback from vendors,
communication could be through direct in-app messaging or conveyed through another pathway in the
network, such as CT to RC to VC.
Formal communication paths, such as the CT coordinating with shippers, are often linked to informal
communications. For example, many small communities cannot be accessed by large barges, or are too
remote for access to be cost effective. In these cases, waste is transported by smaller local vessels, such as
fuel barges or fishing boats. With their on-the-ground knowledge, VCs and RCs learn about local shipping
options (heavy solid blue lines in Figure 2.4) and pass contact information to the CT, which organizes
transport.
Backhaul Alaska participants and regional experts stressed the importance of informal communication
among all parties. One regional expert noted that an informal conversation with local staff was useful for
assessing the level of their knowledge or expertise, implying that a conversation was better than a formal
checklist or training test, though it is important to note that checklists and training tests were frequently
cited as app needs by several RCs. Personal contacts were considered key to facilitate communications and
could even help in reaching out to other communities when RCs had trouble establishing local contacts. The
SC was frequently cited as a critical connection for communications and operations.
There are regional differences in communication patterns. Some VCs regularly reached out to RCs, while in
other cases, the RCs were usually the ones to initiate conversations. Two RCs reported very little communication
amongst villages in their regions, while other RCs noted more frequent communication. VCs and their staff
working in the three villages that we spoke with also provided examples of sharing or receiving information and
materials with other communities for backhaul and associated waste management activities.
There are pros and cons to having informal communication take place within a Backhaul Alaska app and
caution should be taken not to stifle natural exchanges. For example, Section 3.2.4 outlines a case where the
VC and VC Assistant often educate local citizens about waste management when citizens informally ask them
about backhaul waste pickups and drop-offs. Institutionalizing pickup/drop-off in an app might take away
this important opportunity to interact with the community. Several RCs said they envisioned informal
coordination and planning conversations to take place outside of an app; however, they liked the idea of then
documenting these conversations within the app. Program staff highlighted that it would be important to
access and query such a conversation log by location, named individual, and date (i.e., most recent). Other
suggested communication-related app content included checklists to document if communities need packing
materials, if staff are up to date on packing certifications, and the names and contacts of local VCs, their
assistants, and any other personnel. Several RCs indicated that a repository for "key contacts" in communities
(e.g., transportation options, tribal and city administrators, landfill operator, environmental coordinator, etc.)
and among vendors would make communications more efficient and effective. Should this content include
any personal information, data safety should be a continuing topic of discussion with app developers.
19
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
While many communications that take place during Backhaul Alaska's implementation are informal, the
packing and inventory stages are designed to follow a formal sequence of events with several checks and
balances. Even these processes, however, are dynamic and fluid, with different actors stepping in for each
other when there is limited capacity (Figure 2.5). As noted by one RC, "different communities have different
needs and different levels of help." While some communities were easily able to complete and send inventory
records, others lacked capacity and were unable to do so. In these cases, RCs worked with village staff to
pack materials and complete inventory records. Overall, VCs and RCs often worked together on many tasks
(Figure 2.5). Other observed modifications to standard protocols included the following: the CT requesting
document verification before sending packing information to RCs and LCs, or when the CT requested rapid
and direct email confirmation of packing corrections, which was considered "just easier, especially if you're
in a time crunch" (Figure 2.5).
Several communication steps in Backhaul Alaska would benefit from formal task assignment, verification of
completion, and rapid return of information. The request (outlined above) by the CT for rapid feedback is
just one example. Several RCs also described communicating with local staff through emails and text
messages to get confirmation that materials were packed properly. An app could streamline these
notifications and verifications through tasks tagged to specific users (Figure 2.6a). Location and task filters
could enhance the workflow and reduce possible errors, such as sending notices to the wrong party
(Figure 2.6b).
A. Process on Paper
B. Documented Process in Practice
£¦
O
vc
Submit packed J
waste Inventory I
—F"~
Correct
»
Load and
document
Submit to RC
for review
RC
SC
Notified of
1
corrections
Enter in database
>
Review/
revise
Review/
revise
CT
Review&
approve
No Yes
I Complete |
, shipping docs I
Coordinate
jshippiingS track I
fiC
SC
CT
fT\Oocument
Tconfirmation
3_
Shipping
docs
Q
Load and
document
Review/
revise
H
Complete
snipper^ docs
Review/
revise
Coordinate
shipping & track
Figure 2.5. Process flow diagram for the loading and shipping phase of the backhaul process as documented on paper
(panel A on left) and as observed in practice and documented during the research (panel B on right). Arrows indicate
process and workflow between actions, which are captured in text boxes. Each column encompasses actions taken by
responsible parties (VC, RC, SC, CT). Boxes spanning two columns indicate multiple parties. Circular arrows indicate an
interactive workflow between two parties.
20
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Notification and Verification Process
A.)
Event:
Correction or
confirmation
Action:
Notification
& comments
Attribute 1
Who to notify:
VC, RC, SC, CT
(multi-select)
Attribute 2
Verification needed by:
VC, RC, SC, CT
(multi-select)
B.)
Event:
Correction or
confirmation
r
Filter:
Task type or
backhaul stage
Action:
Notification
& comments
Attribute 1
Geographic
location(s)
Attribute 2
Who to notify:
VC, RC, SC, CT
(multi-select)
k. j
Attribute 3
\
Verification needed by:
r VC, RC, SC, CT "]
(multi-select)
--
Figure 2.6. Proposed general process for formal notification and verification of tasks in Backhaul Alaska.
A) Notifications are provided with comments and specific recipients and verification needs are assigned.
B) More advanced notices might include specifying recipients by location. Recipient and verifying party options could
be filtered by task or backhaul stage to reduce error of selecting the wrong actor.
2.2.3 Community Profiles
"[This community] is an old mining town. We've got all kinds of crazy stuff here. ... We've had variations of
military involvement. ... We've just got a lot of stuff laying around here." - Regional expert
All RCs and regional experts stressed that it was important to know about a local community. "Don't go in
blind/' they said. Specifically, it was important to have (1) a historical, social, and environmental profile, as
well as (2) an inventory of key people, places, and resources.
Historical context can provide insight about legacy wastes from old industry, military, or mining activities.
Information about current population and demographics can help practitioners understand what waste
types are currently being generated and their approximate volumes. Knowledge about local transportation
infrastructure is vital (e.g., if there is a deep-water dock, or if materials must be beach-loaded onto a boat).
It is also important to know what transport options exist (air, boat, or road), relevant shipping windows (e.g.,
subject to ice formation and break up, storm frequency, and shipping company schedules), and if there are
any sensitive environmental features or cultural places. Such information could be hosted statically in an
app, or perhaps made as a user generated and editable profile (akin to a wiki page).
RCs and regional experts also stressed that it was important to know about, and have a repository for, key
people, places, and resources (e.g., "key contacts" such as transportation contacts, tribal and city
21
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
administrators, landfill operator, environmental coordinator, etc.). An inventory of local supplies (e.g., PPE,
packing and labeling materials, storage containers/locations) was similarly imperative. One regional expert
also felt that having a sense of the age and background of local staff was helpful. The expert had observed
cases where younger staff were not always comfortable speaking with various local contacts, while in other
cases, an older experienced staff member was vital in communications and facilitation. While community
dynamics are likely to be complicated and case-by-case, such background information could help RCs. As
noted previously, before including any personal information in the app, Backhaul Alaska must work with app
developers to decide what personal information, if any, should be stored and shared within the app.
RCs and regional experts also requested a list or database of shipping companies and schedules per region.
One expert indicated that some shipping companies have specific stipulations. For example, they may only
transport materials in the company's shipping containers, which must then be available for packing. Some
companies may require that certain products are stripped of all rubber, plastic, oil, gas, and batteries before
being loaded.
Finally, knowing local names, the distances between local places, and how hard it is to get materials between
two locations could help RCs (refer to Section 2.4 for further discussion of mapping). The ability to search
information by local names (e.g., "the pit" is what one communicant called their waste staging area),
incorporate photographs of places, and view aerial photography were all seen as helpful features by RCs and
regional experts.
2.2.4 Capacity of Staff and Personnel
"I did a double site visit in [one village] and then turned around the next day and did an overnight trip to
[another village] because [the villages] just didn't have the human power to be able to package ... and do the
backhaul. ... People just didn't have the capacity to do it. ... And we lost the environmental coordinator [in
that village] who was... the lead. [Now there is] a new person that is being trained." - Regional Coordinator
Some of the most frequently mentioned challenges for completing backhauls were staff turnover, a lack of
qualified workers, and overtaxing existing staff. These concerns about staff and personnel capacity were more
common at the village level, but we also observed staff turnover among RCs and the CT. Staff turnover can
lead to the loss of institutional knowledge, delays in work, or decreases in efficiency while new staff get up
to speed.
An app could play a role in documenting institutional knowledge and smooth transitions. Many of the
features described under communication and community profiles (refer to Sections 3.2.2 and 3.2.3), such as
communication logs, question and answer forums, contact lists, and community infrastructure maps, might
help overall operational capacity. In-app training materials could also help RCs and Backhaul Alaska staff train
new workers.
Backhaul Alaska administrators felt that peer-to-peer in-app communications, to allow village staff to openly
discuss work and associated challenges, could help with staff retention. Features to rate and report job
satisfaction were also considered as another way to improve retention. These ideas were rooted in direct
observations during training programs that it was typical for job satisfaction related conversations to be
initiated by participants. We were unable to further test or explore these ideas in the current project. These
ideas and specific app features could be the subject of future work.
22
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
2.2.5 Equipment and Materials
"We're pretty fortunate to be tooled up and have had different opportunities to get tools ... That makes all
the difference. You ain't got a Sawzall, you ain't got a pair of clippers, ... makes it almost impossible. And a
lot of villages don "t." - Village Coordinator
Having the right equipment is essential for backhaul. PPE and packaging materials are critical. Storage space
for waste is also important. Tools to disassemble waste into component parts, to weigh materials, and to
load packed waste onto barges are similarly necessary. The types and quantities of tools and equipment that
each community has, however, varies greatly. For marine backhaul shipments, it was common in some
communities to transport backhaul materials to the beach/port in carts pulled by all-terrain vehicles (ATVs)
and then to pack the materials into shipping container at that location, because these communities lacked
loading equipment for large containers. Other communities had loaders that could lift and transport a filled
shipping container from the site of packing to the water. Most communities lacked a scale to weigh pallets
of waste, which RCs reported as problematic when working with the Surveyl23 pilot app, which required
forced completion of the data entry field for weight (this should inform future app development; see
Appendix A).
Several opportunities exist to address equipment and material needs through an app. While Backhaul Alaska
provides participating communities with PPE and some packing materials, these need to be regularly
inventoried, likely through a seasonal checklist that could be included as an app resource (refer to Figure
2.2). Equipment and infrastructure inventories (refer to Section 2.2.3) are also essential information. Several
RCs suggested that an app could include information and locations for purchasing equipment and supplies.
This may be helpful, especially for useful items not furnished by Backhaul Alaska.
Communication between communities (refer to Section 2.2.2) may also be an important part of the
equipment puzzle. Local and regional staff frequently highlighted the importance of learning from other
communities to develop creative solutions for equipment shortages. For example, one regional expert
described the use of old DOT dust suppression bags (which many communities have lying around) to pack
batteries, thus greatly reducing costs. Possibly because most live in remote communities without road access
to commercial stores (except a small grocery store), village staff are resourceful and creative at finding
workarounds to many problems. Whether shared through direct question and answer forums or as a
standalone section in which to report success stories and workarounds, an app could document and facilitate
knowledge exchange and idea sharing.
2.2.6 Opportunities for Synergies
"Many of these environmental staff and just people with jobs ... in the rural communities, wear multiple
hats." - Regional Coordinator
For most Backhaul Alaska participants, especially RCs, VCs, and their assistants, backhaul activities dovetail
with other activities. Among the village level participants that directly participated in this project, there was
often little separation between backhaul-related work (targeting specific types of hazardous waste) and
other waste management and recycling activities in which they were engaged. These participants often
shared information or sought advice from other communities on wider recycling and solid waste
23
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
management issues. Speaking to someone about non-Backhaul Alaska activities (e.g., other community
waste issues) might be an opening to learn about more relevant, effective, or safer backhaul practices.
Equipment and tools obtained from other programs and grants were often considered essential for doing
backhaul work. While it is likely beyond the scope of a Backhaul Alaska app to serve all related environmental
management efforts, certainly in the short-term, the communication features (outlined in Section 2.2.2)
should seek to embody the holistic way that participants think about waste management and environmental
programs in their communities, as opposed to thinking of backhaul as wholly separate. Backhaul Alaska
participants, including Backhaul Alaska administrators with decades of experience working in rural waste
management, live this holistic perspective of waste management and understand the end-health and
community benefits of backhaul can only be assessed within the larger waste management circumstances
of the community.
Beyond embodying this holistic perspective, there may be specific opportunities for a Backhaul Alaska app
to support other activities and needs. Regional experts and Backhaul Alaska administrative staff frequently
encountered materials or situations that they wanted to share with others in the wider waste management
community. For example, the SC said:
"A contractor ... was coming in to fix equipment in a community. I was able to share info about where the
equipment is, which saves the community money. [It's] a win-win for everyone."
Similarly, an RC helped administer a program that repaired old computers to give them to community
members. Any salvageable e-waste in Backhaul Alaska inventories could be repurposed along these lines. A
Backhaul Alaska app might support such activities through general communication pathways (refer to
Section 2.2.2) or more specific features designed for records export and sharing.
2.2.7 Liability and Safety Concerns
"We don't want to get involved with people getting themselves in trouble for trying to share something that
they didn't really understand." - Regional Expert
Several liability and safety concerns came up during our conversations; though participants repeatedly noted
that safety, especially related to packing and shipping, is a cornerstone of Backhaul Alaska. Concerns that
were discussed by people during research activities for this project fell into two categories: (1) concerns
related to Backhaul Alaska and how they might be addressed within the app and (2) specific app design
issues.
In terms of concerns related to Backhaul Alaska, one RC suggested that some communities might worry
about liability if they did something wrong; and this concern could, potentially, deter participation. The issue
is not whether liability is a real risk (Backhaul Alaska has liability insurance and training/certifying individuals
to properly pack waste is a cornerstone of the program). Rather, the fear of liability may be enough of a
deterrent, irrespective of real liability rules and training certifications. Another RC commented that staying
up to date on federal regulations was daunting and unrealistic as coordinating backhaul activities was not
their full-time job. In both cases, providing easy-to-access information about rules and liability (ideally that
clearly outlined immunity) in an app could go a long way to alleviating such concerns.
24
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Several worst-case scenario stories also came up during research activities for this project, including that of
finding old explosives at a backhaul event (Box 2.1), as well as a community having to pay when a shipper
picked up the wrong pallet. Clear guidance and information about who to contact in the event of these types
of scenarios (e.g., finding things that you are not equipped or authorized to handle, or what to do in the
event of a spill or environmental contamination) could be content within a Backhaul Alaska app. Of course,
app content should not replace existing emergency protocols, including but not limited to, an existing 24-
hour emergency service that Backhaul Alaska utilizes.
Among the people that participated in research activities for this project, no one voiced concerns about
health or safety while doing backhaul work, which may reflect the fact that Backhual Alaska focuses heavily
on safety training and provides PPE to communities. As mentioned in previous sections, communicating
safety protocols and ensuring that PPE inventories are appropriately stocked are important features of a
Backhaul Alaska app.
In terms of app design issues, one regional expert was
concerned that users might unintentionally disclose
incriminating information, such as regulatory or safety
violations. These violations might be seen in the
background of a photo, for example. One solution
would be an app feature to blur photo backgrounds;
however, unintentionally documenting a violation or
safety issues through photos could allow Backhaul
Alaska to learn about and correct something important.
Thus, app features that might protect the individual
might also hinder the wider safety and compliance
goals of Backhaul Alaska.
2.2.8 Data Sovereignty and Cultural
Considerations
"We survive ... it's hunting and gathering and that's
where a lot of the main source of food comes from. It's
expensive to get stuff here. ... All that stuff [subsistence
activities] is really important to us."
- Village Coordinator
The academic literature (Kouril et al. 2015, Ford et al.
2016, IARPC 2018, Kipp et al. 2019) and several experts
that we spoke with emphasized the importance of
understanding traditional knowledge and cultural
activities when working with Native communities.
During the RC workshop (Methods, Appendix B),
participants noted that the app could provide opportunities to enhance language revival - a goal of many of
the communities participating in Backhaul Alaska. While information in the app could be displayed in Native
Box 2,1. A Nightmare Scenario
"Hey, we got a bunch of stuff up at our school, in
the back room, that we just had over the years.
Can I bring it in [to be backhauled]?" This well-
intentioned request set in motion a day that
waste managers would never forget. {Note, this
was not a Backhaul Alaska event.)
Unaware that the school had once been used as
an army holding facility, the waste management
expert running the backhaul event invited the
school down. "Everything they brought in was
old military stuff, including dynamite," the waste
management expert recalls. "My legs were
shaking. I mean, literally!" he said. 'The guy
[brought it] in his truck, had this truck on the
state ferries."
With swift thinking, our expert had the truck
moved to an empty parking lot and isolated it.
Later they brought in a specialist from Seattle
that dealt with military waste. "We got it stored
in a building, segregated the building off, put
signs up: Do not enter, danger," our expert said.
"You run across stuff like that [working in
Alaska]," they concluded.
25
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
languages, it would be challenging to have information recorded that way since others involved with
reviewing the data and inventories would also need to be familiar with the language. Several people
(including VCs, RCs, and Backhaul Alaska administrators) with firsthand knowledge about communities
repeatedly confirmed that almost everyone working in Backhaul Alaska was comfortable using English
(Section 3.0). Capturing local terminology and units of measures (e.g., "a fish tote") could be an important
way to tailor the app for local users.
During the RC workshop, people also felt there were limited opportunities for a Backhaul Alaska app to
engage with traditional knowledge, partly because hazardous waste packing is not about environmental
observation or monitoring, and because waste management is subject to multiple state and federal
regulations. Program staff suggested, however, that it might be important and helpful to provide space in
the app for local staff to note if waste or waste storage was impacting species or areas of cultural significance.
Reporting local weather conditions, as well as ice arrival and breakup, could also help with coordinating
transportation. Planning for local site visits could similarly be enhanced by knowing more about important
traditional harvesting activities, and when local staff might be unavailable because they are hunting or
fishing.
Academic literature (Lewis 2020) and the technology experts that we spoke with also highlighted that data
sovereignty and privacy were often important issues in primarily rural Alaska Native communities. In general,
data sovereignty related to Backhaul Alaska did not raise many concerns among the VCs and RCs that we
spoke with, though several RCs highlighted the importance of clear user agreements if using third party apps
and data hosting sites. One regional expert noted that some tribal communities might have reservations
about databases managed by the federal government, but this is not likely to be the case for Backhaul Alaska.
Program administrators noted that community concerns over waivers or user agreements could arise on an
individual basis and that in their experience, it was best to deal with any specific issues on a case-by-case
basis.
During conversations for this project, at least one regional expert and VC brought up the idea of inventorying
caches of legacy waste located outside of the village. Some of this waste might be associated with old mining
camps or hunting and fishing activities and might include areas used for traditional hunting and gathering
activities. Inventorying waste at subsistence sites could become sensitive since GPS coordinates are recorded
during inventory, but hunters and fishers often want their harvesting spots to remain private. While Backhaul
Alaska administrators did not foresee inventorying waste outside of the village in the immediate future, they
are acutely aware of proprietary and cultural issues related to subsistence site location and purpose and
recognized the potential sensitivity. An app would need to take such sensitivity into consideration.
2.3 Data Management Needs
"I wish I had a place where I could put all my correspondences. Sometimes I'm not sure if I put it under
correspondence or... under a village." - Backhaul Alaska Program administrative staff member
Backhaul Alaska collects text and numerical data about waste inventories, equipment, and community
infrastructure. Photos and point locations (Global Positioning System (GPS) coordinates; Section 2.4) are also
collected. Data must be stored on a server that can be accessed remotely through a relational database.
Likely queries that RCs, SC, and CT discussed include but are not limited to, community name, waste type,
26
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
date, packaging status (i.e., complete vs. incomplete), training and equipment needs, and names of
individuals or companies when documenting correspondence (Appendix A: Surveyl23 user feedback, for
more specific examples and discussion about data use and management). Program staff were pleased by the
mapping and geospatial capabilities made possible with the use of the Surveyl23 app during the 2021 pilot
(Section 2.4 and Appendix A). Geospatial data collection and visualization are likely needed for future app
development, which might include community and public-facing dashboards (Section 3.0 and Section 4.0
outlining how different users might access information).
Data viewing will likely require some restrictions by user type and will depend on what information, if any,
is sensitive or private for a given community. Several RCs noted that some communities may not want their
waste inventories to be seen by other communities. According to one VC, waste inventories can certainly be
sensitive, though the culture seems to be changing: "In the beginning years," said the VC, "people were
embarrassed about the garbage. I don't think so much anymore. Now .... we are ... showing that we're
cleaning it up. We're proactive, ... doing the right thing, rather than ... ignoring it." RCs and program staff
also noted that sharing village waste inventories was important for regional coordination. Training records
and staff turnover were more likely to be seen as sensitive and more feasibly kept internal to community
viewing, although they still need to be viewed by RCs and the SC. Finally, communication records among
regional actors, including RCs, SC, CT, and possibly statewide partners (e.g., Federal and State; Figure 2.4),
may also require restricted access that would likely be managed by the SC (Table 2.2).
Table 2.2. Possible scenario of generation and access for records among Backhaul Program actors.
Type of Record
Example
Creation by
Accesses by
Village level, non-sensitive
Waste inventories
All Villages
RCs & SC
Regional Partners
All Villages
RCs, SC, & CT
Regional Partners*
Village level, possibly
sensitive
Training records and
staff turnover
All Villages
RCs & SC
Regional Partners
Village (only its own)
RCs, SC, & CT
Regional Partners*
Regional Information
Regional coordination
activities
RCs, SC, & CT
Regional Partners
RCs, SC, & CT
Regional Partners
* At the time of publishing, it was uncertain if regional partners needed to access these records.
As previously discussed, the number of individuals who need to be app users within Backhaul Alaska is likely
to be relatively small. Although Figure 2.3a shows a maximum capacity of 160 communities, we rounded this
up to 200 communities to be conservative in case Backhaul Alaska expands even more. At 200 communities,
the program would have about 600 to 650 app users. Each community will have on average between zero
and three licensed users (zero for those communities lacking staff capacity, in which case the RC will assist;
refer to sections 2.2.2 and 2.2.4). At the time of this project, Backhaul Alaska had seven RCs, though several
more may be recruited. Finally, the SC and CT require user licenses, with the possibility of several more
administrators in the future.
27
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Backhaul Alaska is also unlikely to need large amounts of data storage. For the 2021 pilot program, consisting
of 16 communities, Backhaul Alaska collected 50 - 100 megabytes (MB) of data using the Surveyl23 app.
While the program is expected to grow rapidly over the next decade (Figure 2.3a), the generation of data
records is unlikely to grow at the same pace because many communities only backhaul every few years
(Figure 2.3b). Furthermore, even using an app with more advanced features and content than Surveyl23
(e.g., training materials, communication, and task assignments), storage needs are unlikely to be a limiting
factor for the program. With terabytes of information able to be stored currently for just a few hundred
dollars, data storage costs are low.
Finally, several data quality assurances and quality controls (QAQCs) may be needed for a Backhaul Alaska
app. These QAQCs are specific to data management and differ from established safety, packing, and labeling
"readiness checks" currently used by Backhaul Alaska for waste inventory and packing operations (e.g.,
training certified packers and photo verification of packed waste). Ensuring that data records are uploaded
to the server is a key concern given remote offline working conditions (described in section 2.2.1). Two RCs
experienced problems with data uploads using the Surveyl23 app (detailed in Appendix A, list 2, item 12).
In both cases, the RCs assumed that the records had been uploaded upon reconnecting their phones with
data or Wi-Fi services, only to learn later that not all of them had been. In one case, the RC only discovered
the failed upload when they demonstrated use of the app for the EPA research team during the
interview/conversation. Incorporating an easy-to-read upload status in the app and perhaps a reminder
notification, could reduce upload errors from the field user's side. Notifying the SC and CT how many out of
their total number of records had been received could also alleviate problems at the administrative end.
Another QAQC concern, while not observed and only hypothetical, is ensuring that duplicate records are not
sent, and if they are sent, easily detecting them. Duplicate records could occur, for example, if state or
federal partners (outlined in Section 2.1) and RCs working in a village at different times were unaware of
each other's activities. While in-app communication records (Section 2.2.2) might reduce the potential for
uncoordinated activities, they may not be reliable enough to prevent every possible case of duplicate
records, which could have significant implications if duplicate inventory led to more expensive or
unnecessary shipping arrangements.
Ensuring that mobile devices are up to date with any app build or content changes may also require QAQC
measures. This is primarily a concern for village level staff, whose devices may be offline for long periods of
time and not able to receive app updates immediately (Section 2.2.1). One solution may be to record app
build versions and time stamps of most recent content updates among users. This information could then
be used to trigger notices to both the VC and SC, who could monitor and assist if a device, or group of users
(perhaps in a specific region) require updates.
28
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
2.4 Geospatial Needs
"A lot of times the waste is located offsite because [the local community] knows that it's potentially toxic,
hazardous, ... so they tend to place it somewhere hidden. [If] we can document it right then and there with
the GPS location, then we have a better idea for when that time comes ... to help them ship it out."
Backhaul Alaska benefits greatly from a geospatial data collection and
mapping. The 2021 piloting of the Surveyl23 app included features to
geolocate waste and waste management infrastructure and to
visualize these using a Geographic Information System (GIS). The RCs
and program administrators were very enthusiastic about this feature
as mapping helped them get a feel for local communities when they
were unable to visit them and help new staff learn. Mapping also
helped communicate and share information both within the program
and to external partners (for example, the SC's account of sharing
repair needs with external contractors, including location information,
in Section 2.2.6).
When using Surveyl23, Backhaul Alaska recorded all information as
points, which is only one type of data that can be collected and
mapped in a GIS (Box 2.2). When asked, several RCs said they saw little
benefit to using polygons, an alternative geospatial data structure,
which could be used to outline areas in detail, but require more effort
to record than points (Box. 2.2). The RCs were open, however, to using
polygons if there was a clear benefit. Several ways in which people thought polygons might be useful
included recording the shape of landfills or buildings in the general sites survey, or documenting fences
delineating landfills or other storage areas (a single length of fence would be a line, while a full enclosure
would be a polygon). There was unanimous feedback that point locations were sufficient and desirable for
locating pallets of materials for backhaul in the inventory survey.
Given the extra effort in the field to record polygons over points, app development should continue with
using points. However, since mapping was described so favorably by program participants, it may be
desirable to create more detailed community infrastructure maps for Backhaul Alaska using polygons for
waste storage and staging areas. Data could be collected in the field when there is capacity, recorded from
aerial images, or obtained from relevant administrative authorities to see if such GIS records already exist
for some or all communities. Refer to Appendix A for details.
Note, while geospatial data collection and mapping are important for Backhaul Alaska, there is currently
limited or no need for more advanced geospatial app functionality such as creating spatial buffers or
searching for sites based on proximity to another site. However, Backhaul Alaska's needs and interest could
change in the future.
Program Administrative Staff
Box 2.2. What is Geospatial Data?
In Geographic Information Systems
(GIS), data can be recorded as
points, lines, and polygons. Points
document a single x-y position and
are often used to record small
features in the landscape (e.g.,
storm drains). Lines connect two or
more points with a different start
and end position; lines are often
used to record features such as
rivers or roads. Polygons connect
three or more points to create a
closed shape; polygons are often
used to document building outlines
or water bodies. A
^or\
29
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
2.5 Section Summary
• Backhaul Alaska operates through a seasonal cycle of activities, each with unique needs from an app.
• An app must work offline when telecommunications services are limited and there is unreliable
service or low bandwidth. Devices may be offline for several days at a time.
• The app will be used where there is a risk of damage from water, dust, and crushing and should
accommodate users wearing gloves in cold weather or to protect hands from injury and dermal
exposure (e.g. lead) when packing.
• Several types of communication must take place within the app including training and education,
verification and confirmation, information sharing, and coordination. Communication may be one-
or two-way, informal or formal, and may depend on the occurrence of specific events.
• Documenting informal communications among people working at state and regional levels is
important. At the village level, however, caution should be taken to ensure that app-based
communication does not stifle opportunities for Backhaul Alaska participants to talk with and
educate citizens, which occurs when citizens call for waste pickup/drop off or other information.
• An app should contain information about individual villages and people to contact.
• Staff turnover, a lack of qualified workers, and over-taxing existing staff were frequently cited
challenges for completing backhaul work. Documenting communications, local information, and
training materials may help alleviate these challenges.
• Backhaul requires specific tools and equipment. An app can host inventory lists, notifications when
equipment needs restocking, and communication features to share equipment, backhaul best
practices, and creative solutions to equipment shortages.
• Many Backhaul Alaska participants have other jobs and responsibilities that dovetail with Backhaul
Alaska. Communication features should embody the holistic way that participants think about waste
management and environmental programs in their communities, as opposed to thinking of backhaul
activities as wholly separate.
• App features and content should consider, and ideally alleviate, any liability concerns that
participating communities may have, even if the concerns are only hypothetical.
• Future app development should be cognizant of and evaluate any tradeoffs among individual user
protections (such as blurred photo backgrounds) and opportunities to learn about and address
unintentionally documented safety issues or regulatory violations.
• The concept of using the app to help support communities' language revival goals by displaying
content in local languages was considered but because a primary Backhaul Alaska goal is uniform,
efficient, and cost effective backhaul management, English is the recommended language for app
content. Almost all current participants are comfortable with English and incorporating a variety of
local languages into an app could slow down development and make it expensive. However,
Backhaul Alaska could revisit this idea as the program evolves.
30
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
• There were few, if any, concerns over data sovereignty related to the information collected with a
Backhaul Alaska app, though specific villages may have concerns that are not documented in this
report.
• The app should provide opportunities for users to document any concerns about culturally
significant areas, local weather conditions to help coordinate waste transport, and any staff
unavailability due to subsistence hunting or fishing activities.
• Backhaul Alaska will require a relational database and there should be QAQC to ensure that records
and content are synced properly considering poor internet connectivity. Any duplicate records
should be easily flagged and corrected.
• Backhaul Alaska has modest data storage needs.
• Backhaul Alaska benefits greatly from geospatial capabilities.
31
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Section 3. Users of the Backhaul Program App
3.1 Introduction
Human-centered design is fundamentally concerned with understanding who will use an app (or other
technology), their concerns, and their activities (Teacher et al. 2013, Andrachuk et al. 2019, Siegler et al.
2021). While Section 2.0 fully embodies this perspective, it does so at a holistic programmatic level. In
Section 3.0, focus is on specific types of users by outlining user profiles for different user groups. The profiles
outline user's roles and responsibilities, working conditions, backgrounds, key challenges related to Backhaul
Alaska, and what they are likely to do with a Backhaul Alaska app. Importantly, the profiles also outline any
unknowns and unanswered questions for different use types and items that might warrant investigation for
future rounds of human-centered design research.
The user profiles are based on information collected through the various research activities for this project
and there are two data limitations worth noting. First, the village level participants that directly participated
in research activities for this project likely fall towards one end of the capacity and technology-infrastructure
spectrums (described in Section 2). For example, the VCs and village staff that participated in the
conversations had time to participate and were, therefore, likely to be less overburdened than people in
some other villages. They also had the digital resources to speak remotely, even if their internet bandwidth
was not great (for example, the conversations were punctuated with transmission delays such that we had
to turn off video at times to improve call quality). To account for this selective participation, we include
information about village level users and their needs from the expert opinions of RCs, program
administrators, and regional experts who have direct experience working with village level users in a variety
of contexts. Such expert opinions help document the conditions and needs found in the diversity of villages
that are served by Backhaul Alaska; however, there is always the potential that certain aspects about village
level users may be missing since direct participation for this project by villages level participants is by
necessity selective.
Second, the research activities for this project did not include direct participation from shippers and vendors
(receiving and supply), a decision we made due to restraints of research time and logistics. Data about
shippers and vendors are based on insights from RCs, program administrators, and regional experts who
work with shippers and vendors during their backhaul work. While an important source of information, these
past interactions where not directly focused on understanding shippers' and vendors' needs and interests in
relation to a Backhaul Alaska app. Additional future work may be needed to better understand the needs
and interests of shippers and vendors.
The following section, Section 3.2, provides a one-page profile for each user group: SC, CT, RCs, VCs, shippers,
and vendors, followed by documentation of several ideas gathered about additional potential users. Section
3.3 summarizes user needs into four general categories and indicates how to address these needs in design
and development of a Backhaul Alaska app.
3.2 User Profiles
Profiles start on next page.
32
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.1 The State Coordinator (SC)
Roles and Responsibilities
The SC oversees overall program implementation and
day-to-day operations. They plan for each year's
enrolment of new villages, train RCs, work with RCs to
train VCs and village staff, and support village backhauls.
The SC might be a single individual, or a lead and
assistant. The SC works with the CT to review and
approve waste packing and shipping. The SC was
frequently cited as a key resource and information
broker that both RCs and VCs would contact.
Working Conditions
Administrative work is done from an office using several
software tools, including Microsoft SharePoint, Access,
and EsriSurveyl23; the SC commented that collecting,
storing, and viewing all data within one software might
be easier and more efficient. The SC and assistants also
visit local villages to help train staff and assist with
inventory and packing (Figure 3.1). While visiting
villages, the SC and assistants might have limited access
to the internet.
User Background
The SC's role requires a high level of computer literacy
and ability to work with spreadsheets and relational
databases. The two individuals occupying the
SC/assistant roles both had extensive experience with
backhaul and waste management in rural Alaska. Their
institutional knowledge helped make them a key
resource and central figures.
Key Challenges
SC capacity may not scale linearly with the number of
added communities as Backhaul Alaska expands.
Program administrators added the SC assistant to help
carry out the SC's role. As the program expands to serve
more communities, the SC and assistant may not be able
to maintain the same level of close community support
without improved efficiency or additional assistants,
which might add to communication and coordination
challenges.
App Use
A SC would primarily use a Backhaul Alaska app for
inventory, verification, communication, and
coordination. They also view and manage data records
and app user licensing. SCs may generate and update
training and educational content for the app.
Unanswered Questions
As the program expands, Backhaul Alaska administrators
envision reducing the amount of on-the-ground
activities carried out by the SC and assistant(s) while
transferring some of this work to the RCs. The SC and
assistants will maintain some level of on-the-ground
work to ensure program continuity, maintenance of
their own field skills, and serving as RC backups. These
changes may alter existing communication and
coordination dynamics and app development should
remain attuned to such changes.
Photo credit: Backhaul Alaska
Figure 3.1. SC assisting with local village backhaul (A)
and tracking program training with a spreadsheet (B).
33
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.2 The Control Tower (CT)
Roles and Responsibilities
The CT coordinates shipping and receiving of waste at
final destinations and may work with supply vendors to
ensure that villages have certain equipment, such as
CONEX shipping containers to load waste. The CT works
closely with the SC and coordinates with the RCs for
waste transport from villages. The CT also does a fair
amount of logistical work with counterparts in Seattle,
WA once materials are offloaded by the shipper. This
includes arranging and overseeing storage of waste until
various materials can be taken by recyclers.
Working Conditions
The CT works from an office and does substantial
coordination by phone and email (phone calls are often
followed up with email for documentation). They
generate many records and communication logs that
need to be entered into a relational database. The CT
uses the same digital tools as the SC for waste tracking
and verification.
The CT monitors and tracks shipping vessels; existing
methods used include marine traffic automatic
identification system (AIS) tracking apps and software.
User Background
The CPs role requires a relatively high level of computer
literacy as they must work with spreadsheets and
relational databases (e.g., Figure 3.2). The two people
occupying the role of CT that participated in the
unstructured conversations varied in their self-reported
comfort with computers. One was quite comfortable
with tools such as the Surveyl23 app, while the other
felt less comfortable. In their own words: they "grew up
when computers were big in college and not in
elementary school." In part due to what they saw as
a generational shift in the ease of use of digital tools.
Key Challenges
We observed staff turnover for the role of CT.
App Use
The CT would work with the data management and
dashboard side of an app. They need to view and edit
records, as well as send and receive communications
to/from the SC, RCs, shippers, and vendors (shippers
and vendors may not be app users; see below).
Unanswered Questions
Backhaul Alaska is still refining the role of the CT and
who should fill it. There may be future changes to this
position. Specific to a Backhaul Alaska app, it is
unknown to what extent communications and records
management with shippers and vendors would occur
within versus outside the app.
~*' Outbound Transactions
¦a
Shipping Manifest
transporter AMI
Booking Number: kM*
ConUiner/T'ailer *: n/a
tracking Number: .tkdakMn
Vew I/Voyage: >
Ship Date: 10/t*/»ll
Consignee: Total Reclaim Alalia
Address: lilOHndu»l«y w«vtOiO
Seals: A
Pallet Count Gross Weight Tate Weight Ne
1 1.800 95
Weight
a,095
l.WM
8.070
sss
Photo credit: Backhaul Alaska
Figure 3.2. Database software (Microsoft Access)
used by the CT for creating shipping manifests.
34
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.3 Regional Coordinators (RCs)
Roles and Responsibilities
RCs work with and coordinate among individual VCs.
Primary tasks include training village staff (Figure 3.3),
ensuring they have necessary safety and packing
equipment, ensuring that village waste inventories are
documented and received by the SC, and working with
the CT to coordinate waste shipping. RC activities vary
according to the level of support needed by each village.
They work closely with VCs. An RC might do all waste
inventory and packing for some villages.
Working Conditions
The RC is a part-time position, usually carried out by
someone with a related job in the region, such as
working for a tribal health or environmental
organization. RCs wear multiple hats and make trips to
villages for related activities (e.g., health and
environmental monitoring), as well as backhaul. The RCs
that participated in research activities for this project
tended to use personal phones to collect information in
villages, including photos, which were not just for
backhaul activities. They preferred mobile phone
cameras over dedicated cameras because phones were
easier to use and produced higher quality photos. One
RC wished for a separate work phone to separate work
and personal content. Several others were comfortable
using personal phones.
RCs may be in remote villages for several days with
limited internet access. While assisting villages, they
may work in extreme weather (wet and cold) and may
be lifting heavy objects with the potential to damage a
phone or tablet. Several RCs thought positively about
being provided with ruggedized devices but questioned
who would pay for them (e.g., personal expense, local
tribes, Backhaul Alaska).
User Background
RCs are generally professionals with work experience in
related fields of health and environmental management.
Overall, the RCs were comfortable working with digital
technology and a variety of software tools, though one
person said they were uncomfortable with mobile apps
and digital technology. Several RCs said they would like
to have hard copy backups of the content in a Backhaul
Alaska app due to personal preference or a lack of trust
in technology. This was expressed by users at both high
and low ends of the self-identified digital literacy
spectrum. An app could allow easy selection and
formatting of content to be printed.
Key Challenges
Because backhaul is only a small percentage of their
work, staying up to date on best practices and
regulations can be daunting. Not necessarily specific to
their Backhaul Alaska work, RCs travel frequently in
difficult work environments and find themselves having
to work in villages about which they have limited, or no,
prior knowledge. Burnout is a concern and there can be
staff turnover. New RCs may have multiple steep
learning curves for their duties with Backhaul Alaska, in
addition to the other work their new position entails.
App Use
RCs would primarily use the app for inventory,
verification, communication, and training. RCs need to
view and manage data records. They may generate and
update training and educational content.
Unanswered Questions
RCs may take on more autonomy as the program grows.
It is unknown if this change would affect app use or
needs.
Photo credit: KodiakArea Native Association 2021
Figure 3.3. RC training village staff in an office.
35
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.4 Village Coordinators (VCs) and Supporting Staff
Roles and Responsibilities
VCs inventory and pack waste. They may employ one or
more local staff for assistance. VCs and their staff may
be community waste management leaders.
Working Conditions
Village conditions and resources vary (see section 2.0).
In general, villages have poor internet services which
may be limited to municipal/tribal buildings, while
waste is stored and processed in shipping containers
(e.g., CONEX) or other make-shift facilities that are
sometimes far away
"Our [waste] consolidation [area] turns to mud sometimes,
... No electricity; no heat. It's not really that fun to be outside
... in the winter ... when it's pitch black and 20 below. ... In
the springtime, when it's [milder] we... get pallets ready."
"When they told us ... the barge was [coming] ... the
[CONEX] that our e-waste was in was buried under like 12
feet of snow."
VCs and their staff often work in extreme weather (wet
and cold, Figure 3.4), lifting heavy objects with the
potential to damage a phone or tablet. One local laborer
described crushing a phone while lifting items for
backhaul. They liked the idea of a ruggedized device,
though preferred a tablet over a phone for the larger
screen size. A dedicated professional device would also
prevent mixing personal and professional photos, which
they were concerned about.
Many local communities have small, close-knit
populations. In one community, according to one
person, "everybody knows everybody." Community
members will call to arrange battery pickup or drop-off
and might reply to paper fliers or Facebook posts by the
VC. The VC and staff use such engagements as
opportunities to educate people about recycling and
proper waste disposal.
User Background
Village participants are likely less comfortable with
technology than RCs, but this is not a given.
Notwithstanding possible rare exceptions, village staff
are likely comfortable working in English. Many people
participate in subsistence activities (e.g., hunting or
fishing), and are absent for periods of time.
Key Challenges
Staff turnover and availability of qualified workers are
major challenges. Village-level staff may leave for
significant periods of time for subsistence activities.
Backhaul inventory work is often interrupted; VCs and
staff might not use an app for several months and can
forget how to log in or use an inventory app. The
seasonal pace of work also makes it hard to remember
what waste materials, packing supplies, and safety
equipment are on hand.
App Use
The VC and staff would primarily use the app for
inventory, communication, and receiving training, They
may want to view community and regional information
on a dashboard.
Unanswered Questions
RCs and regional experts provided insights about
community variability; however, this study likely does
not capture the full spectrum of community app users
because direct participation by village level staff in this
research was limited by logistics. Thus, village level
personal preferences (e.g., phones vs. tablets and
specific user preferences) may not be fully characterized
in this report.
_
M
I
i fcl
1
JX& x
I
Photo credit: Backhaul Alaska
Figure 3.4. Packing e-waste in 12 feet of snow.
36
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.5 Shippers (Small and Large)
Roles and Responsibilities
Shippers move packed waste from small communities to
larger regional hubs, or from hubs to southern ports
(generally Seattle). They may bring packaging, safety,
and other equipment into communities.
Working Conditions
Shippers transport materials in variable, often extreme
weather conditions. Shipping may be subject to
seasonal weather windows and delayed due to weather
or other extreme events. For example, the global Covid-
19 pandemic delayed some shipping.
User Background
Shipping is done primarily by sea and air but can include
overland transport for interior Alaska. Since the pilot
communities were mostly coastal (Figure 1.2), this
report primarily documents marine transport.
Large marine shipping companies serve the larger
communities, which may serve as regional waste
consolidation hubs, and transport materials to Seattle.
Large companies have established protocols for
shipping manifests, scheduling, and loading materials.
The CT is familiar with these protocols and
communicates with large shippers by email and phone.
Large shippers generally do not visit smaller
communities due to navigational or cost restrictions.
Smaller communities are served by small shippers,
which might include local marine or river transport
operators, fishing vessels, or fuel barges. Working with
smaller shippers is often on a case-by-case basis and
many opportunities to utilize small shippers are learned
about by RCs and VCs through word of mouth.
Key Challenges
Working with smaller shippers can be challenging
because transporting backhaul materials is often a
supplement to their primary work and subject to an
external schedule and set of objectives. Small shippers
may not have proper permits, insurance, or training to
carry certain materials, creating logistical obstacles.
Communication with small carriers is primarily through
cell phones; reception can be an issue. Many small
shippers are also involved with subsistence activities and
may be unavailable for long periods of time.
Credibility may also be a concern for some small
shippers. One VC described stories of so-called "bad
actors" who will take backhaul materials for a fee, and
then dump them off the coast. Whether real or
hypothetical, this concern highlights the value of
delivery verification.
App Use
It is uncertain how shippers might use a Backhaul Alaska
app. Some participants in this research project felt that
shippers would not be interested in an app because they
had their own well-established systems (e.g., for routing
and lading). Others thought that local shippers would be
interested in viewing transport opportunities so they
could schedule pickups to make extra money, somewhat
akin to a ride sharing app.
Unanswered Questions
There are many unknowns about shippers' needs and
interests in an app. Shippers' interests in a Backhaul
Alaska app remain speculative, and they may lack the
required internet services to engage with more
advanced app features such as communication,
verification, and feedback. Very High Frequency radio
(VHF) and satellite communications could play a role
with small marine shippers, but this is unknown to us.
Backhaul transportation can also encompass overland or
air components for some communities. Materials
management and safety issues would be similar;
however, there would be differences in scale,
seasonality, and timing from those locations that use
marine transport. Each transportation industry also has
unique regulatory, reporting, and communication
standards.
37
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.2.6 Vendors (focusing on receiving and
not supplying)
Roles and Responsibilities
Vendors include those receiving and supplying
materials. Conversations and other research activities
for this report tended to focus on vendors who receive
materials after shipment south, which remain the focus
here. While currently not a formal role of receiving
vendors, several VCs and RCs requested verification that
materials were received and feedback on the packing
quality.
Working Conditions
Backhaul Alaska works with various recycling and waste
management vendors in the Seattle, WA area. Different
vendors receive different waste streams. Once materials
are offloaded at destination (e.g., from shipping barges),
they are staged in holding facilities in the Seattle area
(e.g., Figure 3.5) until further transport of specific
materials occurs. The CT oversees this process and
coordinates with a colleague on the ground at holding
facilities.
User Background
Very little was documented, during this research
project, about this user group's background, other than
that they include professional recycling and waste
management companies.
Key Challenges
Timely transfer of materials to vendors is important. If
materials reside too long in holding facilities, fees are
charged to Backhaul Alaska.
App Use
Vendors would likely use an app to verify receipt of
shipments and to provide feedback on packing.
Unanswered Questions
A lot remains unknown regarding app needs and uses
for this user group, since we were unable to speak
directly with these potential users during our research.
Vendors likely have adequate internet and other
infrastructure to use an app at the jobsite. User needs
and interest in using a Backhaul Alaska app may largely
depend on company and individual interest.
The administrators of Backhaul Alaska, in planning for
program expansion, are also researching alternative or
additional shipping destinations, including locations in
the United States, Canada, and other international
markets. Key considerations include cost, logistics, and
ethical disposal practices. It is unknown how any
possible shipping changes may impact app
development.
Photo credit: Kawarak 2021
Figure 3.5. Properly packaged waste received in Seattle, WA
(source: Kawarak 2021).
38
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
" Oihor1 Kt"
Other Program Partners
This group includes federal, state, and nonprofit
employees working on similar waste management
issues in Alaska. This might include SWAT and members
of the Backhaul Advisory Council. Several of the regional
experts that participated in research activities for this
project include such individuals; they were enthusiastic
about using a Backhaul Alaska app in their work. These
partners work with many local villages and are aware of
waste management activities at local and statewide
levels. Their needs are similar to those of the RC: they
may need to inventory waste, communicate and
coordinate with the SC and RCs, and view dashboards of
Backhaul Alaska waste inventory and operations.
In general, these program partners are comfortable
using apps and digital technology. They may live in rural
hubs with more reliable internet, more up-to-date office
equipment, and may have support staff. Program
partners work in similar conditions to RCs and SCs and
face similar internet and technology access limitations
when in the field. Program partners may also want to
use a dashboard associated with the app to generate
and communicate Backhaul Alaska activities and
accomplishments.
Local Community Groups and Members
One regional expert suggested that local school groups,
council members, or individual community members
might assist with Backhaul Alaska activities and thus, use
the app. While Backhaul Alaska focuses on training and
certifying community packers and does not generally
work with volunteer or school groups, this person's
vision still must be documented. Community members
or school groups might engage in one time, or annual,
events - possibly with an educational objective - and
might only need a simplified version of an app limited to
basic inventory and educational information.
Any safety or liability issues related to having
community members, especially children, involved with
backhaul inventory and packing, would need to be
considered and addressed. Therefore, there are likely
some important unknowns about citizen and
community user groups that are not considered here.
There are also possible downsides to individual
community members using an app to inventory waste at
home, or to communicate about waste to VCs and staff.
Shifting to virtual communication might erode the
important informal communications that happen
between the VC, their assistant, and community
members as outlined in section 3.2.4. Alternatively, in-
app communication could create opportunities. Many
remote Native communities have increasingly used
social media as an important resource. Thus, the
implications of channeling community level waste
management interactions into an app should not be
assumed and warrant further study.
Resource Development Projects, Private Companies,
and Social Investment Corporations
Several regional experts highlighted the important role
of private sponsorship for some activities, which is
echoed in Backhaul Alaska planing documents (e.g.,
Zender Group, 2017). For example, backhaul funding,
barge availability, and supplies/materials are sometimes
offered as potential development opportunities in the
region. Private and corporate partners probably do not
need to use a Backhaul Alaska app, and there could be
potential conflicts of interest if some companies had
access to information while others did not. Local villages
may also have concerns about sharing information that
can be accessed by private companies and developers.
Nonetheless, future research should continue to track
what, if any, app needs exist for these groups.
Additionally, private sector businesses may participate,
in the future, with Backhaul Alaska in lifecycle
management of specific wastes in these rural
communities.
39
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
3.3 Summary of Generalized App Needs by User Type
While the exact features and user interface of a Backhaul Alaska app and potential supporting dashboard
will depend on the specific development path(s) chosen by the program (see section 4.0), the needs for
both realized and potential user types can be simplified into four general categories, which are presented
in Table 3.1.
Table 3.1. Generalized App needs for Backhaul Alaska user, by user type.
App needs
Users
Description
Full functionality
SC, CT, RCs, VCs, and
other partners
App needs include communication and verification, inventory,
accessing and/or updating training materials and tutorials, and
other information relevant for program function.
Simple or limited
functionality
Receiving vendors
Supply vendors
School and
community groups
App needs are more basic, limited to documenting information.
For receiving vendors, this might be as simple as sending a survey
form confirming receipt of shipments and rating or commenting on
packing quality. Less is known about supply vendors.
For school or citizen groups, users may have some inventory needs
and access to certain educational information, but they are unlikely
to require more advanced features such as communication and task
assignments
Dashboard or
website (one-
way information
sharing)
Shippers
SWAT and Backhaul
Advisory Council
App needs are likely limited to viewing information. This could be
accomplished through a web-hosted dashboard, or something
supported on a mobile device (see section 4.0).
Shippers may want to know what is available for shipping, its
location, and contact information. Should in-app communication
and verification be desired, shippers may need a more
comprehensive app configuration.
Entities such as SWAT, and the Backhaul Advisory Council, may wish
to view information on a web-based dashboard or website for
internal uses or to communicate Backhaul Alaska activities and
accomplishments. This might also provide options to communicate
to the general public who also may be interested to learn about
Backhaul Alaska activities.
Unknown
Resource
Development
Projects, Private
Companies, and
Social Investment
Corporations
It is unknown how these entities intersect with a Backhaul Alaska
app. Interest may increase in the future if the capacity of Backhaul
Alaska increases to include additional waste types or develops
private-public collaborations for lifecycle waste management.
40
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Section 4. App Development Options
4.1 Introduction
This section explores the suitability of several options for developing a Backhaul Alaska app. These include:
(1) developing a custom, all-encompassing app, (2) using third-party "off-the-shelf" apps, (3) using a
website-based system, and (4) combining a suite of apps ("off the shelf" or custom built) and a website.
Section 4.2 outlines these four options, analyzes them using a customizability and functionality vs. cost
framework, and considers how each option supports Backhaul Alaska's operations.
Section 4.3 details the testing of two different apps during the 2020 - 2021 pilot session. The pilot
experiences and their outcomes help illustrate an important component of a human-centered design
approach: time. Human-centered design, as embodied here, is often a long road, continually iterating
through phases of development, application, research, and reflection (Holeman and Kane 2019). The pilot
experiences illustrate important decision points, historical dependencies, and long-term thinking that
affects Backhaul Alaska's app development choices.
Finally, given the remote, offline environment and harsh weather conditions of rural Alaska, which may not
be kind to digital devices, we also consider whether a digital solution (e.g., an app or website) is even best
for village level implementation tasks. Section 4.4 challenges the assumption that a digital solution is best
and considers paper records as an alternative.
4.2 Development Options
While developing a customized app for Backhaul Alaska provided the original vision for this project, our
early research showed that building a full custom app should be considered alongside several other
development options. The technology experts and representatives from programs with similar experiences
that we spoke with (see methods appendix), as well as the academic literature (Teacher et al. 2013,
Andrachuk et al. 2019, Siegler et al. 2021), highlighted the high costs and challenges of developing a custom
app. They encouraged us to consider third-party or "off-the-shelf" apps and websites, if appropriate for
needs (Table 4.1).
These sentiments are in line with wider app development trends for community-based environmental
monitoring (a likely analog for Backhaul Alaska). A recent review by Andrachuk et al. (2019) documents an
increase in use of third-party apps, as opposed to developing new ones. This trend is likely driven by cost
saving measures, lower technology barriers (compared to computer programming for custom app
development), and management convenience (Andrachuk et al. 2019). Software updates are dealt with by
the hosting company; server space and a website to view and share information collected by the app are
often (but not always) provided, for a fee (Teacher et al. 2013, Andrachuk et al. 2019, Siegler et al. 2021).
The availability of convenient cloud data storage via third-party apps is noteworthy since data sovereignty
was not a major concern among Backhaul Alaska participants that participated in research activities for this
project (section 2.2.8). Finally, most third-party apps work with multiple operating systems such as Apple
and Android (Siegler et al. 2021). On the contrary, updates, server space, and dashboards for viewing data
all add substantial and ongoing costs to custom app development, which is already very costly.
41
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Table 4.1. Comparison of several different development options.
Single "All Inclusive" App
Third-Party, "Off-the-
shelf App
Website (and Web Apps)
Suite of Apps/Website
Pros
- Tailor made to meet
program needs.
- Possibly more control
over data ownership.
Pros
- Ready to use.
- Data storage and
viewing is likely provided.
- User support.
Pros
- Easy to update and
maintain.
- Inexpensive
- Highly customizable
within limitations
Pros
- Individual apps /
websites targeted for
specific functions.
- May offer cost savings
over a single app build
(depending on specific
suite of apps).
Cons
- Expensive to develop
and maintain.
- Harder/slower to
update.
Cons
- Ongoing subscription
costs.
-Third-party data storage.
- Not custom built.
Cons
- Online access
- (Web apps) reduced
user experience
(compared to operating
system specific apps (Box
4.1))
- Ongoing subscription
costs.
Cons
- Potentially complicated
or cumbersome.
- Multiple systems to
update and maintain.
While academic research on app costs is sparse (Andrachuk et al. 2019), Siegler et al. (2021) estimate that
developing a mobile health app with modest functionality costs $150,000 USD6; and Teacher et al. (2013)
describe a modest app that was developed for £43,000 GBP (about $65,000 USD in 2013)7. Such pricing is
in line with what the technology experts that we consulted with hypothesized a custom built Backhaul
Alaska app might cost. Importantly, neither of these prices include maintenance overtime, server space,
or other services that may be required (Figure 4.1). Indeed, Andrachuk et al. (2019:439) conclude that "the
costs of maintaining apps is highly underappreciated and underestimated (e.g., to refine functionality,
address bugs, and update software to match operating system and hardware updates)."
Custom development also involves a lot of uncertainty. Siegler et al. (2021) outline several notes of caution,
many of which were echoed by the technology experts with whom we spoke. Working with a trusted and
reputable developer is important. Computer code can be written in a way that makes it hard for a new
developer to work with should the client choose to change developers for future updates (perhaps because
the original developer goes out of business or increases prices for example). Certain coding choices can
lead to expensive bug fixes or other legacy effects that can be hard to change. Unplanned but desired app
features can lead to substantial cost increases and development time delays. To address these concerns,
6 Siegler et al. (2021) focus on developing mobile health apps, which may not be a perfect comparison to
environmental management. Mobile health app development can include additional costs to meet Federal privacy
laws, which may not be the case for environmental management. However, app development costs depend on many
factors and advanced features such as geospatial needs that may be central to environmental management may not
be needed for health applications. Notwithstanding some uncertainty, Siegler et al.'s (2021) price estimate is likely
comparable to Backhaul Alaska's needs.
7 Development costs are likely to differ by countries as they reflect developer salary rates.
42
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Siegler et al. (2021) advocate having an independent developer on the research or product management
team that can monitor and oversee the contracted development. The added cost of the additional
researchers is offset by reducing potential problems that would be costly to fix later.
>
-M
c
o
' Single app \
¦M
u
c
3
4—
i
\
\
\
custom 1 ~
build / Maintance,
y web hosting,
o3
>*
15
CD
N
E
o
Third Party
rv-r+o
/ \ Survey \ "other"
1 Web \ 123 Wjthout
data storage
and viewing
Different but
comparable
CO
D
U
\ Pa§e J reduced
v pricing
100 1,000 10,000
100,000
Cost ($)
Figure 4.1. Comparison of app development options considering customizability and functionality vs. costs.
The x-axis is a log scale, while the y-axis is unitless. Web page and single app custom build costs are estimates based
on reports in the academic literature and expert opinions and do not represent any kind of market research. Third-
party app costs (n = 2) represent actual price quotes and are presented in section 4.3. While web pages are highly
customizable and offer substantial functionality (Andrachuk et al. 2019), we have placed web pages low on the y-axis
because they cannot support Backhaul Alaska's offline needs.
Several technology experts that we spoke with also highlighted the benefits of using websites and web-
apps (Box 4.1). Websites are inexpensive and easy to update. Unless a highly customized site is desired, a
website can easily be built and maintained using any number of third-party subscription services. Websites
and web apps must be accessed online, which is a major limitation of their use for Backhaul Alaska (section
2.2.1). Mobile apps do exist, however, that allow single web pages, or an entire website (a collection of
pages under a unique URL) to be downloaded and stored on a mobile device. Using such apps could allow
Backhaul Alaska to host and easily update content such as training materials, checklists, and community
profiles on a website that users could download to their mobile devices. We did not, however, test or
investigate such options in detail. Therefore, the suitability of downloading websites for offline use requires
further investigation.
While there are many pros and cons to each development approach, the technology options are not
mutually exclusive. Several third-party apps, or apps and websites could be combined for a "suite of
apps/website" approach (Table 4.1). The suite could also involve developing a small app, with limited
functionality, which can be cheaper than producing an all-encompassing custom app (Teacher et al. 2013),
though likely still subject to ongoing maintenance, data storage, and server costs. A suite of apps, however,
may become cumbersome to manage and the total costs for several expensive third-party apps might
quickly add up in price to what an all-encompassing custom build might cost.
43
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Box 4.1. What is an App?
An app is a software program, designed to work within a specific operating environment (e.g., Android or
Apple iOS), that often interact with specific hardware (e.g., phone GPS sensors and cameras), other apps,
and external services such as remote data storage and websites (Jabangwe et al. 2018, Gerlitz et al. 2019,
Lai and Flensburg 2020). While an app may look like a stand-alone tool, it cannot be divorced from its
larger context. An app is built to work with, and is dependent on, a complex assemblage of supporting
hardware, software, inter-app communication choices made by its developers, as well as compliance with
the requirements of its underlying operating system (OS) and distribution environments. App
development and configuration also have important economic, socio-political, and data-security
implications (Gerlitz et al. 2019, Lai and Flensburg 2020) - apps cannot be separated from the larger
context in which they are developed and used.
Jabangwe et al. (2018) categorize three categories of apps: OS-specific apps, also called "native apps" are
designed to "live" on a mobile device with builds that are specific to a given operating system (e.g.,
Android or Apple iOS). OS-specific, or "native apps" maximize the benefits of a particular OS and generally
can work offline. Web-apps, alternatively, are developed to be accessed by internet browsers (requiring
nniing access), and are therefore, more independent of specific operating systems. Hybrid apps comhino
wo but are closer in function to web-apps (Jabangwe et al. 2018).
During the 2020 - 2021 Backhaul Alaska pilot, the program used the Esri software tool, Surveyl23, to
inventory waste and community infrastructure. Surveyl23 was adopted by Backhaul Alaska prior to the
start of this research project, to provide a digital inventory tool during one of the program's pilot efforts.
EPA had been working with Zender Group to identify technology-based solutions to help with backhaul
(e.g., shipping route optimization). Several EPA scientists (authors TB and PL) who were involved in those
conversations had been working with Surveyl23 for other projects and suggested that it could be used for
Backhaul Alaska's inventory needs. Backhaul Alaska administrators decided to try Surveyl23 and were
provided access to Esri online ArcGIS Surveyl23 through EPA's administrative license for research and
piloting purposes (see Appendix A for more details).
Surveyl23 was adopted to address an immediate applied need and was not necessarily intended to be a
long-term solution for the program. Yet as we learned more about the upfront and long-term costs and
administrative challenges of custom app development (section 4.2), third-party app solutions began to look
more promising, at least in the near term.
Surveyl23 was not the only third-party option tested. Over the course of our research, we identified and
evaluated other app-based data collection options. Subsequently, several conversations were held to learn
from a company that had experience working in remote Alaska communities and that was making digital
tools with great potential to meet many of Backhaul Alaska's app needs. In November and December 2021,
Backhaul Alaska administrators decided to pilot this company's app in several Alaska communities, with
very promising results.
4.3 Pilot App Case Studies
44
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Deciding which app would ultimately serve Backhaul Alaska's near-term needs, specifically, between
Surveyl23 or the "other" app, was not an easy choice. While both options met Backhaul Alaska's waste
inventory needs, the "other" option had several features that program administrators favored. The user
interface was streamlined and easy to use and the app was able to support communication and
coordination needs though in-app group chat that also could forward to personal email accounts. Program
administrators valued the opportunity to work with a company that understood the challenges of working
in rural Alaska. They also identified the long-term potential to integrate waste management services with
other infrastructure services (an all-in-one community tool), one of which this company was providing
already to many rural Alaska communities.
Backhaul Alaska, however, already had trained RCs and VCs to use Surveyl23 (summer 2021) and the next
backhaul season (summer 2022) was fast approaching with Surveyl23 ready to be used. Using a tried and
tested tool reduced uncertainty as the program started its first formal (post-pilot) implementation year,
bringing many new communities on board (see Figure 2.2). If Backhaul Alaska purchased its own Esri
administrative license, data migration from EPA's administrative license would be simple, and program
administrators valued the fact that EPA colleagues could help with this transition. Finally, the geospatial
capabilities and visualizations that came with Surveyl23 had become invaluable to the program and they
saw exciting options in Esri's wider suite of tools, such as Story Maps, an Esri product that would allow
communities participating in Backhaul Alaska, as well as program administrators themselves, to make
interactive, online map-based stories to communicate their experiences to wider audiences. Such story
mapping would fall under Backhaul Alaska's goals for local capacity building and help build wider financial
support for the program.
Cost was a big factor as well. The "other" option would cost around $20,000 per year to support the
program at full capacity. Licensing costs for Surveyl23 would only cost the program about $6,000 per year,
a price that reflected discounted pricing for non-profit organizations. Entities working with government
(e.g., local, state, or tribal) and university partners may also be able to benefit from licensing discounts. The
commercial licensing rate for Surveyl23, however, is on par with the cost for the "other" app option (Figure
4.1).
Surveyl23 was selected for the near-term because of the existing investment to train RCs and VCs to use
Survey 123, the app's geospatial capabilities and visualizations, and the financial savings, While Surveryl23
was chosen for near-term use, working with a company that understands rural Alaska remains important
to Backhaul Alaska and developing an integrated utility app that includes waste management is exciting to
Backhaul Alaska administrators. The "other" option is something that may be reconsidered in the future.
Backhaul Alaska's experience with these two pilot apps highlights several important considerations for app
development. First, path dependency - the idea that current options are constrained by past choices - can
be a significant factor. Backhaul Alaska had already trained RCs and VCs to use Surveyl23 and felt that it
was best to stay the course and work with an existing solution that Backhaul Alaska participants were happy
with. Using Surveyl23 was somewhat serendipitous; it was a practical solution to an immediate and applied
need for the Backhaul Alaska pilot. Other projects engaged in human-centered design, however, might seek
to delay adopting a specific technology solution until more research can be done about the range of
options. Testing several options from the outset might place each on more equal footing and reduce the
45
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
potential for path dependency. We recognize, however, adherence to this recommendation in the real
world might be difficult, just as it was in the case of Backhaul Alaska.
Second, Backhaul Alaska's experiences with these two apps reveals an evolving story, which is in line with
human-centered design: it is a process, continually iterating through phases of development, application,
research, and reflection (Holeman and Kane 2019). Research, development, and application are
intentionally blurred (Holeman and Kane 2019). Backhaul Alaska administrators emphasized that the
program might benefit from a different app solution in the future. While Surveyl23 provided an immediate
solution, it can also be a tool for continued learning. Indeed, part of Surveyl23's appeal was that it was a
low-cost option (both financially and in human capacity), with promising room for growth (e.g., combining
with other Esri apps) that the program could use while finding its stride. As the program grows over the
next ten years (adding 10-20 communities per year) it will continue to make app development choices that
meet evolving user needs in a dynamic technology landscape.
4.4 Future Development Options for Backhaul Alaska
While Backhaul Alaska is using Esri apps for its near-term needs (see section 4.3), there are many options
that the program might consider. As the name implies, Surveyl23 is first and foremost a survey to collect
information, which can include text, numerical, and geospatial data as well as photos. It is not designed to
support many of the communication and coordination needs outlined in section 2.0.
Backhaul Alaska could, however, work creatively with Surveyl23 to address many needs. For example, user
guidance and other documentation can be included as collapsible notes and images within the Surveyl23
form. Only static images are supported in the Surveyl23 mobile app (offline), while the online web app also
supports animated graphics (e.g., files in Graphics Interchange Format (GIF)). Supplemental information,
such as quick reference guides (i.e., cheat sheets) outlining shipping options and contact information, or
what to do with broken or damaged materials (e.g., broken batteries) can also be embedded in the form.
Static community profiles could be integrated in a similar way. Organizing such content with skip logic (a
feature that changes what question or page a respondent sees next based on how they answer the current
question) or conditional questions can help keep the forms streamlined, avoiding information overload.
Surveyl23 forms could be used to make checklists for documenting things such as PPE, training activities,
or packing materials inventories (see appendix A for more detail). A form could also be developed for
recycling vendors to provide feedback about the packing of received shipments (as requested in section
2.2). This form might be very simple, documenting a photo, star rating, and comments. Lastly, several Esri
apps support task assignments and confirmation, which could address certain two-way communication
needs outlined in section 2.2.
Other options might include shifting to a different third party "off the shelf" app at a future date, one that
provides desired features unavailable with Esri products. For example, at the time of writing, Esri apps did
not support text messaging or chat (personal communication with Esri staff), a feature considered highly
desirable by program administrators. Additional third-party apps could replace Surveyl23 or be used
alongside it, embodying the "suite of apps" approach (Table 4.1), which also might include websites or
small custom app development. Developing an all-encompassing Backhaul Alaska app also remains an
46
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
option, should the program decide such an app is needed. The data and analysis in this report provide an
important foundation to base any such app development on the needs and experiences of its users.
4.5 What About Paper?
Photo credit: Backhaul Alaska
Figure 4.2. The challenge of trying to do Backhaul inventory using paper, which is being blown around.
Given the harsh working conditions and offline environment of backhaul activities, it is tempting to question
if a digital solution is really better. What about good old-fashioned paper? Backhaul Alaska participants and
regional experts quickly quelled this concern, but also highlighted the important role that paper should play
as a vital information backup.
Even in some of the most remote communities with the worst digital infrastructure, cloud-based digital
records were often favored over paper because paper records could easily be lost, especially during staff
turnover. Though as noted in Section 3.0; this report may not document the full spectrum of local villages,
and there may be different opinions on the merits of paper records.
Additionally, printer cartridges are hard to dispose of properly in rural Alaska communities and are often
thrown in the trash or mailed back for recycling, which incurs additional costs. These factors can make
printing undesirable for some people. For example, one local expert said that when they ask a community
if they want a paper copy of a permit or other document, the response is usually that they want a digital
one.
Paper is also hard to work with during backhaul inventory. According to one VC support staff: "[before
Surveyl23], we relied on good old clipboard and paper... and it gets dusty down there and we're wearing
gloves ... and that paper would get dirty or stepped on, or dropped, and rained on." Indeed, the EPA
research team saw firsthand in the video analysis, how hard it was to record information on paper while it
was flapping in the wind (Figure 4.2).
The switch to a digital app over paper may also promote behavior changes conducive to better records
management. For example, one VCs assistant described what would happen when they needed to verify
information that had been recorded on paper:
"[the recycling] center, it's like three or four miles away [from the office, where there is an internet
connection]. ... If ... somebody has a question about [a] pallet [that was packed at the recycling
center] then we're going to sit here like, 'oh, man, where is our darn paper here?'... But now with
the [Surveyl23] app, it's easier because now we have the entry of what we have ready ... [We]
just open the app and simply look up what we got going on."
47
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
This account is interesting. First, it demonstrates that a digital app is favored over paper. However, the
digital device still needs to be brought back to the office to be synced over Wi-Fi or entered manually into
a computer (which the assistant had been doing because they were unaware that they could upload the
Surveyl23 form over Wi-Fi). It is still possible, however, to leave the phone or tablet at the recycling center,
putting the assistant back in the position of, where is our darn record? Presumably, however, using the app
forces them to take the record back to the office since the record cannot be completed until synced. The
technology may be creating a behavioral change to take the record with them. The records can also be
accessed from multiple locations, by phone or computer.
If paper is undesirable for inventory, what role might it play? Several RCs said they would want hard copy
backups of the community profile and training records content in a Backhaul Alaska app either because of
personal preference or a lack of trust in technology. Preferences for hard copy backups were expressed by
users at both high and low ends of the self-identified digital literacy spectrum. Hard copies or printouts
were also highlighted as an important way to provide training or educational information to communities
that might lack technology infrastructure or resources to use the app.
Paper resources were often stored in an iconic work binder used by many RCs and local staff. The exact
content of a binder might vary by community because it would contain local contacts, shipping routes and
logistics, and other context or lexicon specific issues. Other information might be more universal, such as
DOT packing regulations. One RC explained that it might be nice to have a binder template, with standard
materials included, plus placeholders to enter customized information. An app could help create these hard
copy backups by allowing easy selection and formatting of binder content and then a feature to send
material to print. (Note, functions to export material for printing must account for the often-poor digital
resources outlined in Section 2.2.1. Providing several export options such as USB in addition to Bluetooth,
Wi-Fi, and sharing to email, would be important as some communities may lack advanced technology like
Bluetooth or Wi-Fi connections.)
48
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
4.6 Section Summary
• There are several options for app development including: custom developed, third-party apps,
websites and web-apps, or working with a suite of multiple apps.
• Each development option has pros and cons and there is a general cost trade-off with
customizability and functionality.
• In general, third-party apps can address many user needs and there is a trend among community
based environmental monitoring programs to choose third-party apps over custom app
development.
• Custom app development is expensive, with long-term and uncertain costs for maintenance, data
storage, and data access.
• Website and web apps can provide many features at low costs but cannot work offline, which is a
problem for Backhaul Alaska.
• Backhaul Alaska had positive experience piloting two third party apps, both with offline capabilities.
While neither app addressed all desired features that the program wanted, they will continue using
one of these apps in the near-term. The app addresses their most pressing inventory and data
management needs for a reasonable price.
• Backhaul Alaska had used one pilot app longer than the other and had trained staff in its use, which
was an incentive to continue using this app and not have to retrain staff.
• Cost was another important consideration when comparing the two apps that were piloted.
• Backhaul Alaska administrators emphasized that the program might benefit from a different app
solution in the future, since the chosen pilot app does not fulfill every functionality need.
• Among users that we spoke with, the pilot app chosen was vastly favored over a paper-based
inventory system.
• Several users, however, said they would want hard copy backups of the community profile and
training records content in a Backhaul Alaska app, either because of personal preference or a lack
of trust in technology. An app could help create hard copy backups by allowing easy selection and
formatting of content to print. Multiple export options for digital files are needed (e.g., via Wi-Fi,
Bluetooth links, email, or USB connections) as computer resources and technology vary among
communities.
49
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Section 5. Conclusion and Recommendations
5.1 Report Summary
In this research project, we set out to understand: What are the best design options for a Backhaul Alaska
data management application (or similar technology) that addresses different users' needs for
supporting program operations and decision-making? We used human-centered design, combined with
several tenets of community-based research, to explore the needs, interests, and options of people
involved in Backhaul Alaska. We learned from multiple sources of information including unstructured
conversation and workshops with Backhaul Alaska participants and external experts, direct (albeit remote
for the EPA research team) observations of backhaul activities and pilot app use, and co-interpretation of
data among program administrators and the EPA research team. The data were analyzed qualitatively using
several thematic coding approaches. Results are meant to (1) support and improve Backhaul Alaska by
helping the program make decisions about app selection and development, (2) help build capacity in the
communities participating in Backhaul Alaska, (3) provide a foundation for app developers to understand
the program and potential app users, and (4) provide a case study that other programs and projects can
learn from.
Section two detailed Backhaul Alaska operations in relation to app development. It outlined the
experiences of the people doing backhaul that an app must support. The section identified and analyzed
eight important themes and several other critical components relative to app development (i.e., how the
program will grow overtime, data management needs, and geospatial needs). Section two is a blueprint
for developing a Backhaul Alaska app.
Several themes in section two touched upon app features or content with user safety implications. These
included: key contact lists and other information within community profiles (sections 2.2.2 and 2.2.3),
documenting institutional knowledge (section 2.2.4), information about liability and non-compliance issues
(section 2.2.7), data sovereignty and, though unlikely at this time, potentially documenting culturally
sensitive areas such as resource harvesting areas (section 2.2.8), and tiered data access as part of data
management protocols (section 2.3). Several people (but not all), that participated in research activities for
this project, also noted preferences for dedicated mobile work devices to avoid mixing-up personal and
work photos or contact information. In designing an app, safety for app users and Backhaul Alaska should
be a top concern.
Section three focused on specific app users, chronicling what they need from an app and how they will use
it. User profiles were developed for key users (SC, CT, RCs, VCs and staff, shippers, and vendors) and other
potential users were discussed. Critically important for app development, section three also outlines
existing knowledge gaps about users' needs and how these unknowns should be taken into consideration
when developing an app. Some things about users and their needs are unknown because of the limitations
of this study. For example, (1) the EPA research team was unable to speak directly with shippers and (2)
the report is not based on a statewide representative sample of local village staff. Unknowns and
uncertainties related to user groups may warrant more specific user-based research. Other items remain
unknown because Backhaul Alaska is continuing to evolve and learning how best to remove hazardous
household waste from rural Alaska communities, build local and regional capacities, support job creation,
50
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
and improve human well-being. Examples include possible changes to the role of the CT and planned
changes for more autonomous RCs. Therefore, app development must stay attuned and adapt to future
program changes.
Section four focused on the pros and cons of four app development options and how they suit Backhaul
Alaska. There are tradeoffs in functionality and customizability vs. cost, which program administrators need
to consider. Importantly, there is no one best solution. This project also documented Backhaul Alaska's
experience piloting two third party apps. This mini case study highlighted the importance of financial costs
when making decisions and how previous investments in learning to work with a specific tool can be an
obstacle to adapting something new. Finally, section four compared record collection with a digital app and
paper, verifying that digital technology was preferred. This observation contrasts with observations in the
Canadian Arctic, where paper has been preferred over digital apps for environmental monitoring due to
the remote, offline, and harsh environmental working conditions (Kipp et al. 2019).
5.2 Looking to the Future
Backhaul Alaska is a long-term program by design. It has a ten-year rollout from a pilot in 27 villages to a
statewide program covering all rural Alaska; and it will continue to operate thereafter. Rural Alaska is likely
to experience important environmental, social, and technological changes during this time (AEC 2021, Ford
et al. 2012, 2015, Kipp et al. 2019) that may intersect with app development needs.
Backhaul Alaska is investigating how to improve integration with a circular economy8 that might involve
private sector partners (e.g., manufacturers of performance clothing looking to make jackets from recycled
plastics). The program is also hoping to develop product stewardship relationships with manufacturers that
might ultimately help pay for a component of the local programs (Kawerak 2021, Zender Group 2017).
These private sector partners may want or require certain information on packing and inventory practices.
Future app development should stay aware of any needed changes and apps should be developed in way
so that they can accommodate circular economy and product stewardship needs.
Local environmental changes that are driven by human induced climate change (e.g., (Ford et al. 2015,
Debortoli et al. 2019)) may impact Backhaul Alaska operations with important implications about how an
app is used. Sections two and three document how seasonal weather and local conditions have profound
impacts on Backhaul Alaska activities. These impacts include, but are not limited to, shipping windows when
materials can be picked up for transport, the time(s) of year when local village staff are able to work outside
to inventory and backhaul materials, and working conditions including snowfall, rain, and groundcover.
Climate changes are also likely to impact when and where important subsistence resources can be hunted,
fished, and gathered (Green et al. 2021), which may alter the availability of local staff engaged in
subsistence activities. Finally, alterations to permafrost will impact local infrastructure such as roads,
runways, ports, riverbanks, landfills, and backhaul facilities (Hinzman et al. 2005, Reimchen et al. 2009, UAF
et al. 2019).
8 Circular economy is defined as an economy that reduces material use, redesigns materials, products, and services
to be less resource intensive, and recaptures "waste" as a resource to manufacture new materials and products
(https://www.epa.gov/recvclingstrategy/what-circular-economv).
51
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
The most immediate implication of these potential environmental changes for a Backhaul Alaska app is a
heightened role for documenting local and traditional knowledge for the benefit of participating
communities. Several participants in the various research activities carried out for this project noted some
limited opportunities for a Backhaul Alaska app to include features where local environmental observations
and subsistence hunting, fishing, and gathering schedules could be documented (section 2.2.8). Given the
potential for environmental changes to significantly impact several key dynamics that affect the timing of
backhaul activities and local village capacities, such observations should be given more attention in a
backhaul app. The ability to document observations under specific headings such as shipping, local
infrastructure, or working conditions could be one path forward. There might also be benefits or
opportunities to integrate a Backhaul Alaska app with other apps being used in the region for
environmental monitoring, such as the Local Environmental Observer (LEO) Network app
(https://www.leonetwork.org/en/mobile) to avoid replicated efforts.
As the climate changes, backhaul activities may also be impacted by changing Arctic shipping conditions
(Hildebrand et al. 2018), which might include changes in access to international shipping routes as well as
local shipping season and access changes. As outlined in section three, the research documented in this
report did not evaluate what role, if any, international shipping may play for Backhaul Alaska in the future.
Changes in seasonality and shipping routes, however, will likely impact the financial costs and logistics of
getting materials in and out of communities. While many of these considerations likely fall outside of the
design of specific app features and user experiences, they would directly affect the system in which the app
is used and warrant consideration.
A final important factor to consider is the changing technology landscape. Section two outlines the need
for an app solution that works offline, which constrains the suitability of options presented in section four.
While rural Alaska currently has extremely poor internet infrastructure, several groups are working to
improve internet access among northern communities (e.g., AEC 2021). The possibilities exist for a very
different technology future in rural Alaska that would enable vastly different app solutions for Backhaul
Alaska.
One way for Backhaul Alaska to consider and make sense of the aforementioned large-scale environmental,
socio-political, and technological changes would be through scenario planning. Scenario planning is a
visioning and modeling activity that considers several plausible futures under several intersecting dynamics
of change (Carpenter et al. 2006). Scenarios are not designed to be predictions; instead, they depict a range
of plausible futures that can be used to help prepare for the unknown and unexpected. Scenario planning
could help Backhaul Alaska consider which app development choices hold up against the largest range of
possible futures. Scenario planning might also contribute to long-term economic and other planning for
Backhaul Alaska and its participating communities.
52
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
5.3 Transferable Lessons Learned
There are several important lessons learned that are transferable to other programs with similar contexts
and needs.
First, when developing an app or any tool for a geographically dispersed, community-based environmental
management program such as Backhaul Alaska, start by focusing on what people need in the broadest
sense. We (the entire project team) started our research using an open and unstructured data gathering
approach to listen and learn from program participants and external experts. This helped to develop a
holistic understanding of needs for and interests in an app. Not everything we discussed was entirely
relevant to developing an app, but it allowed the EPA research team to gain a full understanding of the
context in which an app would be used and for the on-the-ground practitioners to gain new perspectives
about how their work translated into app development needs.
Second, tradeoffs between cost and functionality must be considered. The initial motivation for this
research project was to develop a custom, standalone app that met the unique needs of Backhaul Alaska
participants. Part of our research involved investigating the upfront and long-term maintenance costs of
custom app development. As large costs became apparent, custom development became less attractive.
Custom development is not necessarily off the table indefinitely, nor are any number of combinations of
app solutions. By assessing the cost vs. functionality tradeoffs, Backhaul Alaska avoided overextending itself
at a critical and early moment in its development, when it was not ready to take on the initial and ongoing
costs of custom app development. Rather, Backhaul Alaska needed to focus on developing a fast and
reliable solution for its most critical app needs.
Third, do not develop a tool based on an assumed problem. The initial idea for the project was to design
a custom app to meet the needs of Backhaul Alaska. A fundamental principle of human-centered design,
however, is to validate the problem statement through research. You may not end up where you thought
you would. As all members of the project continued to better understand the interests and priorities of
different users, and Backhaul Alaska's short- and long-term needs, we ended up downplaying the focus of
custom app development, instead prioritizing more affordable alternatives that met these needs. We chose
not to develop and test specific user interfaces, which would use valuable researcher and participant time
for something that might not be needed if Backhaul Alaska never builds a custom app. Rather, we continued
to build foundational knowledge about Backhaul Alaska's app development needs to inform any app
development options, from custom development to a suite of third party off the shelf apps. We did,
however, document important user interface improvements based on our evaluation of Surveyl23 (see
appendix A).
Fourth, it is possible to do a lot of work remotely. Because of the global Covid-19 pandemic, the EPA
research team was unable to travel to Alaska to do direct observations or user testing. The EPA research
team was able to speak with app users through video conferencing and at times have them demo an app
(Surveyl23), which provided important learning opportunities. As a collective project, we also asked
Backhaul Alaska participants to film short videos with their phones while doing backhaul activities and using
Surveyl23. These videos provided important learning opportunities, providing insights that may not have
been discovered otherwise. Because we did not want to overburden participants, and because the video
53
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
approach was experimental, we only asked for a handful of videos. Since these videos turned out to be very
useful, we recommend that researchers who are unable to perform site visits and user testing (due to travel
or costs restrictions) collect ample video footage of work being done in the field.
Fifth, researchers should commit fully to ensuring the best outcomes even if it is not part of the formal
research process. Human-centered design, or at least the approach adopted here (i.e., Bannon 2011,
Holeman and Kane 2019), takes time and commitment. EPA and Zender Group continued to work together
to optimize Backhaul Alaska implementation well after the research phase of the project was formally
closed. For example, the EPA research team helped support Backhaul Alaska administrators in discussions
with third party app companies, evaluate prices for different licensing options, and deliberate on a near
term app solution. As researchers familiar with the technology, the EPA research team was able to help
translate what concepts such as data management and feature options meant for Backhaul Alaska. There
is overlap here with sentiments voiced by researchers involved with translational science (Littell et al.
2017): some of the most important parts of working within a collaborative transdisciplinary project can be
in opposition to what research scientists are evaluated on, i.e., tangible research outputs. Building a
trusting and long-term working relationship among everyone involved, project partners and research
participants, is vitally important when designing from a human-centered perspective. Our project team
encourages other programs doing similar work to commit fully to developing these relationships, even if it
means pushing back deadlines, or engaging in extra tasks.
In conclusion, our project was enhanced by focusing first on what Backhaul Alaska and the people involved
needed from an app in the fullest sense, both in the near and long-term. Our understanding was informed
by repeated engagements with a variety of Backhaul Alaska participants and other people and programs
with connections to Backhaul Alaska, as well as by having regular internal team discussions that included
co-interpretation of data. Multiple engagements with key participants, such as the presentations and
workshops that we held, also likely helped build trusting relationships. Our successful teamwork, combined
with the fact that the chosen pilot app (section 4.0) is adequately and favorably supporting Backhaul Alaska
illustrates successful near-term outcomes of the project. We hope that: (1) this report can further support
Backhaul Alaska under any future app development paths; and (2) the report serves as a proof of concept
that provides insights for other programs interested in developing similar apps in remote and/or tribal
contexts.
54
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Sec
Abras, C., Maloney-Krichmar, D. & Preece, J. 2004. User-Centered Design. Bainbridge, W., Ed.,
Encyclopedia of Human-Computer Interaction, Sage Publications, Thousand Oaks, CA, 445-456.
Alaska Department of Fish and Game (ADFG). 2015. Alaska Wildlife Action Plan. ADFG. Juneau, Alaska.
236pp.
https://www.adfg.alaska.gov/static/species/wildlife action plan/2015 alaska wildlife action pi
an.pdf.
Alaska Department of Environmental Conservation (ADEC).2022. ADEC Regulations 18 AAC 60, Solid
Waste. State of Alaska. 122pp. https://dec.alaska.gov/media/1042/18-aac-60.pdf.
Andrachuk, M., M. Marschke, C. Hings, and D. Armitage. 2019. Smartphone technologies supporting
community-based environmental monitoring and implementation: a systematic scoping review.
Biological Conservation 237:430-442. doi.org/10.1016/i.biocon.2019.07.026.
Arctic Economic Council (AEC) 2021. Arctic Connectivity Working Group 2021.
https://arcticeconomiccouncil.com/wp-content/uploads/2021/05/aec-cwg-report-050721-6.pdf.
Backhaul Alaska. 2017. Backhaul Alaska overview brochure.
http://www.zendergroup.org/docs/trifold.pdf.
Backhaul Alaska 2023. Backhaul Alaska Pilot Program, https://backhaulalaska.org/pilot-program/
Bannon, L. 2011. Reimagining HCI: Toward a More Human-Centered Perspective. Interactions 4(18):S0-
57. doi.org/10.1145/1978822.1978833.
Bernard, R. H. 2006. Research methods in anthropology: qualitative and quantitative approaches. 4th Ed.
AltaMira Press. 824pp.
Barnes, D. L., Connor, B., Trost, B., McTigue, E., Krauss, K., and Bluehorse, B. 2020. Managing Alaska's
Road-Dust Problem: A Model for Road Dust-Impacted Regions. J Transp Eng A Syst. 2020 Apr;
146(4): 10.1061/JTEPBS.0000314. doi.org/10.1061/JTEPBS.000Q314.
Carpenter, S. R., Bennett, E. M., & Peterson, G. D. (2006). Scenarios for ecosystem services: an overview.
Ecology and Society, 11(1), 29. doi.org/10.5751/ES-0161Q-110129.
Chen, E., C. Leos, S. D. Kowitt, and K. E. Moracco. 2020. Enhancing Community-Based Participatory
Research Through Human-Centered Design Strategies. Health Promotion Practice 21(l):37-48.
doi.org/10.1177/1524839919850557.
Chipchase, J. 2017 The Field Study Handbook. Field Institute. ISBN: 978-1939727091. 505pp.
Daae, J. & Boks, C. 2015. A classification of user research methods for design for sustainable behavior. J
Clean Prod 106, 680-689. doi.org/10.1016/i.iclepro.2014.04.056.
Debortoli, N. S., D. G. Clark, J. D. Ford, J. S. Sayles, and E. P. Diaconescu. 2019. An integrative climate
change vulnerability index for Arctic aviation and marine transportation. Nature Communications
10(1):2596. doi.org/10.1038/s41467-019-10347-1.
55
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
EPA. 2021. Kawerak earns federal grant for hazardous waste 'backhaul' efforts in rural Alaska. EPA Region
10 News, June 16, 2021. https://www.epa.gov/newsreleases/kawerak-earns-federal-grant-
hazardous-waste-backhaul-efforts-rural-alaska.
Eisted R., and T.H. Christensen. 2013. Environmental assessment of waste management in Greenland:
current practice and potential future developments. Waste Manage Res 31:502-509.
doi.org/10.1177/0734242xl3482175.
Ford, J. D., K. Bolton, J. Shirley, T. Pearce, M. Tremblay, and M. Westlake. 2012. Mapping human
dimensions of climate change research in the Canadian Arctic. Amblo 41(8):808-22.
doi.org/10.1007/sl3280-012-0336-8.
Ford, J. D., G. McDowell, and T. Pearce. 2015. The adaptation challenge in the Arctic. Nature Climate
Change 5(12):1046-1053.
Ford, J. D., E. Stephenson, A. C. Willox, V. Edge, K. Farahbakhsh, C. Furgal, S. Harper, S. Chatwood, I.
Mauro, T. Pearce, S. Austin, A. Bunce, A. Bussalleu, J. Diaz, K. Finner, A. Gordon, C. Huet, K.
Kitching, M. Lardeau, G. McDowell, E. McDonald, L. Nakoneczny, and M. Sherman. 2016.
Community-based adaptation research in the Canadian Arctic. Wiley Interdisciplinary Reviews:
Climate Change 7(2):175-191. doi.org/10.10Q2/wcc.376
Gerlitz, C., A. Helmond, D. B. Nieborg, and F. N. van der Vlist. 2019. Apps and Infrastructures - a Research
Agenda. Computational Culture (7). http://computationalculture.net/apps-and-infrastructures-a-
research-agenda/.
Green, K., Beaudreau, A., Lukin, M., & Crowder, L. (2021). Climate change stressors and social-ecological
factors mediating access to subsistence resources in Arctic Alaska. Ecology and Society, 26(4): 15.
doi.org/10.5751/ES-12783-260415.
Harte, R., L. Glynn, A. Rodrfguez-Molinero, P. M. Baker, T. Scharf, L. R. Quinlan, and G. OLaighin. 2017. A
Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User
Experience of Connected Health Systems: A Three-Phase Methodology. JMIR Human Factors
4(l):e8. doi.org/10.2196/humanfactors.5443.
Hildebrand, L. P., L. W. Brigham, and T. M. Johansson, editors. 2018. Sustainable Shipping in a Changing
Arctic. Springer. 486pp. doi.org/10.1007/978-3-319-78425-0.
Hinzman, L. D, N.D. Bettez, et al. 2005. Evidence and Implications of Recent Climate Change in Northern
Alaska and Other Arctic Regions. Climatic Change 72, 251-298. doi.org/10.1007/sl0584-005-
5352-2.
Holeman, I., and D. Kane. 2019. Human-centered design for global health equity. Information Technology
for Development 26(3):l-29. doi.org/10.1080/02681102.2019.1667289.
IARPC. 2018. Principles for Conducting Research in the Arctic U.S. Interagency Arctic Research Policy
Committee (IARPC). 4pp.
https://www.iarpccollaborations.org/uploads/cms/documents/principles for conducting resear
ch in the arctic final 2018.pdf
56
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Jabangwe, R., H. Edison, and A. N. Due. 2018. Software engineering process models for mobile app
development: A systematic literature review. Journal of Systems and Software 145:98-111.
doi.org/10.1016/i.iss.2018.08.028.
Jasny, L., J.S. Sayles, M. Hamilton, L. Roldan Gomez, D. Jacobs, C. Prell, P. Matous, E. Schiffer, A. Guererro
Gonzalez, M. Barnes. 2021. Participant engagement in environmentally focused social network
research. Social Networks, 66: 125-138. doi.org/10.1016/i.socnet.2021.01.005
Kawerak. 2021. Proposal for the Rural Alaska Household Hazardous Waste Backhaul Service Program
Funding Opportunity. Kawerak, Inc., Internal document. 36pp.
Keske C.M.H., M. Mills, T. Godfrey, L. Tanguay, and J. Dicker. 2018. Waste management in remote rural
communities across the Canadian north: challenges and opportunities. Detritus 2:63.
doi. org/10.31025/2611-4135/2018.13641.
Kipp, A., A. Cunsolo, D. Gillis, A. Sawatzky, and S. L. Harper. 2019. The need for community-led, integrated
and innovative monitoring programmes when responding to the health impacts of climate
change. International Journal of Circumpolar Health 78(2):1517581.
doi.org/10.1080/22423982.2018.1517581.
Kouril, D., C. Furgal, and T. Whillans. 2015. Trends and key elements in community-based monitoring: a
systematic review of the literature with an emphasis on Arctic and Subarctic regions.
Environmental Reviews 13(November 2015):1-13. doi.org/10.1139/er-2015-0041.
Lai, S. S., and S. Flensburg. 2020. A proxy for privacy uncovering the surveillance ecology of mobile apps.
Big Data & Society 7(2):2053951720942543. doi.org/10.1177/2053951720942543.
Lewis, J. E., editor. 2020. December 1. Indigenous Protocol and Artificial Intelligence Position Paper.
Honolulu, Hawai'i: The Initiative for Indigenous Futures and the Canadian Institute for Advanced
Research (CIFAR). 205pp. doi.org/10.11573/spectrum.librarv.concordia.ca.00986506.online at
https://spectrum.library.concordia.ca/id/eprint/986506/7/lndigenous Protocol and Al 2020.pdf
Littell, J. S., A. J. Terando, and T. L. Morelli. 2017. Balancing research and service to decision makers.
Frontiers in Ecology and the Environment 15(10):598-598. doi.org/10.1002/fee. 1739.
R Core Team, 2022. R: A language and environment for statistical computing. R Foundation for Statistical
Computing, Vienna, Austria. https://www.R-project.org.
Reed, M. S., L. C. Stringer, I. Fazey, A. C. Evely, and J. H. J. Kruijsen. 2014. Five principles for the practice of
knowledge exchange in environmental management. Journal of Environmental Management
146:337—345. doi.org/10.1016/i.ienvman.2014.07.021.
Reed, M. S., S. Vella, E. Challies, J. de Vente, L. Frewer, D. Hohenwallner-Ries, T. Huber, R. K. Neumann, E.
A. Oughton, J. S. del Ceno, and H. van Delden. 2018. A theory of participation: what makes
stakeholder and public engagement in environmental management work? Restoration Ecology
26:S7-S17. https://doi.org/10.llll/rec.12541.
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Reimchen, D., G. Dore, D. Fortier, B. Stanley, and R. Walsh, 2009: Cost and constructability of permafrost
test sections along the Alaska Highway, Yukon. Proceedings, Transportation Association of
Canada Annual Conference, 20 pp.
Saldana, J. 2013. The Coding Manual for Qualitative Researchers, Second Edition. Thousand Oaks, CA:
Sage Publishing. 329pp. online at https://emotrab.ufba.br/wp-
content/uploads/2020/09/Saldana-2013-TheCodingManualforQualitativeResearchers.pdf
Samuelson G.M. 1998. Water and waste management issues in the Canadian arctic: Iqaluit, Baffin Island.
Can Water Resour J (4) 23:327-338. doi.org/10.4296/cwri2304327.
Siegler, A. J., J. Knox, J. A. Bauermeister, J. Golinkoff, L. Hightow-Weidman, and H. Scott. 2021. Mobile app
development in health research: pitfalls and solutions. mHealth 7(0):32-32.
doi.org/10.21037/mhealth-19-263.
Skarlatidou, A., A. Hamilton, M. Vitos, and M. Haklay. 2019. What do volunteers want from citizen science
technologies? A systematic literature review and best practice guidelines. Journal of Science
Communication 18(01): A02. doi.org/10.22323/2.18010202.
Solid Waste Alaska Taskforce (SWAT) 2017. Solid Waste Alaska Taskforce Guide to Solid Waste
Management in Alaska. 72pp. https://anthc.org/wp-content/uploads/2015/12/Solid-Waste-
Guide- Feb2017.pdf.
Solid Waste Alaska Taskforce (SWAT) 2023. About Solid Waste Alaska Taskforce (website).
https://907swat.org/about/. accessed February 2023.
Teacher, A. G. F., D. J. Griffiths, D. J. Hodgson, and R. Inger. 2013. Smartphones in ecology and evolution:
a guide for the apprehensive. Ecology and Evolution 3(16):5268-5278. doi.org/10.10Q2/ece3.888.
University of Alaska Fairbanks Institute of Northern Engineering (UAF), U.S. Army Corps of Engineers
Alaska District, and U.S. Army Corps of Engineers Cold Regions Research and Engineering
Laboratory. 2019. Statewide threat assessment: identification of threats from erosion, flooding,
and thawing permafrost in remote Alaska communities. Report prepared for the Denali
Commission, report #INE 19.03 Denali Commission. 199pp. https://www.denali.gov/wp-
content/uploads/2019/ll/Statewide-Threat-Assessment-Final-Report-20-November-2019.pdf
Zender Group. 2017. Sustainable Statewide Backhaul Program Draft Plan. SWAT, 106pp.
http://zendergroup.org/docs/sustainable statewide backhaul draft.pdf
Zender Group. 2022. Backhaul Alaska Pilot Program Status Report, March 2022. Internal document.
Backhaul Alaska,13pp.
58
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Appendix A. Survey 123 User Feedback
A.l Objective
This appendix outlines user feedback about Esri's Surveyl23 app for Backhaul Alaska. Surveyl23 is an off-
the-shelf web and mobile app that enables customized survey forms to collect information, including
photos and geospatial information. References to "Surveyl23" in this appendix include the user generated
form and the application's user interface, database, and workflow. Where relevant, the Surveyl23 form
(i.e., the survey designed to collect information for Backhaul Alaska) is distinguished from the Esri
software/app.
User feedback about Surveyl23 and its use in Backhaul Alaska was solicited as part of larger discussions
about the program's app development needs (see methods section A.3.0, below). While feedback about
Surveyl23 is not mutually exclusive from broader app needs (discussed in the main report), this appendix
attempts to distinguish items specific to using Surveyl23 from the larger conversation of app development.
For example, during research for this project, Backhaul Alaska participants requested that vendors send
feedback about the quality of packing when the vendors received shipments. While this feedback can easily
be supported by Surveyl23 (e.g., by developing a rapid, standalone Surveyl23 form for vendor feedback,
which could include a picture, star rating, and comments field), the desire for vendor feedback is not
specific to Surveyl23. It is a broader app development issue that might be addressed in different ways for
different app development approaches. Other desired app features are beyond the scope of Surveyl23,
for example hosting video tutorials or detailed training manuals, as Surveyl23 is not designed to
communicate such content. Therefore, such issues are not covered in this appendix. Rather, this appendix
focuses on issues specific to Surveyl23.
While specific to Surveyl23, many of the documented favorable features and desired improvements are
informative across platforms. For example, observing that the "save" and "send" buttons in Surveyl23
confuse some users (see below), which might benefit from more descriptive names, can help improve
Surveyl23 and is relevant for other apps. While this appendix documents feedback specific to Surveyl23,
the information is relevant beyond Surveyl23.
This appendix might be used to:
• Improve Surveyl23 (and/or other Esri products) for continued use by Backhaul Alaska.
• Inform Backhaul Alaska app development regardless of chosen platform or software.
This appendix is organized as follows:
• Section A.2.0 provides background about Backhaul Alaska's use of Surveyl23.
• Section A.3.0. briefly outlines methods and data sources for collecting user feedback.
• Section A.4.0 details positive and negative user feedback for Surveyl23.
• Section A.5.0. outlines possible expansions to Surveyl23 that advance Backhaul program.
59
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
A.2 Background
Prior to the September 2020 start of human-centered design project that is the focus of this report, EPA
had been working with Zender Group to identify a number of technology-based solutions to help with
backhaul (e.g., shipping route optimization). Several EPA scientists involved in those conversations had
been using Surveyl23 for other projects and suggested that it could be customized and used for Backhaul
Alaska's inventory needs. Surveyl23 forms were developed for the program quickly thereafter, through
conversations between EPA scientists and Zender Group. Surveyl23 was adopted to address an immediate
applied need and was not explicitly intended as a prototype test for a human-centered design study about
Backhaul Alaska app needs. Surveyl23, however, provided an important opportunity to learn about the
use of a digital app to inventory waste in Backhaul Alaska.
A3 Data and Methods
To understand the strengths and weaknesses of Surveyl23, the EPA research team sought user feedback
during unstructured conversations with Backhaul Alaska participants (four local coordinators (LCs), two
regional coordinators (RCs), and three staff fulfilling state coordinator (SC) and control tower (CT) roles), a
three day workshop-style discussion with staff from Zender Group, during which preliminary project
findings were discussed, and two meetings with Esri program management and Zender staff to discuss the
use of Esri products beyond the backhaul pilot (see materials and methods in Appendix B).
Participants provided feedback about Surveyl23, including their
likes and dislikes. Because conversations were unstructured and
covered additional topics beyond Surveyl23, participants were
able to discuss Surveyl23 in response to broader discussion topics
and prompts (e.g., digital tools they used for their job, or when
describing their workflow). RCs and the SC were also asked to take
short videos of themselves and others using Surveyl23 in the field
in order to directly observe and learn about its use. During these
trips to local communities, RCs and the SC were also training some
village level staff how to use Surveyl23 therefore providing an
opportunity for the EPA research team to observe some short
periods of training (Figure A.l and Figure A.2). All conversations
and videos were qualitatively coded as outlined in Appendix B.
The discussions around Surveyl23 prioritized understanding what
needed to be improved and thus, tended to document needed
improvements rather than strengths of the tool. This uneven
feedback is not a quantitative measure of pros vs. cons and simply
reflects the nature of the conversations.
This appendix is written from the perspective and voice of the EPA
research team, which analyzed the data.
Photo credit: Backhaul Alaska
Figure A.l. Backhaul Alaska staff
using Surveyl23 in the field to
inventory waste located in a shipping
container.
Photo credit: Backhaul Alaska
Figure A.2. Local village staff
completing a survey on the app.
60
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
A.4 Participant Feedback
A.4.1 Surveyl23, Positive Sentiments
Backhaul Alaska Program participants were generally positive about using Surveyl23. Several participants
said that Surveyl23 was preferable to the old method of using paper inventory forms, which were entered
into a spreadsheet or photographed and shared as an image over email. Notwithstanding some confusion
about how to install and login to Surveyl23 (see below), most users felt that the app was easy to use. One
person described Surveyl23 as "pretty straightforward." Another said, "Surveyl23 [is] very easy ...
everything is in order [and] comes in steps. [The backhaul program] makes sure that steps are completed.
And if there's something missing, ... someone looks it over and says, we need [it] like this. So, I like how
[it's] made." This comment not only illustrates a positive experience using Surveyl23, but also an
appreciation for the quality controls and feedback that Backhaul Alaska used.
In addition to the survey form itself, program participants appreciated Surveyl23's geospatial capabilities.
RCs and Backhaul Alaska administrators praised the ability to view data on a map. Mapping helped them
get a feel for local communities when they were unable to visit them, was seen as a useful way to
communicate or share information and was considered an essential tool to help new staff get up and
running when working in the field. Statements such as the following were common:
"[A new] coordinator comes in and they're like, 'we don't know anything about this waste.' ... A
lot of times the waste is located offsite because [the local community] knows that it's potentially
toxic, hazardous, ... so they tend to place it somewhere hidden. [With Surveyl23] we can
document it right then and there with the GPS location. ... We're ready to help."
- Program Administrative Staff
"An infrastructure map is a good way to put it, even for folks that are not part of our program.
For example, I knew a contractor that was coming in to fix equipment in a community. I was able
to share info about where equipment is, which saves the community money. So, it is a win-win
for everyone. - Program Administrative
Staff
Additional favored or praised features that participants mentioned included the following:
• Improvement over paper (as discussed above)
• GPS and mapping capabilities (as discussed above)
• Copy and paste feature to help repetitive inventory (such as multiple boxes of light bulbs).
• Saving a favorite profile to auto populate records for a common waste type.
• Satellite base map when taking GPS points.
• On the database side, benefits of filtering and searching items by community.
• In the videos, we observed that the auto-population feature and dropdown menus seem to be
adopted by users and helped minimize the time required to complete a survey.
61
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
A.4.2 Surveyl23 Improvements
While people were generally positive about Surveyl23, they also reported a number of possible
improvements, which are outlined below. The order of items does not indicate importance. In the following
reporting, a description of the type of user is provided to help illustrate user needs, the data source is given
(conversations or videos), and often the context in which users experienced problems or undesirable issues
is also provided. Several items in the discussion of potential improvements (Lists A.l, A.2 and A.3) are
preceded by italic text that presents Analyst's notes that document suggestions or qualifications for app
development solutions for the given issues.
List A.l. Training Materials, Software Access, and Related Issues
• Video tutorial and training, in a small file size: According to a RC, local staff were not using the
Surveyl23 training materials. The RC recommended creating a video on how to use the tool for
those people that cannot attend in person training, or when there is staff turnover in the middle of
a season. The RC recommended use of multiple kinds of training materials to accommodate
different learning styles. The RC also noted that video content should respect local bandwidth
issues; e.g., downloading large files in communities could be a limitation. Similarly, several Village
Coordinators (VCs) and/or VC assistants commented that they did not always remember how to
access or use the app, since they only used the app intermittently, often going weeks or months
without using it. This episodic use pattern highlighted the need for improved training and refresher
content.
• Promote use of phone app over desktop app: According to a RC, local staff were using the
Surveyl23 desktop version, which does not work as well as the phone version. The RC felt that local
staff were used to working on the computer and did not realize that a phone app was available.
The RC observed that using the phone app allowed people to take photos as they completed the
survey, however, when they took photos with their phone and went back to the office to complete
Surveyl23 on the computer, they could not always remember which photos were from a particular
location in the community since some communities had multiple storage locations. While one
person working at the village level said they preferred using Surveyl23 on the computer because
of the larger screen space and the ability to use a computer mouse, the participant noted that they
liked the idea of using Surveyl23 on a tablet because it would be more comfortable to use than a
phone. Furthermore, the tablet would allow them to enter all their data while at the waste storage
location and later syncing it over the office Wi-Fi (the participant did not have Wi-Fi or data access
when outside of the office).
• Assign dedicated usernames and passwords: Several village level and regional participants had
trouble logging in to Surveyl23 or could not remember their usernames and passwords. Backhaul
Alaska had assigned generic usernames and passwords to participants, which they found hard to
remember. In some cases, VCs and their assistants were sharing usernames and passwords, which
they found problematic. Analyst's note: Individual usernames, perhaps the user's name, and self-
chosen passwords, would likely facilitate access, as would dedicated assistance for login issues.
62
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
List A.2. Form and User Interface Improvements
• Tooltips: Several people expressed a desire for some type of tool tip feature to provide in-app
instructions. Analyst's note: Recent builds of Surveyl23 Connect allow for short pop-up style tools
tips that are limited to a few lines of text. Other workarounds include putting tool tips or guidance
as collapsible notes or images within the form. (Only static images are supported in the Survey 123
field app (offline), while the online web app also supports animated GIFs.)
• Multiple favorite profiles: One person wished for the ability to save multiple favorite profiles, e.g.,
one for e-waste, batteries, lamps, etc., in order to auto-populate fields and then take individual
pictures of the waste.
• Allow for multiple photos: Several people (both at the village and regional level) said they would
like to be able to add multiple photos in a survey. Multiple photos seemed to be more important
for the general sites survey. Several regional staff said a single photo of the pallet was sufficient for
packaged materials. The videos showed regional staff taking multiple photos when doing general
inventory, often one close-up photo and one from afar, corroborating the aforementioned
expressed desires.
• Annotating photos: One person wished to annotate photos, for example, commenting or labelling
to indicate that some pallets in the photo had been finished and others needed to be completed
later. Analyst's note: While photo annotation may not be possible in Surveyl23, it may be possible
to include a comments field.
• Copy and paste photos between surveys: One person wished to be able to copy photos from a
general site survey to a shipping survey so that new photos did not need to be taken. Analyst's
note: Copying photos may save time but it may be a good idea to force new photos for shipping to
ensure the photo is as up to date as possible.
• Separate "General Materials Survey" into two surveys: Several people suggested creating
separate surveys for 1) sites where materials were stored (e.g., the landfill), and 2) the actual
materials to be inventoried. User observations confirm the benefit of separating these surveys: In
one video, a local staff member expressed confusion about answering waste inventory questions
in a survey that was meant to document waste management facilities.
• Improve default map location when opening survey: The video analysis showed that a local staff
member had trouble determining their location on the survey map as the map appeared to open
in Los Angeles. A RC confirmed this could be a stumbling block during conversations. Analyst's note:
Auto locating the user seems desirable, though for cases where the user lacks internet access, it
may be sufficient to have the map default to Alaska.
• Improve how surveys upload, possibly with notification: Two people expressed concern with how
surveys were uploaded. In demonstrating the use of Surveyl23 during a conversation, one person
was surprised to find several surveys in their outboxthat had not been sent. This person thought
they would automatically upload when they got back onto a Wi-Fi connection, and if this was an
issue of user settings, the participant was unclear about the settings. Another person explained a
problem they had where they thought they had uploaded all their surveys, only to learn from the
63
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
SC that not all had been uploaded. They speculated that an internet interruption caused this error
and wished for an easy to see notification that surveys were sent. Analyst's note: The ability to
change how surveys upload may be limited in Surveyl23, but can we think creatively about some
way to incorporate notifications or feedback?
• Easy viewing of unique IDs: One person described how there was insufficient time to write down
information that was briefly displayed. They explained that a unique identification number (ID) is
generated when creating a new record in Surveyl23, and this number must be written on the pallet
of waste. The unique ID only appears on the screen for a short period of time, which is not long
enough to write down. The IDs can be viewed by going into the outbox, but a way to improve
unique ID retrieval would be desirable.
• Remove forced completion of weights field: Two people commented that the survey form
required them to fill in weights, but they did not know the weights and could only estimate them.
Similar confusion and dissatisfaction among users was also documented in the video analysis.
Analyst's note: There may be several workarounds depending on the chosen field type for the survey
question. If using a numeric field, a tool tip could prompt participants to enter "-999." Switching to
a text field could allow for "unknown." The simplest solution may be to allow non-responses.
• Allow for responses of "not relevant": A RC said that some entries in Surveyl23 do not apply to all
loads, and they told local staff to just "click yes" for everything. Certain fields in the survey would
likely benefit from an option to enter 'Not Applicable (N/A)" or "Does not apply.".
• Address photo quality and storage issues: An RC noted that photos taken with the in-app camera
on Surveyl23 were of lower quality than those taken with their own camera app. The RC also had
trouble with photo storage. They were unable to find where photos taken using the camera
function within the Surveyl23 app were stored on the phone, or if they were stored at all. This
confusion created problems later when the RC wished to access photos for further evaluation of
the packed waste. Analyst's note: If these issues cannot be addressed by changing Surveyl23
settings or performance, these issues may be cataloged under List A.l as training issues and
guidance on photo capture and storage can be provided.
• Allow surveys to be edited: A program administrative staff member noted that surveys could not
be edited, and they would like to be able to edit them. Analyst's note: I believe the participant was
referring to saving a survey and then reopening it to edit, but I am not sure. When improving
Surveyl23, the developer should verify if this is the case.
• Address "check-mark error" when completing surveys: The video analysis showed that users make
a common error of accidentally pressing a "check mark" that appears at the top of the keyboard or
number pad when completing an individual field within a survey. Pressing the check mark prompts
the dialog box that contains options for completing the survey. This check box was repeatedly
pressed as a means to complete the specific field, rather than being used only to indicate readiness
to complete the survey. Analyst's note: I am unsure if this checkbox can be edited in Surveyl23 but
if so, this ambiguity could be removed by replacing the check with a more specific icon/button.
64
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
• Address ambiguity over survey completion actions: The video analysis showed that some users
were confused about the survey completion buttons, and a RC also expressed a similar concern
during conversations. When users are ready to complete the survey in Surveyl23, they are given
three options related to saving, sending, and continuing the survey. In the video, local staff were
confused by the difference between saving and sending and received instructions that if they did
not have internet service, they should save the survey, rather than send it. Analyst's note: The
ability to edit these buttons in Surveyl23 may be limited but, if possible, more descriptive names
should be used. Otherwise, this issue might be addressed through tool tips, explanatory text, or
educational materials. If development occurs outside of Survey 123 and custom button names are
allowed, descriptive names should be used.
List A.3.: Database and Data Management Improvements
• Allow program administrative staff to flag and ask for resubmission of form: A program
administrative staff member wished for a straightforward way to flag when something needed to
be corrected in a submitted inventory and a simple way to get verification of the correction.
Analyst's note: Such workflow may be possible using Esri Workforce; Section A.5.0. outlining
possible expansions.
• Improve reactivity between Surveyl23 and SharePoint: The current workflow uses Power
Automate to automatically update records in SharePoint once a Surveyl23 form is uploaded.
However, if Backhaul Alaska administrative staff edit the survey record in Surveyl23, the update is
not made in SharePoint and they have to manually edit SharePoint as well. Analyst's note: The
solution likely has to do with adjusting the Power Automate web-hook function.
• Improve control for program administrators: Backhaul Alaska administrators commented that
they had little control over many backend aspects of Surveyl23 because they were using EPA
licenses. They did not have full control over developer view and did not have permission to change
certain things. The staff also noted that in dashboard mode, they could only edit records that they
had created and would like to see what was possible if they had full administrative rights.
• Desktop version runs slowly: Backhaul Alaska administrators noted that the desktop version of
Surveyl23 ran slowly and that the phone app version was much faster. They hoped the desktop
version could be made to run smoother.
• Batch download all of a community's photos: One Backhaul Alaska administrators noted that on
the 'analyze' pages, they could filter by community and see all pictures for that community.
Pictures, however, could only be downloaded individually, which was undesirable. They would like
to download all the photos at once.
• Edit an entire map in map view: One Backhaul Alaska administrators said that when working in
map view, they could edit data in "individual map view," but they could not edit data when the
entire map was up, which would be nice to be able to do.
• Enable surveys to sync across a user's devices: One Backhaul Alaska administrators said that they
use both their phone and tablet to complete surveys and would like their completed surveys to
sync across both devices.
65
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
A.5 Possible Expansions
While this appendix is primarily meant to document user feedback about Surveyl23, it is worth outlining
several possible expansions of Surveyl23 that may address Backhaul Alaska app needs. This section is brief
and focuses on additional capabilities of Surveyl23 and other Esri products (since all Esri apps and products
can be combined, to a certain degree), to avoid redundancy with the discussion of broader app
development outlined in the main report. (Note: this is not a promotion of Esri products; refer to disclaimer
on first page of report).
Points vs. Polygons for Mapping: Surveyl23, as part of the Esri suite of apps and tools, is built around
geospatial data and mapping. In geographic information systems (GIS), data can be recorded as points, lines,
and polygons. Points document a single x-y position and are often used to record small features in the
landscape (e.g., storm drains). Lines connect two or more points with a different start and end position and
are often used to record features such as rivers or roads. Polygons connect three or more points to create
a closed shape; polygons are often used to document contiguous land cover types, building outlines, or
waterbodies. The Backhaul Alaska pilot Surveyl23 app currently records all features using points. The EPA
research team discussed possible benefits of recording information using polygons with several RCs and
Backhaul Alaska program staff.
These individuals saw little benefit to using polygons but were open to it if there was a clear advantage.
Several people commented that they could see some benefit to recording the shape of landfills or buildings
in the general sites survey as polygons. One person also commented that documenting fences around
landfills or other storage areas might be useful (a single length offence would be a line, while a fill enclosure
would be a polygon). There was unanimous feedback that point locations were sufficient and desirable for
locating pallets in the inventory survey.
Given the extra effort needed in the field to record polygons over points, Backhaul Alaska should probably
continue using points in the Surveyl23 forms. However, since mapping was described so favorably (Section
A.4.I.), Backhaul Alaska might, as a secondary task, seek to create polygons of waste storage and staging to
generate more detailed infrastructure maps of communities. GPS points could be collected in the field for
a given village if staff had time and capacity. Polygons could also be created by tracing features on satellite
images (a process called "digitizing') in a GIS program like Esri's ArcGIS online (there are also free open-
source alternatives that could be used). Of course, before undertaking this work, Backhaul Alaska staff
should consult with relevant administrative authorities (Alaska state, federal, or Indian Environmental
General Assistance Program) to see if such GIS records already exist for some or all communities. If GIS
records exist (or have been created by Backhaul Alaska), they can be easily integrated into Esri dashboards
and apps.
Creatively adding content and other solutions to Surveyl23: Creative uses of existing Surveyl23
capabilities could help address some of the desired app features and functions that Backhaul Alaska
participants described. There is limited capacity to include pop-up style tool tips in Surveyl23 (analyst's
note under "tool tips" in list A.2 within Section A.4.2). Tool tips and other guidance documentation can also
be included as collapsible notes or images within the form. (Only static images are supported in the
Surveyl23 field app (offline), while the online web app also supports animated GIFs.)
66
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Supplemental information embedded in the form could also include items such as cheat sheets outlining
shipping options and contact information, or what to do with broken or damaged materials (e.g., broken
batteries). Skip logic or conditional questions can keep the forms streamlined, avoiding information
overload. For example, the form might include a question: "Do you need additional information about
packing, dealing with damaged goods, or shipping options?" Replying "no" would simply move to the next
part of the inventory form, while a "yes" might open a multiple-choice question to display the needed
guidance (e.g., broken batteries, shipping information). To avoid burdening administrative staff with having
to update information, content could be limited to relatively static information (e.g., packing broken
batteries) or information that is updated at regular intervals (e.g., DOT transport guidelines). Content and
access should be user tested to ensure it helps users and does not complicate their workflow.
Surveyl23 forms could be used to host a variety of checklists for documenting things such as PPE, training
activities, and packing materials inventories. All of these items and app features were requested by various
Backhaul Alaska participants during conversations (in main report). Developing such content in Surveyl23
could be a low-cost, low-tech way to integrate and document checklists within the currently used
technology.
Finally, small surveys could be built to address other Backhaul Alaska user needs. For example, while not
specific to Surveyl23, several people requested that vendors send feedback about the quality of packing
upon receiving materials. This feedback could be recorded in a small standalone Surveyl23 form that
includes a picture, star rating, and comments field that the vendor could install on their phone to send
feedback.
Using Other Compatible Esri Products: The data recorded by Surveyl23 can readily be used with other Esri
tools that may address some additional needs or desires of Backhaul Alaska. The people spoken to for this
project liked ArcGIS Online's mapping capabilities (Section A.4.1). Other options might include using Esri
Story Maps to present information. Backhaul Alaska could even invest in training community members to
use tools such as ArcGIS Online and Story Maps, enabling communities to use their data and tell their
own stories. Such activities align with the program's goal of local capacity building and job creation.
Another expansion might be to integrate Surveyl23 with other Esri apps, such as Field Maps or Workforce.
These apps include features that allow administrative staff to assign tasks and for field workers to confirm
completion, and provide feedback that can include notes, photos, videos, and audio. There is a lot of
potential for these apps to fulfill some of the communication and confirmation needs that Backhaul Alaska
staff and participants requested. If these kinds of features are attractive to Backhaul Alaska, there are a few
ways to access them. Surveyl23, Workforce, and Field Maps can exist as side-by-side applications that use
the same underlying data. Access to Surveyl23 forms can also be embedded in Workforce and prompted
by a Workforce task assignment. Backhaul Alaska may also consider transitioning to Field Maps, which
integrates many task assignment and tracking features with data collection forms. Since these apps are all
part of the Esri tool suite, anyone familiar with Surveyl23 should be able to transition to one of the other
apps.
67
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Appendix B. Details of Materials and Methods
B.l Research Approach and Materials
This research project sought to answer the question: What are the best design options for a Backhaul Alaska
program data management application, or similar technology, that addresses different users' needs for
supporting program operations and decision making? The research was centered around three activities
that built on one another through iterative rounds of research synthesis and participant engagement
(Figure B.l). These research activities provided complementary strains of inquiry that were appropriate for
establishing a) foundational research, which provides a full understanding of the users and variables
affecting app design and use, and b) evaluative research, which builds from the foundational work to
understand the strengths, limitations, and needed refinements of specific ideas or prototypes (Chipchase
2017). All research was done remotely due to the Covid-19 pandemic, which restricted travel.
Our research was also informed by a pre-planning phase leading up to the project proposal during which
time EPA scientists and Zender Group (who are the administrators of Backhaul Alaska on behalf of the Solid
Waste Alaska Taskforce (SWAT) walked through expected implementation tasks and brainstormed
potential steps that might be expedited through the use of an app (see appendix G). Following this pre-
planning phase, the three research activities that we undertook included the following:
First, we sought to learn from existing environmental or resource management apps that are being used in
Alaska, and in similar remote areas, in order to understand the local context and best practices for app
development. We held a series of conversations (N = 8, Table B.l) with product teams representing these
apps in order to understand how they were developed for, and received by, their target audiences.
Conversations were held by videoconference software and were mostly unstructured, although an agenda
was shared with participants ahead of time. Conversations were documented using written notes that were
used to create a synthesis of technology factors and locally relevant norms and practices that should be
considered for app development with rural Alaska communities. To the extent feasible, we also sought to
learn about the environmental and community contexts within which the app will be operated.
Second, the EPA research team held conversations (n = 16, Table B.l) with key Backhaul Alaska participants
and staff, including CTs, SCs, RCs, and VCs. Following the guidelines of the Interagency Arctic Research
Policy Committee's (IARPC) "Principles for Conducting Research in the Arctic" (IARPC 2018), all RCs and
representatives of SWAT were invited to have a series of conversation about Backhaul Alaska and its app
development activities (Table B.l). The EPA research team also worked with Backhaul Alaska administrators
to invite additional participation by VCs and their assistants from several pilot communities (Table B.l).
Several RCs, VCs, and assistants participated in more than one conversation, which allowed the team to
learn about program operations overtime and follow up on questions that arose during initial rounds of
analysis. The EPA research team also held conversations with a purposely selected sample of regional and
technology experts (N = 9, Table B.l) to learn about regional waste management, technology, and app
development considerations. Further details about conversation structure, agendas, and analysis are
provided in section B.3.0 and Appendix C.
68
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska.
Third, the EPA research team asked RCs and SCs to collect and share video footage (details in section B.4.0)
of backhaul activities in the field, including training and use of Surveyl23. These materials provided direct
but limited observations of backhaul and inventory app use, which were important for understanding the
setting and context in which an app would be used. They also helped to observe and document
development needs that the users might not be directly aware of and were an important complement to
the conversations (Abras et al. 2004, Daae and Boks 2015).
Each of the above research activities (regional app investigation, conversations, and user observations)
informed the others and were integrated and synthesized at multiple time periods during the project
(Figure B.l). These intermediary syntheses were then used as the basis for several workshops with RCs,
SCs, and program administrators to discuss project findings, explore outstanding questions, and co-
interpret the work with the EPA research team.
Table B.l. List of Conversations by Participant Type. The middle column lists the total number of conversations and
includes repeated conversations with several individuals and/or programs, while the right-hand column lists the
number of unique programs or individuals that participated.
Participant Type
Total
conversations*
Unique
entities
(individuals
or programs)
Technology experts
7
6
Regional waste management experts
(external to Backhaul Alaska)
2
2
Regional programs using environmental or
resource management apps
8
3
Representatives of SWAT
3
3
Control Tower (CT) and State Coordinator
(SC) operators
2
2
Regional Coordinators (RCs)
6
*
Village Coordinators (VCs) and their
assistants
5
4
Total:
33
24
Notes:
*Several programs and companies were represented by more than one person;
thus, the total number of unique programs or individuals (N = 24) is less than the
total number of participants that we spoke with.
**A fifth RC participated through the RC workshop.
69
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04,11.23 rev 16:00 w/Notes for Graphics
Project phase
Regional App investigation
Conversations
User observations (remote)
Workshops and presentations
Pilot app scenarios & evaluatio
Research milestones*
Sept-2C)| Oct-20 | Nov-20 | Dec-20
Project
start
Project planning & scoping
Jan-21 Feb-21 Mar-21
Apr-211 May-21
Research
Jun-21 | Jul-21 | Aug-21
Systems
worksho
Systems
workshop
synthesis
report
RC press
RC
worksho
Survey 123
Regional
app
synthesis
doc.
1) First
round
convo.
coding
2) coding
synthesis
doc.
Synthesis
report of
research
to-date
Sep-21 | Oct-21
Planed completion
Nov-21
I I
Research
Admin
workshop
Dec-21 | Jan-22 | Feb-22 |22-Mar
Extension Ongoing support
(participants' request & Covid)
1) RC
workshop
report
2) Admin
workshop
document
3) Video /
photo
report
Surveyl23 and "Other"
Second
round
convo.
coding
Pilot
evaluation
and
comparison
report
Overall synthesis, analysis, and write-up
Figure B.l. Activity Timeline of the RSTIP. From project start date (September 2020) to extended completion date (December 2021) and ongoing support.
The timeline does not cover activities between EPA ORD, Region 10, and tribal groups prior to the start date (pre-planning phase Jan-Aug, 2020). Such pre-
project buildup is outlined in Figure 1.3 in the main text. Note that in the milestones (*), "reports" are internal documents that are shared with the entire
RSTIP team and beyond as needed. "Documents" are internal write-ups for use within a smaller EPA research team that completed human subjects training
and are listed on the Institutional Review Board (IRB). (This project was deemed exempt from IRB approval of human subjects research, see Appendix H)
70
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
B.2 Project Timeline
At the start of the project, the EPA research team held a systems diagram workshop (October 2020) with
Backhaul Alaska program administrators (Zender Group) to understand the process and stages of backhaul
and how they might relate to a supporting app. Results from the workshop (presented in Appendix E) were
used to develop conversation agendas for use in eliciting input from all participants (Tables B.2 and B.3).
Additionally, the project and its objectives were presented to RCs (March 2021) prior to staring conversations
with the RCs to familiarize them with the project and receive feedback and questions. At this point, RCs
requested that the project timeline be extended from September 2021 to late December 2021. The
extension would allow everyone involved in the work to learn from the upcoming summer backhaul and the
use of the Surveyl23 app, for which planning was underway. Extending the project end date also made sense
due to research and administrative slowdowns caused by the ongoing Covid-19 pandemic.
Initial conversations with RCs were held from late March through April 2021. Thematically coded results of
these conversations, along with findings from other research activities to date (Figure B.l), were compiled
into a report and discussed with RCs in a 60-minute workshop (July 2021). The workshop consisted of three
short presentations of the research findings and discussion questions. Following each set of questions,
participants were directed to virtual and collaborative workspaces, using Google Jam Boards, where they
could post responses to the questions (lasting 5 to 7 minutes) followed by short (3 to 5 minute) periods of
open discussion.
Following the workshop, the EPA research team held conversations with representatives of SWAT, VCs, and
their assistants (July through early September 2021). Conversation topics varied according to the expertise
of the participant, but Surveyl23 use was a major focus during conversations with VCs and their assistants,
who had by now had experience using the app. This stage of the research culminated in three day workshop
between the EPA research team and Zender Group. The EPA research team shared all research syntheses to
date, as well as a list of discussion points and questions that emerged during analysis. These items were
discussed by the group to arrive at conclusions and establish future lines of inquiry.
The EPA research team then held a final round of conversations (October 2021) with several RCs, CTs, SCs,
and one VC's assistant. Surveyl23 use was a major point of discussion. These conversations also provided
opportunities to discuss observations from the video footage with RCs (Figure B.l).
Throughout the above outlined activities, the EPA research team also held conversations with several people
with expertise about regional waste management and technology considerations. Several conversations
with technology experts transitioned into a series of feasibility scoping and pricing activities with Esri and
another app development company (as discussed in the main text, Section 4.3). Additionally, Backhaul Alaska
conducted a pilot of the app from this other development company during December 2021. Because we had
reached the extended research project deadline, we were unable to document and analyze use of the local
app through videos and conversations. Instead, the EPA research team and Backhaul Alaska administrators
(Zender Group) held several discussions to reflect on the experiences working with the apps. These
conversations culminated in a formal written comparison that was shared with developers to document
Backhaul Alaska's app choice in moving forward. The evaluation criteria included the following (the order of
items does not imply importance): (1) the readiness of the apps for immediate use in the upcoming backhaul
71
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
season, 3) ease of transition, (2) costs, (3) desired features, (4) available technical support, (5) the
background of the app company, especially its familiarity with work in rural Alaska, and (6) short term vs.
long-term needs of Backhaul Alaska.
B.3 Details of Conversations, Participant Recruitment, and Coding
All participants were selected for app development project participation based on their involvement in
Backhaul Alaska and as individuals with expertise in regional waste management or app development in
similar contexts. Zender Group, the administrator of Backhaul Alaska on behalf of SWAT, assisted in
identifying Backhaul Alaska participants and regional experts. Technology experts were identified through a
review of published or gray literature (materials and research produced by organizations outside of the
traditional commercial or academic publishing and distribution channels).
While the research project was deemed not to constitute human subjects research by the University of North
Carolina at Chapel Hill, which assists with administration of human subject research for EPA (Appendix H),
all participants were provided a written summary of the project outlining their participation and use/storage
of all data collected following standard protocols for the involvement of human subjects as outlined by the
University of North Carolina at Chapel Hill. All of the EPA team members who participated in the
conversations and/or worked with data prior to its being de-identified and aggregated had completed
human subject research training.
Conversations with representatives of SWAT, SCs, CTs, RCs, VCs, and assistants (N = 16, Table B.l) lasted
from 60 to 90 minutes, followed a meeting agenda (Table B.2), and were largely unstructured, allowing
participants to skip questions and introduce new lines of inquiry during the discussions. Conversations were
audio recorded, transcribed, and thematically coded using a deductive approach, in which a set of codes are
established beforehand and then applied to the text (Bernard 2006, Saldana 2015).
After completing the first round of RC conversations (N = 4), two researchers read through all four
transcripts, plus an additional fifth transcript from a conversation with a regional waste expert, to generate
a set of thematic codes. The RC transcripts where then coded by a single researcher and reviewed by the
second, in order to allow discussion of how well the codes were capturing the desired themes and if anything
was missing. Several codes were added and removed over multiple rounds of coding and discussion, until a
final code book was generated (Appendix C); this code book was then re-applied to the original RC
conversations and subsequent conversations with representatives of SWAT, SCs, CTs, RCs, VCs, and
assistants (N = 16).
Conversations with regional and technology experts (N = 9, Table B.l) were also unstructured and guided by
a meeting agenda (Table B.3). These conversations were audio recorded and were not transcribed; rather,
these audio files were played back to document a detailed set of meeting notes, which were integrated into
intermediary research synthesis reports and documents (Figure B.l).
72
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Table B.2. Example conversation agenda used with representatives of SWAT, RCs, VCs, and their assistants during
the early stages of the project.
Meeting objectives:
1. Learn about logistical, administrative, and budgetary requirements for the Backhaul Alaska program, the
community contexts in which the program operates, and its participants' needs.
2. Discuss future meetings, including those to get feedback on prototypes of the app and incorporate this into the
app's design.
Agenda:
1. EPA ORD describes the project's background and goals
2. Learn about Backhaul Alaska program
a) Participants' role in the Backhaul Alaska Program.
b) Time and resource constraints
c) Management and logistical specifics
d) Existing tools being used
3. Learn about app use
a) Helpful and unhelpful app features
4. Learn about challenges and opportunities when doing backhaul
5. Next steps
a) Scheduling future meetings
b) Additional contacts
6. Wrap up and sharing any final thoughts
Table B.3. Example conversation agenda used with technology experts during early stages of the project.
Meeting objectives:
1. Learn from past experiences working on technology and app development.
2. Develop a background level of understanding about app development needs and choices to support a Backhaul
Alaska app and to help contextualize what is feasible and under what conditions.
Agenda
1. EPA ORD describes the background and goals of the project
2. Participants describe past work on apps and technology
3. General app development process, tips, and advice
4. Backhaul Alaska app development
a) Development choices
i. Potential tradeoffs in functionality
ii. Path dependency or lock-ins?
iii. Programming languages/environments
b) App maintenance, integration, longevity, and ownership?
c) Thoughts on development costs and time
d) Advice on development needs based on what you heard about our project
5. Next steps
a) Future feedback?
b) Recommendation for additional contacts
6. Wrap up and sharing any final thoughts
73
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
B.4 Details of Photo and Video Interpretation
The EPA research team provided RCs and SCs with guidelines about the kinds of video footage content that
they thought would be useful for foundational and evaluative research (Chipchase 2017). Training and use
of Surveyl23 were important topics that the project wished to record. Participants were also provided
instruction to document general day-to-day backhaul activities (see Appendix D). To avoid burdening RCs
and SCs while they conducted important backhaul work and training, the EPA research team's requests, in
terms of number of videos and length, were minimal. The EPA research team also asked for photos and while
they were important for observing the overall context in which backhaul took place, the photos were less
informative about potential app user activities. Therefore, analysis was restricted to the videos.
One RC was able to provide ten short cellphone videos, ranging from 30 to 90 seconds, from three different
village visits. A professional videographer, who was contracted by Backhaul Alaska to accompany visits to
several other villages for developing promotional videos, also provided content. The professional
videographer provided 445 video files, ranging from several seconds to approximately nine minutes.
A single researcher reviewed all footage from the professional videographer. Forty-four video files (9.8%)
that included backhaul activities (e.g., people doing backhaul work, using or being trained on the surveyl23
app, or depicted storage or staging of backhaul materials) were determined relevant to the analysis. The
remaining files were largely landscape shots or other supplementary footage (i.e., "b-roll") that might be
used as by the professional videographer for their work. The selected 44 files, along with the ten others from
the RC, were reviewed a second time by the same researcher for coding and analysis. Each video was tagged
with a binary yes/no to indicate if it included content for each of the codes in Table B.4 and a set of key
insights learned was then written up for each video. This data was further synthesized into a report outlining
the use of Surveyl23 in the field and associated trainings, local working conditions (in field and office),
considerations about technology hardware, waste packing and storage, the use of paper in the field, and any
new questions that arose to be addressed in future rounds of research.
Table B.4. Thematic codes for binary tagging of video content.
Codes are listed in alphabetical order and presented in two columns to reduce table length.
Thematic Codes
shown In alphabetical order
Communications
Packing
Culturally specific issues
Paper use in the field (or other analog tools)
Hardware and OS (tablet use/device use)
Storage
How would community coordination work/look like?
Synergistic opportunities/challenges and problems
Liability and safety concerns
Training
Loading and Shipping
Waste inventory and ID
Local Office Conditions (distinct from field work)
What might be sensitive information among
communities?
Non-local waste location/storage (e.g., at a camp or hunting
site)
74
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Appendix C. Thematic Code Book
The table below lists the code book used for coding conversations with representatives of SWAT, SCs, CTs,
RCs, VC, and their assistants. The coder was instructed to code entire ideas (most likely one or more
sentences), as opposed to single words or snippets, in order understand the context and meaning of any
coded statement even after extracting the coded section from the main transcript.
Table C.l. Thematic code book used to analyze content of conversations related to Backhaul Alaska
Thematic Codes in Groups (bold)
Explanation and Examples (where needed)
Backhaul Alaska system and process
A thematic set of codes to document stages and key items in
the backhaul. This group was largely based on the systems
diagram activity, with some modifications.
1.
Communications
2.
Training
3.
Initial planning and equipment
inventory
4.
Waste inventory and ID
5.
Site visits
6.
Packing
7.
Storage
8.
Loading and shipping
9.
Tracking and receiving
10.
Other Backhaul Alaska Program
systems and processes
App Functions and Features
Record sentiments and statements about...
11.
Desired functions and features
... favorable technology features (directly stated or indirectly
(general apps)
implied)
12.
Undesired functions and
... unfavorable technology features (directly stated or
features (general apps)
indirectly implied)
13.
Wishlist for an app
... desired features specifically about the app (directly stated
or indirectly implied)
14.
App alternatives and concerns
... technology concerns or alternatives (directly stated or
(Sub theme: Use of paper in
indirectly implied), e.g., technophobia, paper-based options,
the field)*
etc.
15.
Hardware and Operating
... hardware and OS that are being used, desired or undesired,
System
and other related issues and challenges
16.
Surveyl23 (specific to app)**
... the use of and/or training on Surveyl23. Coded material
should include, but not be limited to, strengths and
weaknesses of Surveyl23, desired improvements (specifically
expressed or implied), and general discussions about the app.
Table continues next page
75
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Thematic Codes in Groups (bold)
Explanation and Examples (where needed)
Governance and Infrastructure
Record sentiments and statements about...
17. Community level
... any issues occurring within a community that affect the BHA
process. This includes community infrastructure and cultural
or political issues (which we might sub-code later on). This is
NOT the place to record information about collecting a
community profile or history, however.
18. Regional level
... any issues occurring among communities or about regional
planning and logistics that affect the backhaul process.
19. Larger governmental
... any issues involving higher levels of government/
governance, such as state or interstate.
20. People and organizations
... people and organizations that participants discuss in
relation to the Backhaul Alaska Program.
21. Places
... place names or other locations that participants discuss.
Backhaul Program Items
22. Regional coordinator site
reconnaissance
... information, activities, and resources that RCs want, collect,
or use to understand a community and its context ahead of
time (directly stated or indirectly implied)
23. Regional coordinator
community of practice (Informal
sharing, peer-to-peer learning,
etc.)
information, activities, and resources that RCs
share/coordinate among themselves and/or other regional
actors (directly stated or indirectly implied)
24. Culturally specific issues/
indigenous technology and data
sovereignty
... cultural issues, opportunities, or concerns as they relate to
the app or Backhaul Alaska Program. This includes, but is not
limited to, issues related to technology use and data storage /
access
25. Synergistic opportunities
... opportunities and ideas for the Backhaul Alaska Program to
complement, tie in, or otherwise support other community
and regional goals (e.g., jobs, conservation, etc). One example
that a RC brought up was a computer recycling program that
they are working on.
26. Challenges and problems
... key challenges and problems that people talk about. E.g.,
staff turnover is a recurring example. Note: while solving staff
turnover might have synergistic effects with other programs,
as staff turnover effects many related jobs in these
communities, we did not code it as synergistic opportunities
unless someone specifically framed it that way. We did make
this link later in the analysis, however.
27. Liability and safety concerns
... actual safety situations and hypothetical issues or concerns
that an actor might have. E.g., a community concerned that
they could be liable/responsible if they do not do something
correctly. Independent of their actual liability, they still have
the concern, so it is an issue that needs to be addressed.
28. Covid related
... issues related to the ongoing Covid-19 pandemic.
Table continues on next page
76
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Thematic Codes in Groups (bold)
Explanation and Examples (where needed)
Data Management and Next Steps
29. Scheduling and follow-ups
Document any key scheduling information or follow-up items
that people mention
30. Additional contacts
Document any people or organizations that are recommend as
possible participants or people/organizations that we should
follow up with
31. Resources
Document things such as websites, documents, etc. Not every
mention of such things needs to get coded if, for example, it is
clearly not relevant. However, we should document such
content and keep in mind that something may not at first seem
relevant but turn out to be relevant later as we learn more, so
it is better to code than not to code.
32. Key Quotes
Highlight any statements that just stand out and seem like
they might be used to illustrate a point or used in a
presentation as an example about why we are proposing
something.
Notes:
*During the first round of coding, paper, and the use of binders came up as app alternatives, which led to a more
focused coding of the sub-theme "Use of paper in the field" during the second round of coding.
** The Surveyl23 theme was predominantly used during the second round of conversation coding.
77
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Appendix D. Storyboard Provided to Professional Videographer
: tivities
D.l Storyboard Provided to Professional Videographer:
Objectives and instructions for collecting video clips and photos of Backhaul Alaska Program activities
Overview: The "user centered design" approach for app development puts the user's needs and experience
at the center of the development process. Directly observing people doing tasks in the field or at their place
of work provides valuable information about how to customize an app to meet their needs. Because of
distance, cost, logistics, and the Covid-19 pandemic, we are unable to travel and observe backhaul activities
in person. Therefore, we want to use photos and videos to learn about what people are doing. Watching
how people conduct their work for the backhaul events during site visits—and seeing other activities during
the site visit—can provide important learning opportunities.
While this information will help us, your needs and your work come first! We do not want this exercise to
disrupt anyone's work or responsibilities during site visits and we do not want the presence of a camera
or camera-phone to influence the way someone is doing their work. We want to be "flies on the wall."
Ultimately, we hope to gain a better understanding about backhaul operations on the ground.
Questions we hope to answer from this content are:
1. Who is involved in the backhaul operations? What are their tasks and objectives?
2. What are the specific stages/processes of backhaul operations? What are the activities occurring
within each stage?
3. Where are backhaul operations taking place (e.g., indoors vs. outdoors) and what does the place look
like? What is around them?
4. What types of devices or tools are used during backhaul operations (e.g., phone, tablet, computer,
fax, etc.)?
5. Is anything restricting or complicating data collection activities (such as PPE, winter or field clothing,
holding other devices/tools, inclement weather, etc.)?
78
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Instructions for Collecting Videos and Photos (ideally for each village/site visit)
Please collect brief videos and photos of a few different activities/scenarios as you are able.
Table D.l offers guidelines on what to film and photograph. For each scenario, we think 2 to 3 videos of 30
to 60 seconds and 3 photos will be sufficient; however, the table is just a guide; you are free to adjust what
you film or photograph based on the opportunities presented to you in the field.
Table D.l Guidelines for Professional Videographer on What to Film and Photograph
Media
What to Record and Photograph
What We Hope to Learn
Videos (4-6), 30-60 sec
Photos (10)
Processes/Operations: How do local
backhaul staff perform their backhaul
duties?
Who is involved in the process? What are the jobs
to be done?
How are these tasks completed?
Videos (2-3), 30-60 sec
Photos (3)
Tools/Devices: What tools/devices are
being used by the backhaul staff to
perform backhaul duties?
What tools/devices are used for data
collection/entry?
How are these tools/devices used?
Videos (2-3), 30-60 sec
Photos (3)
Materials, documentation, training,
paperwork: How is data managed in
the field?
How is data managed/recorded? How is training
administered?
Videos and/or photos,
as you see fit
The different workspaces where the
app will be used.
What are the different places and settings where
the BHA operations occur?
Videos and/or photos,
as you see fit
Other photos and videos you think we
might find interesting.
See something that sticks out to you about using
the app? Document it and let us know!
79
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
D.2 Storyboard Provided to RCs:
Objectives and instruction for collecting video clips and photos of Backhaul Alaska Program activities
Overview: The "user centered design" approach for app development puts the user's needs and experience
at the center of the development process. Directly observing people in the field or at their place of work,
provides valuable information about how to customize an app to meet their needs. Because of distance,
cost, logistics, and the Covid-19 pandemic, we are unable to travel and see BAP activities in person.
Therefore, we want to use photos and videos to learn about what people are doing. Watching how people
use the Surveyl23 app during site visit-and seeing other activities during the site visit-can provide
important learning opportunities.
While this information will help us, your needs and your work come first! We do not want this exercise to
disrupt anyone's work or use of Surveyl23, and we do not want the presence of a camera or camera-
phone to influence the way someone is doing their work. We want to be "flies on the wall." Ultimately,
we hope to gain a better understanding about how the app will be used in the field.
Questions we hope to answer from this exercise are:
1. What type of device(s) is the app being used on (e.g., phone, tablet, Android, Apple, etc.)?
2. Where is the person using the app working (e.g., indoors vs. outdoors) and what does the place look
like? What is around them?
3. Is anything restricting or complicating their use of the app (such as PPE, winter orfield clothing, holding
other devices/tools, etc.)?
4. Is the person using the app doing other activities at the same time?
Instructions for Collecting Videos and Photos (ideally for each village/site visit)
Please use your cell phone to collect videos and photos of a few different activities/scenarios.
Table D.2 offers guidelines on what to film and photograph. For each scenario, we think 2 to 3 videos of 30
to 60 seconds and 3 photos will be sufficient; however, the table is just a guide; you are free to adjust what
you film or photograph based on the opportunities presented to you in the field. You are on the ground;
you are the expert!
Table D.2 Guidelines for RCs on What to Film and Photograph
Media
What to Record and Photograph
What We Hope to Learn
Videos (2-3), 30-60 sec
Photos (3)
Wide-shot, or landscape (app user and
surroundings), of people using the app as
they would for a future backhaul event.
What is happening around the individual that
may affect the app's use.
Videos (2-3), 30-60 sec
Photos (3)
Close-up (approx. 6 feet away) of
someone while they are using the app in
the course of their work.
What the individual is doing, wearing, and
holding while they are using the app.
Videos (2-3), 30-60 sec
Photos (3)
People using the app during app training.
How training is done.
Videos and/or photos,
as you see fit
The different workspace(s) where the
app will be used.
The different places and settings where the app
will be used.
Videos and/or photos,
as you see fit
Other photos and videos you think we
might find interesting.
See something that sticks out to you about
using the app? Document it and let us know!
80
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Appendix E. App Requirements Document Used for
Communicating with Developers During the Project
An app requirements document is a short document intended to communicate a client's wishes and needs
to an app developer. The document helps the developer understand the scope of work and evaluate if their
services can meet the client's needs. The following app requirements document was prepared to
communicate with several developers during the project. The document was updated several times over
the course of our research to reflect the research teams' most current understanding of Backhaul Alaska's
app development needs. It was last updated in December 2021. The content of this requirements
document was sufficient for its intended use of communicating with developers during the project, but it
is far less comprehensive than the information documented in the main text of this report. This document
is nota deliverable of the EPA funded project (i.e., RSTIP); rather, itwasa research and communication tool
used during the project.
At the end of this document are three appendices (Figures E.l, E.2, and E.3) prepared by Backhaul Alaska
administrators outlining various planned workflows. These planned workflows have been compared, in the
main text or body of the report, to the workflows in practice that were documented through conversations
during the research project.
E.l Backhaul Alaska Software Application Functional Specification Document
E.1.0 Background
There are roughly 200 rural Alaska villages that are not connected to the road system, many operating Class
III permitted landfills that may lack containment or treatment. Few options exist to properly dispose of
household hazardous wastes, which often end up being burned if disposed of in local landfills. Improper
disposal releases contaminants to the surrounding environment, creating human exposure risks and
interfering with important cultural and subsistence hunting, fishing, and gathering. One option to deal with
these hazardous wastes is to ship them out by plane or barge. It is economically and logistically infeasible,
however, for each village to independently ship out their waste. A region-wide program is needed.
Backhaul Alaska (https://backhaulalaska.org) addresses this issue by organizing statewide recycling logistics
for rural community hazardous wastes and for bolstering local programs through needed training and
infrastructure assistance. The program has been successful at pilot scale and as it begins its expansion to
serve all villages statewide, seeks to develop a more robust and tailored digital infrastructure that will best
accommodate its reverse supply chain data needs and align with the unique cultural and operating
conditions of its clientele and users.
81
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
E.l.l Objectives & Outcomes
The Problem
There is no safe way to discard hazardous waste in rural landfills.
The majority of Alaska villages have unlined landfills that are not
designed for hazardous waste disposal
The objectives of this document are to outline the software application needs for the Backhaul Alaska
program. Data organization and digital processing efficiencies are needed to reduce local village human
resource needs and move waste materials so that village programs operate safely, storage sites do not
overflow, and village foods and water are not contaminated,
E.1.2 Product Context
The Backhaul Alaska Program has multiple broad functions:
• Program management (i.e., planning, scheduling, village enrollment, etc.)
• Logistics management (i.e., shipping, barge tracking, transporter/vendor selection, packaging/
materials approval, coordination)
• Operations (i.e., site-visits, authorization, loading, collection/packaging, inventory management)
• Job training (i.e., hazmat training, curriculum development)
Each of these broad Backhaul Alaska functions requires multiple program staff to engage in processes with
different data management needs and there are different implications for functional requirements. A
software application would need to address components of each of these functions with a primary focus
on logistics management and operations. Section E.2 presents app requirements and concepts developed
through this research. Table E.l describes what an app must and should be able to do in order to meet
Backhaul Alaska's needs. Three figures following the text outline additional information including: (1) a
conceptual model of the Backhaul Alaska process (and components of this system that need software
application functionality) (Figure E.l), (2) a process flow diagram for the shipment sub-process (Figure E.2),
and (3) an inventory management data flow outline (Figure E.3).
(The rest of this page is left blank to allow the following table to fit on one page).
82
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
E.2. Requirements
E.2.1 Functionality Table
Table E.l Functionality Table Listing functional needs desired for proposed Backhaul Alaska App
#
Function
Description
Priority
1
Inventory and
records
management
Must enable Village Coordinators (and sometimes Regional Coordinators
and other workers associated with Backhaul Alaska) to upload data
(numeric, geospatial (GPS points), and media) and metadata identifying,
organizing, prioritizing, and the status of packing waste materials. These
records must be stored and accessible to Backhaul Alaska staff. This
functionality is currently being addressed by Surveyl23 and an Access
database in SharePoint.
Must
2
Verification and
approval
Regional Coordinators and Statewide Coordinators must be able to see the
status of, verify, and approve* multiple steps of the packing and shipping
process. *Note: An ideal workflow improvement would be to have
functionality that permits records to be returned to the record creator if
something is incorrect/incomplete, and to allow for commenting on
completed records so that the record creator can receive feedback.
Must
3
Survey creation
and
administration
Backhaul Alaska staff must be able to create and deploy surveys (or simple
checklists) for data collection and feedback purposes. Currently this
function is being addressed through Surveyl23.
Must
4
Shipment
planning,
tracking, and
routing
Must enable staff to manage shipping logistics (including scheduling,
booking, tracking, and record keeping). Barge/plane location and status
should be reported and visualized (both en route to communities/hubs and
en route to recyclers).
Must
5
Communication
Likely needs to enable both two-way (peer to peer, peer to expert, and peer
to management) communication and one way communication
(notifications, announcements, status updates, etc.).
Should
6
Resource
access
Must provide access to document library that contains training materials
(videos, documents, manuals, FAQs, communications information, etc.).
Must
7
Community
information
Likely needs to provide community information and data in both geospatial
and tabular/text formats that summarizes a community's profile and onsite
capacity: storage facilities, barge access points, inventory, and relevant
historic information, along with community contact information.
Should
8
Analytics/
Dashboard
Must enable program staff (State and Regional Coordinators) to track and
view data for each community and for each backhaul event along with easy
export formats for data and possible report generation. Should be able to
transform data into shipping logs/bills of lading (BOL) to minimize manual
data migration/form completion.
Must
83
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
E.2.2 Usability Requirements
The inventory and records management functions of the app must be usable by rural Alaskans and must
cover a range of digital literacy and digital access. The user interface must be straightforward and utilitarian,
with intuitive navigation, data entry, and communication. Native language options (or translation
capabilities, especially Yup'ik) would be an enhancement but not a requirement, since virtually all potential
users are comfortable with English. Inventory and records management functions (especially the process
of data collection and uploading by Village Coordinators) must be easy to perform in a rural Alaska context,
which often includes field environments with poor weather conditions and where other program
operations are occurring concurrently. Data entry and upload must take advantage of experience with using
cellphones to take photos and videos and must allow for this to happen while wearing personal protective
equipment (PPE) or winter clothing (gloves).
E.2.3 Technical Constraints
The app must be able to provide offline capability to Village Coordinators and other program staff for
entering data with limited/no cellular access. The app must also support the use of this interface and the
storage/submission of records from limited bandwidth facilities.
E.2.4 Device Specifications
The system must be supported on both mobile devices and tablets and should provide access to certain
information (e.g., the dashboard or analytics) through a desktop accessed website. We currently do not
know if there are preferred operating systems but acknowledge that program staff will be accessing mobile
applications on personal devices and both iOS and Android are currently used.
E.2.5 Data Requirements
Data sovereignty and ownership are key considerations for the communities that we worked with. The
usage agreement must be flexible and permit different communities to specify different parameters for
who has access to their data and how it is used.
E.2.6 Access Requirements
Different components of the app will require different levels of permissions and access. Statewide
Coordinators should have access to all data generated by program participants and staff. Regional
Coordinators should be able to view the data generated by the communities they coordinate. Village
Coordinators should have access to records they generate, including historical records so that continuity of
information can be retained through staff turnover.
E.2.7 Security Requirements
Some of the information captured may be sensitive in nature. Examples might include community dynamics
like staff turnover or locations that people may not wish everyone to know about. Photos may capture
content (e.g., activities in the background) that could be sensitive. Functionality should permit record
generation to specify sensitivity concerns or flag a record as being potentially sensitive.
84
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Figure E.l Backhaul Process Flow Diagram
Process Flow - General Backhaul Process
PRIMARY RESPONSIBLE PARTY
Yllhige Coordinator
s
Inilata BacXtaJ Aridities
PtkmIu© Materials
i|:S
ASSISt re* tfA "0 4Ut*WiSS>00& __ - Review 9tfXT)6SK)nS
packaged material - Approve packaged matwia-s
' ID ovailoc4e containers for
backhaul shipment
• Submit to RC & SC
Rev e* ootor ¦> setect. not
RC 5C
*
AssrI in fwvi§ optons
[Obtainquotas
-Submit
LoocVPtepHre shemert
- Complete smdqimj paperwofc
Document loaded items
Submit pepemofH to RC 4 5C
Asaat »n rev«w and submit — - Review tor end submit to CT
- Reconcile shipments & settle
accounts
- Obtain transporter & vendor
teeback
Obtain VC. RC. CT teed fx**
- Updgfe/generate records
Obtain booking tesoivat cm w
"transporter
'Author i/o shipment
^Bbmrtrinal^pei^orhto^^
transportof^vemdoj^^^^^
Trade shipmonJ to destination
Update rvenhpwis (status)
Submit documents to SC
END OF PROCESS
Figure E.l Backhaul Process Flow Diagram. Flow diagram indicates processes and actions taken by Backhaul Alaska
staff (VC. RC, SC, and CT) during general implementation of Backhaul Alaska. Details are shown for steps in the
process of (1) Inventory submission and approval, (2) Identifying shipping equipment, (3) Shipment approval and
scheduling, and (4) Shipping, tracking, and reconciliation. Pink shading indicates actions appropriate for a app.
85
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Figure E.2 Shipment Sub-Process Flow Diagram
Figure E.2 Shipment Sub-Process Flow Diagram. Flow diagram indicates processes and actions taken by Backhaul
Alaska staff (VC. RC, SC, and CT) during the shipment sub-process of Backhaul Alaska implementation. Details are
shown for steps in the process of (1) shipping inventory submission and approval, and (2) shipping and tracking.
86
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Figure E.S Inventory Management Data Flow Outline
END OF PROCESS
Figure E.S inventory Management Process Flow Diagram. Flow diagram depicts the inventory processes (left panel)
and records generated (right panel) when using the Surveyl23 app during Backhaul Alaska implementation. Data
entry into Survey 123 by the user triggers automated population of fields in a Backhaul Alaska Sharepoint list,
followed by transfer to a iinked Access database, where records are maintained. The alternative method requires
manual entry of data directly into the database. Details are shown for steps in these processes and the records that
are created.
87
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Workshop
The following diagram (Figure F.l) and table (Table F.l) are the finalized output of the app specific systems
workshop held in October 2020 with Backhaul Alaska administrators (Zender Group). The objective of the
workshop was to develop an overall picture of Backhaul Alaska operations and how they might relate to an
app. Major objective of the workshop were to outline (1) potential app users and (2) the processes for
which they might use an app were. The diagram and table below do not represent a final understanding of
Backhaul Alaska's users, operations, or app needs, which are discussed in the main report. Rather, this
figure and table document an early research stage of the project (October 2020) that informed subsequent
stages of the work.
(The rest of this page is intentionally left blank to allow for landscape orientation of the following pages).
88
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rurai Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
—^ Direct relationship (as A goes up, so does B)
indirect relationship (as A goes up, B goes down)
Arrows from actors sometimes Just indicate action, as opposed to increase or decrease
Figure F.l. System diagram of Backhaul Alaska operations pertinent to use of an app,
Final diagram generated during the October 2020 systems workshop with Backhaul Program Administrators.
89
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Table F.l. Table showing lists of many of the key actors and system dynamics components that were generated during the Systems Workshop discussion.
Note that columns are independent lists of items, and the content should not be read or linked across rows. (Page 1 of 2)
Actors
Program
Levels
Local
Disturbances
Regional
Disturbances
Information Exchange
Legal/Liability
Regional
Dynamics
Local-Regional Interaction
Verification
Local lead
Local backhaul
Loss of staff
Loss of staff
Planning and inventory; need to know
what kind of storage capacity
community has.
Spills, fires, on the job
injuries
Work with control
tower to plan
shipping for all
communities
Work with control tower to plan
shipping for all communities
Planning and
inventory: storage
capacity
Local backup
Control Tower
Flood
natural disaster
Planning: if community does not have
storage container, contact vendor to
supply one for south-bound shipping.
Accident when transporting
waste to central location
(e.g., person dumps ATV).
Site visits done by
regional or statewide
coordinator
Site visits done by Regional or
Statewide Coordinator
Planning and
inventory: initial
itinerary
control tower
Regional
coordination
Missing transport
pandemic
Work with control tower to plan
south bound transport for each
community.
Natural disasters: where
does waste go. What is in it?
Natural disasters:
where does waste
go? What is in it?
Villages have many needs that
vary
Planning and
inventory: PPE
Shippers: barge
State
coordination
Pandemic
Site visit: assess readiness and
capacity (storage equipment and
facilities, do they own or need to rent,
what are the barge facilities (or
airport) like). Would be nice to know
what supplies are on hand in a
community, and how much material.
In a mature program, we hope site
visits would only occurfor new staff
situations. For pilot we visit every site.
Shipping wrong or
improperly packed materials
Would be nice to
have waste go to
central hub to cut
down on costs.
Backup staffing often comes from
other villages
Verify that training
is still good (good
for 3 years)
Shippers: truck
Interstate
transport
After site visit, periodically check to
make sure all is ok.
Loss of shipping container at
sea
Virtual site visits
could help program
efficiency
Would be nice to have waste go
to central hub to cut down on
costs.
Site visit: verify
equipment and
capacity
Shippers: plane
Communities currently fill out form to
outline collection methods for
households, businesses, etc.
Distance learning could help to
ensure that villages have certified
person; do not want them to have
to wait 6 -12 months
Control tower
verifies that waste
is received by
vendor
BHA trainers
Would be great if barge companies
could communicate with community
via an app; now they call them or use
VHF.
Virtual site visits could help
program efficiency
Shippers want to
know that waste is
correctly packed
Certified signer
Vendors want to know that the goods
are on the way.
Regional Coordinators tend to
work with communities in other
capacities, so they know
communities
Vendors want to
know that waste is
en route
90
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Actors
Program
Levels
Local
Disturbances
Regional
Disturbances
Information Exchange
Legal/Liability
Regional
Dynamics
Local-Regional Interaction
Verification
Certified packer
Loss of trained person is a "gut
punch" to program.
Smaller communities might not
ship every year; this might affect
staffing
Packer's helpers
Trained people need to be reachable
with two-way communication.
Regional coordinator
Waste that is spread out is a problem.
Need it central and need an
inventory.
Backup staff (for local
packing)
Need a Plan B if a community misses a
shipment. Where will the waste be
stored; what is in the waste?
Statewide coordinator
If there was an accident, need an
inventory of the waste.
Vendor
Record keeping so that when a person
leaves, the community is not left
starting over from scratch
*Local community
member (i.e., waste
producer)
*Local organizations (i.e.,
waste producers)
Note: *Likely not an actor or user from the perspective of the app and its development.
91
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Appendix G. App Wish List Developed Before Commencement
of RSTIP
Table G.l is a wish list of items for a custom developed Backhaul Alaska app that was generated by Zender
Group, EPA Region 10, and ORD prior to the start of research (i.e., the RSTIP funded project). This list was
used to help customize the Surveyl23 pilot app. This is a partial representation of the work during those
meetings. Listed items were also ranked based on priority, however priorities are omitted here, as those
priorities and the list itself represent an early understanding of Backhaul Alaska's app needs. The main body
of the report, and Appendix A, discuss a more up-to-date understanding of app development needs.
Table G.l. Pre-project "Wish List" of items for a custom developed Backhaul Alaska app. (Page 1 of 2)
Category
(Note that order does not indicate priority)
Function
Inventory management (backhaul
items & supplies)
Backhaul inventory management
Inventory management (backhaul
items & supplies)
Generate and scan QR codes for inventory management.
Community & regional information
Contact Info (village, regional, statewide, BHA administrators,
vendors, etc.)
Communication
Contact list management - Village Coordinators, Regional
Coordinators, vendors (transporters and end destinations)
Backhaul activities/operations
Material inventory data entry by Village/Regional Coordinators
with capabilities to attach photos and files
Transportation activities/operations
Transporter options/contacts, route options, etc. per village
Transportation activities/operations
Shipment tracking & status updates
Other
Varying levels of permission
Other
Dashboard where, if there is tiered access, you can get an idea
what's happening at a regional or statewide level
Other
The general ability to take photos and upload to certain
folders/records-for inventory and ready to ship, maybe ability
to upload a photo of the loaded Conex (open door) or the final
palletized e-wastes, etc.
Other
A method for uploaded photos to be tagged/sorted
automatically depending on where in the app they are uploaded
Other
Recordkeeping - being able to pull up past shipping docs,
inventory records, training certs, checklists, etc.
Other
General ability to use the app while offline/out of cell coverage,
and sync data when back online/in data coverage range
Community & regional information
Community profiles & onsite capacity; being able to capture a
summary of a community's collection and storage program
Transportation Activities/Operations
Port (or hub?) infrastructure quality status tracker-ability to
allow villages to confirm that barge docking area is adequate;
document with photos?
92
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Category
(Note that order does not indicate priority)
Function
Backhaul activities/operations
Training credential tracking (maybe this is only for Statewide
Coordinator?)
Backhaul activities/operations
Notification of expiring training certs
Backhaul activities/operations
Tracking when the barge is coming, airplane is coming (notifying
village, vendors)
Backhaul activities/operations
Ability to access, complete, and submit checklists (as
needed/developed) & upload
Backhaul activities/operations
Regional Coordinator site visit review documentation and
acknowledgement that village is "ready to ship"
Backhaul activities/operations
Readiness checklist completion by Regional Coordinator
Transportation activities/operations
Booking & scheduling shipments (through the Control Tower)
Communication
A trouble-shooting file/record maybe for each category-ex: if
folks are having questions on a certain battery type, identifying
a material, or a question on whether their storage area was
organized well, along with the ability to load that into a
checklist record
Community & regional information
Community maps (that folks can tap on the screen or otherwise
to designate locations of storage or what not)
Inventory management (backhaul
items & supplies)
Supply inventory management (packing/PPE supplies,
equipment inventory tracking)
Communication
Peer-to-peer & peer-to-expert contact for troubleshooting
(Facetime/Skype/Zoom, direct messaging options)
Communication
Messaging system - chat box? Message center? For
communication between villages, regions, and statewide
Communication
Feedback/evaluation form submittal by Village and Regional
Coordinators
Communication
Program announcements
Transportation activities/operations
Completing/submitting shipping paperwork (BOLs, manifests,
airway bills, etc.)
Community & regional information
Access to community Integrated Waste Management Plans,
Landfill Operations Plans, and any community specific
procedures they have initiated (if available)
Backhaul activities/operations
Training videos, informational documents, manuals, etc.
Outreach & education media
Community education resources-sharing items or being able to
download posters or materials
Outreach & education media
Templates for PSAs, flyers, event planning, etc.
Transportation activities/operations
Carrier board to manage third party bidding for rural shipments-
-compatibility with external app?
93
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
Appendix H. Statement of Human Subject Exemption
Letter from the Institutional Review Board (IRB) of the University of North Carolina, Chapel Hill, NC,
which assists EPA ORD in administering its Human Subjects Research program. (Page 1 of 2)
3/16/2021 Mail - Sayles, Jesse - Outlook
IRB Notice - 21-0019
IRB
Mon 3/1/2021 8:23 AM
To: TenBrink, Marilyn
Cc: Carvalho, Gabriela ; Sayles, Jesse
To: Marilyn TenBrink
Aux Services Affiliates: EPA
From: Office of Human Research Ethics
Date: 3/01/2021
RE: Determination that Research or Research-Like Activity does not require IRB Approval
Study #: 21-0019
Study Title: Human-centered Design to Improve Management of Household Hazardous Waste
Programs in Alaskan Tribal Communities
This submission, Reference ID 320510, was reviewed by the Office of Human Research Ethics, which
has determined that this submission does not constitute human subjects research as defined under
federal regulations [45 CFR 46.102 (e or I) and 21 CFR 56.102(c)(e)(l)] and does not require IRB
approval as it is not systematic investigation for generalizable knowledge.
Study Description:
Purpose:
From https://www.epa.gov/innovation/region-10-human-centered-design-improve-management-
household-hazardous-waste-programs
"Because of their remote location, Alaska's rural tribal communities have limited options for disposal
of hazardous waste - which often means disposal in an unlined landfill or burning without any
emissions control. Backhaul Alaska is an EPA-funded pilot program that coordinates the hauling of
hazardous waste out of rural Alaskan communities using empty cargo space in barges and planes on
return trips. Beginning in 2021, a 10-year plan will expand the Backhaul Alaska program to serve 160
communities across Alaska."
"This project will employ a human-centered design approach (i.e., a methodology that focuses on the
needs of end-users) to create a mobile application for backhaul service operations. A data
management app will be created that supports different user needs including tribal and municipal
government staff, citizens, program administrators, and others. The app will help individuals in rural
Alaska villages manage inventory, track shipments, and submit observations that impact waste
management operations. The app will also support program implementation such as optimizing
transportation logistics. Anticipated project results include cost-effective program operations, reduced
health and environmental risks, and support for each tribe's capacity to manage their local backhaul
program."
Participants:
https://outlook.offlce365.eom/mail/search/id/AAQkADdlMjdiMTM2LWMxNzUtNDIkOC04ODhhLWYxMzA4ZjAyMGQ0YgAQACg7NmZ1kl5luX48fNgHix... 1/2
94
-------
Using Human-Centered Design to Create a Hazardous Waste Management App for use in Rural Alaska, mtb v04.11.23 rev 16:00 w/Notes for Graphics
3/16/2021 Maii- Sayles, Jesse - Outlook
Regional arid local community backhaul program coordinators that are working for and/or
participating in the Backhual Alaska pilot program, as well as relevant knowledge experts and app
developers.
Procedures (methods):
Participants will be purposefully selected because of their work arid/or participation in the backhaul
pilot program, or because we have identified them as a knowledge expert doing work relevant to our
app development goals. We may also ask participants to refer other people that we should speak
with.
We will hold one or more unstructured conversations with participants to learn from their experiences.
In the case of backhaul program participants, we will also aim to understand their needs, as described
above, and how the app can best be designed to serve their needs.
After conducting an initial set of conversations, we will begin to develop prototypes of the app. These
will be initially be in the form of Microsoft PowerPoint sketches and will later be mocked up in Adobe
Xd (Experience Design) in order to enable more realistic representation and introduce visual designs,
(such as the shape or color of specific elements), content (such as instructions and other "copy"), and
basic interactivity (such as the transition from a button to its associated page within the application).
We will solicit user feedback primarily through unstructured conversations, but may also use other
activities such as card sorting, ranking or choosing among different options if we feel these activities
will enhance our interactions with participants. Are methods are solely intended to facilitate
conversation and gather information about how to customize the app for its intended users. Our
methods are NOT intended to test hypotheses or create generalizable knowledge.
Please be aware that approval may still be required from other relevant authorities or "gatekeepers"
(e.g., school principals, facility directors, custodians of records), even though IRB approval is not
required.
If your study protocol changes in such a way that this determination will no longer apply, you should
contact the above IRB before making the changes.
CC:
Gabriela Carvalho, Aux Services Affiliates: EPA
Jesse Sayles, Aux Services Affiliates: EPAIRB Informational Message - please do not use email REPLY to
this address
https://outlook.office365.com/rr!ail/search/:d/AAQkADdlM|diMTIVI2LWMxNzUtNDIkOC04ODhhLWYxMzA4ZjAyMGQ0YgAQACg7NmZ1 k!5luX48fNgHsx... 2/2
95
-------
PRESORTED STANDARD
POSTAGE & FEES PAID
EPA
PERMIT NO. G-35
United States
Environmental Protection
Agency
Office of Research and Development (8101R)
Washington, DC 20460
Official Business
Penalty for Private Use
$300
^<9
Recycled/Recyclable Printed on paper that contains a minimum of
50% postconsumer fiber content processed chlorine free
------- |