Research SUMMARY
Open Source SWMM: Adaptive Quality Management
Introduction
EPA's Storm Water Management Model
(SWMM) is used throughout the world for
planning, analysis, and design related to
stormwater runoff, combined and sanitary
sewers, and other drainage systems. It can be
used to evaluate gray infrastructure stormwater
control strategies, such as pipes and storm
drains, and is a useful tool for creating cost-
effective green/gray hybrid stormwater control
solutions. SWMM was developed to help
support local, state, and national stormwater
management objectives to reduce runoff
through infiltration and retention and help to
reduce discharges that cause impairment of
waterbodies.
Fifty years old, SWMM is a mature application.
ORD refactored SWMM in the 2000's in the C
language when prior versions were in FORTRAN.
In 2016, Federal Government policy changed,
and US EPA now applies Open Source Software
development practices to SWMM in an effort to
improve cost efficiency, mission effectiveness,
and public experience with Government
programs (https://sourcecode.cio.gov/). The
team developing SWMM is growing to
encompass multiple organizations and
individuals both inside and outside the Agency.
Software development techniques and the
development team itself are evolving. It is
therefore necessary for SWMM's software
quality assurance processes to evolve as well.
Quality assurance is an important aspect of
software project management and can be used
to target an appropriate level of reliability
considering its intended use. The quality of
SWMM software should be managed to help
ensure that it remains the industry standard for
hydraulic and hydrologic simulation of Urban
watersheds.
Quality Management
The SWMM Project Team practices Adaptive
Software Quality Management. The central idea
is that project management actions are
adaptive and based in part on the current state
of software quality and the desired project
objectives. Testing is used as a metric for
software quality and thus feeds into project
management decisions. For more information
about Adaptive Software Quality Management,
see the SWMM GitHub Site
https://github.com/USEPA/Stormwater-
Management-Mode .
The highest level of efficacy for improving
SWMM can be achieved when testing is
integrated early within the software lifecycle.
For example, by emphasizing testing in
preliminary development, deficiencies to be
identified and corrected early, when they are
easiest to identify and the least expensive to fix.
As a software application matures, new
features are developed by adding to or
modifying existing source code. During this
phase of the lifecycle, testing can be used to
detect and prevent regressions in existing
application functionality during feature
development and software maintenance
activities. Finally, when an application is ready
for release to external users, testing can be
used to certify that it meets requirements and is
indeed ready for use. The sections that follow
describe the different types of testing employed
throughout the SWMM development lifecycle
to manage software quality.
Component Testing
The SWMM Project contains several modular
software components. The most significant of
which are the hydrologic, hydraulic, and water
quality solvers. Going forward we will refer to
the solvers collectively as the "SWMM-solver"
or just "solver" for short. The solver has been
Office of Research and Development
Center for Environmental Solutions and Emergency Response
Page| 1

-------
written as a software library to facilitate use by
other applications.
Regression Testing
The main purpose of the solver is to perform
hydraulic and water quality simulations. At a
basic level, it uses numerical methods to solve
systems of partial differential equations that
describe the physics of water movement and
basic water chemistry in pipe networks. In this
respect, it is a prototypical example of a
scientific and engineering software application.
From a software quality assurance perspective,
these types of applications are particularly
challenging to develop and maintain. The
application must work for each user's unique
urban watershed model, but by necessity, is
written to solve the general case. Achieving this
level of generality makes the applications
abstract and complex. Furthermore, numerical
methods can be sensitive to seemingly
insignificant changes made to the source code.

Aiio-Len^Jh Oil
To address some of these challenges, we
employ regression testing to run a suite of
SWMM examples and scrutinize the results. The
suite currently contains approximately 58
examples. These examples taken together,
exercise a broad set of application features. The
simulation output for each example is
compared to a known "good" result called a
benchmark. If any change in results is detected,
the developer is alerted and should determine if
it is nominal or indicative of a potential defect
in the software. We have written custom
components for an Open Source regression
testing framework, command scripts, and SOPs
to make running and maintaining the tests, and
managing benchmarks as straightforward as
possible. The regression testing framework
performs many of these tasks automatically,
however, developer participation and
judgement remain essential. For more
information about regression testing, see the
SWMM GitHub Site.
Continuous Integration
Testing is essential to ensuring software quality.
How frequently tests ought to be run, however,
remains an open question. The SWMM project
has adopted a software development practice
called "Continuous Integration" (CI). The idea
behind it is to automate the build and testing
workflows so they can be triggered anytime a
change to the project occurs. In the course of
routine development, this means the software
is built and tested several times a day. This may
seem extreme; however, because the
workflows have been fully automated, most of
the work is performed by the computer locally
or on a remote server out on the cloud. CI
enables us to maintain the project in a known
state of quality. This is simple yet powerful idea
that is extremely useful to the overall
management of the project.
Release Testing
Preparing a software release involves finalizing
changes made to the source code, fixing
defects, testing, documentation, and
installation packaging. The purpose of release
testing is to certify that the software is ready for
production. We use a release test suite of
approximately 58 SWMM examples that have
been contributed by users over the history of
the project. The regression test framework
described previously is used to run the release
suite and monitor results for failures, crashes,
and other anomalous behavior. Because the
release suite is separate from the regression
suite and has been contributed by users,
CFS SI ICO* K.Y 3M«67
-------
running it is an indicator of how the software
will perform in production.
Conclusions
Software testing is an integral part of the
software lifecycle, not just the last or an
unnecessary step. It can be used throughout the
development process to insure the
"correctness" of each unit of functionality, more
complex components, and the entire program.
Testing helps developers and stakeholders have
confidence in the quality of the software being
created. Therefore, testing is a critical aspect of
software quality assurance and essential to
overall project management. By focusing
attention on software quality, we can ensure
that SWMM continues to meet the needs of our
customers and stakeholders now and in the
future.
For More Information
SWMM Executables and Documentation:
https://www.epa.gov/water-research/storm-
water-management-model-swmm
SWMM GitHub Repository:
https://github.com/USEPA/Stormwater-
Management-Model
Contact Information
Technical Contacts:
Michelle Simon (simon.michelleffiepa.gov)
Michael Tryby (trvbv.michael(a)epa.gov)
swmm(5)epa.gov
Release date: February 5, 2021
Contributing
SWMM has an open source community that
includes software developers, engineers,
students, and other stakeholders. The
community version of SWMM can be found at
https://github.com/OpenWaterAnalvtics/Storm
water-Management-Model. Everyone is
welcome to participate. Get involved!
Page|3
EPA/600/S-21/123
-------