UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. D.C. 20460
July 31, 1985
OFFICE OP
THE ADMINISTRATOR
Hon* Lee M. Thomas
Administrator
U. S. Environmental Protection,
Agency
401 M Street, S. W.
Washington, D.C. 20460
Dear Mr. Thomas;
The Environmental Engineering Committee of the Science Advisory Board was
asked by the Office of Water to review a report entitled "A Probabilistic
Methodology for Analyzing Water Quality Effects of Urban Runoff on Rivers
and Streams*" The Committee has completed its review, and Is pleased to
forward its report.
The Conmittee believes strongly that statistically-based approaches to water
quality management are an important tool for the decision-maker, and commends
the Agency for supporting the effort under review. The method described is
technically sound, but only for the specific applications for which it was
developed. The Committee has serious concerns about apparent Agency Interest
in using the aproach la situations for which It is not technically suitable.
The Cotraalttee does not believe that the technique, as it now exists, should
be extrapolated beyond the purpose and application area for which It was
developed without appropriate additional development and verification, nor
should It be used by individuals who do not fully understand the approach
and the assumptions Inherent therein.
If you have any questions, or should you wish any further action on our part,
please call on us.
Sincerely,
Raymond C. Loehr
Chairman, Environmental
Engineering Committee
cc: E, Longest Science Advisory Board
C. Myers
D. Athayde
S.- Tuller '
T. Barawell
E, Southerland Norton Nelson
T, Yosle Chairman, Executive Committee
Science Advisory Board
-------
REPORT
on the review of
"A PROBABILISTIC METHODOLOGY FOR ANALYZING
WATER QUALITY EFFECTS OF URBAN RUNOFF ON RIVERS AND STREAMS"
by the
Environmental Engineering Committee
Science Advisory Board
U* S. Environmental Protection Agency
June, 1985
-------
I. EXECUTIVE SUMMARY
The Environmental Engineering Committee believes strongly that the Agency
should consider certain water quality phenomena in a probabilistic manner.
The Agency is commended for supporting the effort under review and similar
activities, and is encouraged to pursue other efforts to increase its capa-
bilities to deal with random environmental phenomena.
In its review of a report entitled "A. Probabilistic Methodology .for Analyzing
Water Quality Effects of Urban Runoff on. Rivets and Streams", the Committee
agreed to address, and report to the Agency on, two general questions;
A. Is the technique scientifically valid and adequate for the specific
application for which it was developed?
B. Is it appropriate for further or broader application?
The technique presented in the report was developed for advective systems,
i.e., flowing fresh-water streams. It was developed for nonpoint source pol-
lutants which are conservative (such as total dissolved solids - TDS) or are
reacting substances characterized by first-order kinetics (such as biochemi-
cal oxygen demand - BOD).
No serious flaws were found in the model, but some clarifications and im-
provements are suggested! The Committee believes that the technique is sci-
entifically valid and acceptable forthe purpose and conditions for which it
was developed and presented in the report„
The Committee has serious reservations about the potential for indiscriminate
use of the technique for purposes and applications beyond its current purpose.
In order for the existing technique, as documented in the report, to be appro-
priate for use in water quality situations involving many commonly encountered
real-world aspects, considerable additional development and validation would
be required. Included would be capabilities to deal with non-conservative se-
quential reactants (such as dissolved oxygen or some toxic substances or nu-
trients). The technique as it exists is not applicable to tidal or estuarine
systems, nor to lakes. Aspects such as these are frequently encountered with
both point and nonpoint source pollutants.
The Committee does not believe the technique, as it now exists, should be ex-
trapolated beyond the purpose and application area for which it was developed
without appropriate additional development and verification. This could en-
tail substantial additional effort. Furthermore, the Committee believes
there is cause for great concern if a valid technique with deceptively attrac-
tive simplicity in its ease of application (such as the one described) is
pushed beyond its capabilities. This is, of course, true for any mathemati-
cal modeling, simulation or predictive procedure. The potential exists here
due to the technique's having been developed specifically for the Nationwide
Urban Runoff Program (NURP), but being under active consideration for adoption
and use in other problem areas such as waste load allocation and toxic/pesti-
cide fate and effects.
-------
-2-
II. INTRODUCTION
There appears to be increasing Interest in the environmental community in
the fact that many natural phenomena have strong random components, and that
environmental regulations should explieity acknowledge this fact. The Water
Planning Division, Office of Water Planning and Standards, as a part of the
Nationwide Urban Runoff Program, has been developing statistical methods
for the analysis of urban stormwater for some time. These methods have
had limited use and have not, in general, addressed water quality impacts,
The Environmental Engineering Committee of the Science Advisory Board was
asked, in a memorandum from the Office of Water, to review a. report entitled
"A Probabilistic Methodology for Analyzing Water Quality Effects of Urban
Runoff on Rivers and Streams", dated February 15, 1984» and prepared by the
Office of Water.
The principal Issues for review were:
A. Is the technique described in the report scientifically valid and
adequate for the specific application for which it was developed?
B. Is the technique appropriate for further or broader application?
The Committee agreed to accept the task* and organized a Subcommittee, ia-
cluding several consultants with special expertise in the area of stochastic
modeling* chaired by Dr. Benjamin C. Dysart III (see Subcommittee roster,
Appendix A), to conduct the review. The Subcommittee and its consultants
met on November 26, 1984, and on February 25, 1985. In the course of its
review, the Subcommittee examined the report (together with an earlier report
prepared by the Agency's Office of Water Planning and Standards entitled "A
Statistical Method for the Assessment of Urban Stormwater"), written com-
ments by four individuals, and had an in-depth review of the report by Agency
staff and their consultants (Dr. Dominic DiToro and Mr. Eugene Drlscoll)
who prepared the report under review. The SAB consultants, Dr. Mitchell
Small and Dr* Barry Adams, were asked to prepare written comments, and these
are attached as Appendices B and C,
III. CONCLUSIONS
All natural processes or phenomena such as * hydrology and water quality have
probabilistic or stochastic elements* However, the majority of the mathema-
tical models used today are essentially deterministic in structure, i* e.,
they are based on the concept of conservation of mass and momentum, and
attempt to simulate explicitly, using direct solutions of equations, the
processes that transport and transform material and energy in the natural
system. Statistical models, on the other hand, do not explicitly simulate
natural systea processes, but rather provide estiaates of the value(s) of
output (dependent) variables, given the values of input (independent) vari-
ables in the system and the statistical relationships between them. These
models allow the direct consideration of random phenomena, such as streamflow
-------
-3-
or rainfall, as well as the performance of traditional treatment processes
and other engineered control systems. They also reflect as yet undefined
mechanisms.
They are limited, however, in that any statistical relationship derived from
a given set of data for a natural system reflects the particular spatial
arrangement and natural system processes existing when the data were collected.
For any significantly different system, new data mist he obtained and new
statistical relationships developed,
The technique presented in the report was developed for advective systems
(freshwater flowing streams), and for nonpoint source pollutants which are
conservative or singular-reacting substances, e.g. total dissolved solids
(TDS), suspended sediment, biochemical oxygen demand (BOD), and bacteria.
The Committee concludes that:
A* The technique isscientifically valid and acceptable for the purpose
and conditions for which It was developed and pregented in the report.
No critical flaws were found in the technique as presented in the report* No
substantial statistical problems were found. Several conclusions were reached
by the the Conudttee and its consultants (see Appendices B and .C)* These are
summarized as follows:
1. fhe methodology presented in the report is acceptable for con-
servative and first-order reactive substances in flowing fresh
water streams. In such systems, the assumption of instantaneous
and perfect mixing is customarily made; the Committee believes
that this is an acceptable practical approximation in the vast
majority of cases.
2. While some reviewers contended that a simulation modeling ap-
proach was more appropriate than, the probabilistic model, the
Committee believes that these two approaches should be consi-*-
dered as alternatives, the selection, of which would depend on
the nature of the problem and the question addressed in a spe-
cific application.
3* In response to questions raised about the mathematical vali-
dity of the methodology, the Committee is satisfied with the
approximations concerning log-normality, the mean recurrence
interval, and the uncertainty analysis. This comment is not
meant to preclude the examination of alternative procedures
as described in the reviewers' and consultants' comments.
4. In response to the question concerning incomplete or inadequate
field data, or data which are thought to be incomplete or in-
adequate, the Committee recognizes extrapolation from other
locations of similar characteristics as a common and accepted
practice.
-------
B. The technique, as it is described in Che report, should not be
extrapolated beyond the purpose andApplication area for which it was devel-
oped without substantial additional development and verification*
The Committee believes that a probabilistic methodology has a. very important
role and contribution, as a modern water quality management tool for regula-
tion, planning and facility design. In order for the technique presented in
the report to be appropriate for use in other water quality assessments,
other capabilites would have to be included. Among these would be the capa—
bility to deal with non-conservative sequential reactants such as dissolved
oxygen and some toxic substances and nutrients, with tidal or estuarine
systems» and with multiple-source pollutant inputs within a stream reach.
Furthermore, the Cotamittee believes that there is cause for great concern if
a valid technique with deceptively attractive simplicity in its ease of appli-
cation (such as the one described in the report) is pushed beyond its capa-
bilities. In this case, the potential exists due to the technique's having
been developed specifically for the Nationwide Urban Runoff Program but is
apparently under active consideration in other EPA offices for adoption and
use as is in problem areas such as waste load allocation and toxics/pesticides
fate and effects*
The question of whether it would be worthwhile to modify, extend, and verify
the technique to cover additional application areas is a resource question
for the Agency more than a scientific one that can be addressed by this Com-
mittee. The Committee believes the potential surely exists to extend the
technique to cover more applications. This remains to be done, however, and
someone must decide to devote the necessary resources to the effort if the
technique is to realize any potential beyond its existing limited capability.
These resource requirements could be quite extensive*
-------
APPENDIX A
U.S. ENVIRONMENTAL PROTECTION AGENCY
SCIENCE ADVISORY BOMB
ENVIRONMENTAL ENGINEERING COMITTEE
PROBABILISTIC METHODOLOGY SUBCOMMITTEE
CHAIRMAN • .
Dr. Benjamin C» Dysart, III
Environmental Systems Engineering Department
Clemson University
Clemson, SC 29631
MEMBERS
Mr. Richard A. Conway
Corporate Development Fellow
Union Carbide Corporation
P. 0. Box 8361 (770/342)
South Charleston, WV 25303
Dr. Raymond C. Loehr
Civil Engineering Department
8.614 ECJ Hall
University of Texas
Austin, TX 78712
Dr. Donald J, O'Connor
Professor of Environmental Engineering
Environmental Engineering Science Program
Manhattan College
Manhattan College Parkway-
Bronx » m 10471
Dr. Charles R. O'Melia
Professor of Environmental Engineering
Department of Geography and Environmental
Engineering
The Johns Hopkins University
Baltimore, MD 21218
CONSULTANTS
Dr. Barry Adams
Department of Civil Engineering
University of Toronto
Toronto, Canada M5-S1A4
Dr. Mitchell Small
Department of Civil Engineering
Carnegie-Mellon University
Schenley Park
Pittsburgh, PA 15213
-------
APPENDIX B
REVIEW OF PROBABILISTIC METHODOLOGY FOR ANALYSIS OF
WATER QUALITY IMPACTS OF URBAN RUNOFF
Mitchell J. Small
Carnegie-Mellon University
December 21, 1984
In general, I find the probabilistic model appropriate for the purpose and set-of
conditions for which it was designed, A discussion of pertinent technical details is
provided in the final section of this review, I did find a clear-cut error in the initial
equations given for the mean recurrence interval (MRI). However, the approximate
equation is correct. That is, it approximates the corrected version of the MRI
equation provided in my final section. As such, the results presented ir» the report
are essentially correct. I also examine a number of the issues raised in the review
of Kahn. But first, I'd like to address some of the broader issues relative to the
model's applicability and possible extension.
The authors present the model as a tool for initial assessment. The conceptual
nature of the model limits it to that purpose, and the limitations are highlighted in
the report and in the reviews of Southeriand, Barnwell and Roesner. There is always
a temptation to push tools as easy to use as this into more detailed applications,
waste-load allocation, etc. The reviewers appear to be wary of this. The first issue
that should be raised, however, is not the tool, but the generality and importance of
the question it addresses: the formulation of probabilistic water quality criteria for
nonpoint source impacts. Should we pursue more precise specification of water
quality exceedence frequencies, as is done with air pollution? Is a 10 year MRI value
appropriate to evaluate nonpoint source contributions? If so, then it is up to the
working engineers and planners to devise appropriate analysis techniques to predict
exceedence frequencies for their specific water system arid problems. This could
involve direct, deterministic simulation using inputs from a long-term period of
record; stochastic simulation using synthetic flow and quality inputs generated
consistent with observed or regional patterns; or direct analytical techniques such as
developed in the NURP report. As indicated in the reviews, many analysts are more
comfortable with simulation techniques, in part because random process properties
(temporal and spatial persistence, correlation, etc.) are inherently included. There
should not be an official push to require any one particular approach. As engineers
receive more exposure to the probabilistic method and recognize its ease of use,
some may wish to use it as a complement to their more detailed evaluations. This
will provide working examples which will serve to either build confidence in the
-------
method, or pinpoint its weaknesses and limitations. This may be stimulated if
analysts required to evaluate a 10 year concentration find the simulation approach
too cumbersome. The point is, let the problem and need be better specified , . ,
1
this will drive the selection of tools.
As noted by Southerland and iarnwell. direct short-term concentration impacts from
stormwater runoff are rare. Indeed it Is 'interestrng-that the three test cases cited in
the NURP report lead to a conclusion of no significant impairment. Problems which
do occur, such as lake or estuary nutrient problems, long-term bottom sediment
buildups, or coastal zone bacterial problems also have important probabilistic
features. Research to develop analytical methods for these problems should, I think,
be encouraged. The resulting models may again be limited to initial assessments,
but could provide useful complements to more rigorous modeling approaches.
Another issue of general concern involves the role and characterization of
uncertainty in this, and other similar models. The uncertainty computed directly in
the current model is representative of inherent temporal variability. Some storms are
small, some are large. Sometimes the upstream flow is high, sometimes low.
Similarly for upstream and runoff concentrations. This variability is represented by
fitted or estimated distribution functions. The parameters of these distributions are,
however, also unknown, reflecting uncertainty of another sort. In a narrow sense
this uncertainty may be represented using confidence intervals for means or
variances estimated from observed data, however in reality it reflects our larger
uncertainty in the overall structure and representation of the problem. This scientific
uncertainty is somewhat subjective, but may be represented in the model and is (in
principle) reduceabie.
The NURP report addresses scientific-parameter uncertainty in its sensitivity
analyses for stream-runoff correlation and upstream concentrations (Ch, 5). In recent
years, however, more formal and complete methods for sensitivity and uncertainty
analysis have been developed. These generally involve replication of the underlying
model with random or stratified sampling of the uncertain input ' space, and
examination of the resulting distribution of the output variabie(s). The relative
importance of the different input parameters is inferred using methods such as rank-
order correlation analysis, A nice example of this technique is provided in the
attached article by Jaffe and Ferrara (Water Research, 18: 1169*1174), This type of
analysis should be performed on the NURP model to provide a more complete and
unified picture of model sensitivity. An alternative approach is to allow users of the
-------
methodology the ability to perform rapid and exploratory sensitivity analysts with
interactive graphic software.
Technical Review
The only major technical problem I was able to uncover }s an error In the initial
equations for the MRI, Eqs 3-34 and 3-35, The correct forms arenas follows;- •••*•'•••- •
> c } - 1 - Pr(e < < }W <34a)
MSI = —r"rT"Ty.*l^ C35a)
Eq. 3S is a proper first order approximation for Eq. 3Sa {though not for Eq. 35, which
yields absurd results). The use of Eq, 36 is no longer necessary, however, because
Eq. 35a will not blow-up, as did Eq. 35, when used with a calculator or in a BASIC
program. The approximation is close enough, particularly for long return periods, so
that none of the results presented to date tneed be redone. However, for future
work, Eq. 35a is preferable.
! will address now some of the points raised by H Kahn, Eq* 3-10 is correct
because high values of D corresspond to tow values of $ , so that the 95 percentiie
D corressponds to the 5 pereentile $ , The legends on some of the probability plots
are indeed reversed from usual convention, showing the probability of being greater,
rather than less than, the indicated value. I don't expect this to provide too big a
problem, however, for most users. The use of the term "arithmetic moments" is a
matter of semantics designed to differentiate between the direct sample moments
and geometric (log) moments which are also used. Additional clarification of the
useage could perhaps be provided in the. report,
The suggestion to use a beta distribution for 0 is interesting, I believe that the
beta result is only precisely true when Q^and Qs are independent and gamma
distributed with the same scale parameter (though not necessarily the same shape
parameter). Neither of these limitations is appropriate for the problem at hand, but
the beta distribution is sufficiently flexible over the range 0-1 so that it probably
stili provides a good approximation. It is not clear though that it will lead to either
considerable simplification of the calculation procedure, or a more accurate estimate
for the Co distribution finally computed (particularly in the numerical scheme, which
-------
seems to be the method of choice). 1 guess someone needs to push the calculation
through to see if this is the case. Similarly for the logistics model, I can see
where if -tnD is assumed to have a logistics distribution, then $ corressponds to the
cdf measure. 'But I was unable to follow this through for computation of the
moments of $ t or for other direct use in the model. !n general, I am satisfied with
the level of validation provided for the computational procedure in the NURP report
and the recent paper of D5 Tore (ASCE* June, 1984). The authors of the report may,
however, wish to present some distribution plots for $ to see how large is the
deviation from lognormality, and how iwweh probability is predicted in the
'impossible' range, $ > 1.
I believe Kahn is correct in noting the sensitivity of confidence intervals in the
calculated MRI. I also believe that some of the "nonstatistical" factors which I raise
above (when discussing scientific uncertainty) are most critical in determining the
appropriate range of this sensitivity,
To summarize then, I find the direct model largely appropriate and scientifically
credible for the purpose for which it was designed. A number of more genera!
issues are raised in addressing the model's broader applicability.'
-------
APPENDIX C
IEVIEW OF
A PROBABILISTIC METHODOLOGY FOE THE ANALYSIS OF
WATER QUALITY EFFECTS OF UB1AN RUNOFF
B.J. Adams, Ph.D., P. lag.
Department of Civil Engineering
University of Toronto
January 1985'
This review is provided at eke request of the Probabilistic Methods
Subcommittee/Environmental Engineering Committee/Science Advisory Board of
the U.S. Environmental Protection Agency. Hie review addresses both the
original documene "A Probabilistic Methodology for Analyzing Water Quality
Effects of Urban Runoff pn livers and Streams", Office of Water, U.S. EPA,
Washington, D.C*» 15 February 1984, and comments on the methodology contained
in memoranda from leary D. Kahn (21 July 1984), Elizabeth Sontherland
(undated), Thomas 0, Barnwell, Jr. (29 May 1984) and Larry A. Soesner (11
July 1984).
-------
-2-
BACKGEOUND
There are some issues concerning methodologies and models which require
addressing before evaluating the probabilistic methodology.
Model Function
For Che purposes of this discussion, a distinction is made between
4
two pronounced differences in the function or role of models.
A crude distinction is as follows;
(i) Physical Models - models which focus an relating cause and effect in
terms of the physical variables of the phenomena
involved in the processes being modeled.
(ii) Decision Models - models which focus on relating cause and effect in
terms of the decision variables of the alternatives
used to control the processes being modeled.
As the function of a model is a. major determinant if its formulation, the
formulation of a model directly defines its use: a decision model is less
appropriate for explaining cause and effect at the level of the detailed
physical phenomena being described in the problem; a physical model is
less appropriate at the level of, evaluating the cos ^effectiveness o£
control alternatives for solving the problem.
Examples of commonly encountered physical models are SW£M III,
QUAL II, BECEIV II, etc. while examples of decision models for evaluating
water quality control alternatives are ST01M, HPS and analytical
probabilistic models.
Model Operation
Physical models and decision models are "operated" under different
conditions* Physical models simulate details of the process, including
particular water quality phenomena, under only very specific conditions
-------
-3-
(eg» 7Q10 conditions). Thus,, the system behaviour is known In
detail but only far very limited conditions. On the other hand, decision
models simulate a more aggregate response of the process over the complete
spectrum of conditions (eg, the entire probability distribution of flow
conditions). Thus, the system behaviour is known in less physical detail,
but ie is more generally know over a wide range of conditions,
In a very coarse way, the difference between physical and
decision models is the followingi
fhysicsl models describe the response of the system in terms of the details
of the water quality transformations for surrogate conditions.
Decision models describe the response o£ the system in terms of surrogate
water quality transformations for a complete range of conditions.
In this way, physical models are generally limited to the analysis of
specific events while decision models analyze a continaina of evencs* A
tradeoff in model selection is apparent: more detailed analysis of behaviour
for less general conditions or. less detailed analysis of behaviour for more
general conditions,
(For a more detailed discussion of event simulation deficiencies, see the
attached paper on "Design Storm Pathology" by Mams and Howard).
-------
THE DECISION PROBLEM
The decision problem at hand contains a series o£ elements as
follows;' ISPUT-— DR4IHAGE SYSTEM -*- OUTPUTS -*-B£dI?lR-^-WATES QUALITY-*- USE.
The inputs are meteorologic and ara taken as given. The drainage system
Is comprised of a catchment and engineered components of the catchment
(conveyance, storage, treatment, etc. components). The outputs are point
and non point discharges categorized by variable time series of water and
pollutant masses. The receiver is the stream, river, lake or estuary
accepting the point and noupoint discharges. The combined characteristics
of the receiver and discharges determine the ia-situ water quality which
ultimately affects the level of beneficial use associated with the receiver-.
The decision problem is to determine the appropriate level of
beneficial use associated with the receiver and, simultaneously, the
engineering and management measures to be deployed on_the catchment in
order to achieve that level of use.
It has been, widely accepted that water quality criteria may
be used as surrogate measures of the degree to which level of use is
achieved. It is thus common practice that decisions regarding the
engineering and management of the catchment4are made on the basis of predicted
in-stream water quality. Similarly, the magnitude and frequency of point
and nonpoiat discharges (the outputs of the catchment) may be used in
turn as surrogate measures of the resulting in-stream water quality. In
this way, the engineering and management measures may be evaluated on the
basis of what happens at the "end-of-the-pipe".
The question is what level of information is aost appropriate
to the decision at hand? Is it preferable to have much information on what
-------
happens at the end-of-the-pipe, even though this is a. surrogate for in-stream
water quality •which Is in turn a surrogate for level of beneficial use; or
little information on resulting water quality; or even less information on
level of beneficial use? Clearly t as we. move farther "downstream1* in the
process from input to use, we gain in. detail but lose in scope. At some
paint, the quality of information from modeling efforts most be judged.
For a given application there* is an optical tradeoff between detail and
scope of information.
For example, if two Mutually exclusive engineering alternatives
for pollution control are available at similar cost and if one of the
alternatives results in pollutant1 mass discharges with a smaller mean and
variance than the other, it is clearly preferable. Ho "water quality model-
ling". Is required to make the decision'. A valid decision may be made on, the
basis of wbat happens at the end-of-the—pipe. This esaunple is admittedly
simplistic but it makes the point: the information provided by modeling
efforts should be appropriate to the decision being made,
EVALUATION OF THE METHODOLOG*
It is in the above contest that the probabilistic methodology
must be evaluated. The models of the probabilistic methodology should not
be viewed as simply hydrologic models, water quality models or even statis-
tical models; rather, they should be viewed as decision models which contaia
elements of hydrology, water quality and statistics.
Thus, die adequacy of the probabilistic methodolog7 must be
evaluated with respect to the following:
Physical adequacy - How well do the models represent the physical phenomena
(hydrology, hydraulics, water quality transformations,
etc,) that they attempt to describe?
-------
-6-
Mathematical/statistical adequacy - How good are the underlying simplifying
assumptions made in the model development for reasons
of fccactability? Are the mathematical assmfiptiotts
consistent with the physical processes and are the
derivations correct?
Functional adequacy - Is the Information provided by the methodology
appropriate to the decisions made with the Information?
Is there a good balance between the scope and detail of
the information? Do users have confidence in the
technique?
In essential point to consider is that these different measures
of adequacy cannot be applied in isolation. The model evaluation criteria.
must be applied together.
Physical Adequacy
The basis of the probabilistic methodology is that probability
distributions of upstream flow/concentration and urban runoff flow/concentra-
tion are transformed to a firobability distribution of downstream concentra-
tion. The transformation is described by a simple mass balance. The major
question regarding physical adequacy of this model are: how realistic are
the data needs of the model and how realis tic is the mass balance as a
representation of the receiver?
The data needs are the statistics (mean» standard deviation)
of the ups tream flow and concentration and the urban runoff flow and
concentration. These data requirements are minimal, and it is safe to say
that alternative evaluation methodologies would require much more data.
Hence, these data needs are judged to be quite realistic.
-------
-7-
1 • A potential problem exists in the estimation of the statistics of
urban runoff flow/concentration. The methodology relies on deriving these
statistics from field measurement. The evaluation of the methodology mist
ask whether tdieae field measurements are generally available- If not, can
such statistics be generalized from data at other measurement sites? Even
if field data are available, they are applicable to the current condition
of the catchment. If alternative engineering or management measures for
urban runoff control are to be evaluated by the methodology, then such
Measures would change the statistics of urban runoff flow/concentration.
To use the probabilistic methodology far the evaluation of alternative con-
trol measures, a satisfactory method of predicting the statistics of urban
runoff flow/concentration resulting from these measures must be found. Here
again, a reasonable conclusion is that methods for predicting these statis-
tics may not be perfect but.other methodologies with the same objective would
encounter at least similar if not greater problems.
The second question regarding the physical adequacy of the method-
ology is the adequacy of the mass balance as a representation of the receiver.
The two major problems arising from this simplification are the absence of
reactions in the receiver and the assumption of instantaneous and perfect
mixing. Both problems have been identified in the memoranda of Southerland
and Barnwell.
Three approaches can be taken regarding the absence of reactions
in toe receiver: (i) build appropriate reaction into the model, (ii) re-
strict tie use of the model to "conservative" substances and (iii) impute
the quantitative effects that reactions would have on the model results based
on ao reactions. The first approach may indeed be possible; however, it may
be undesirable in its detraction from the simplicity of the modal. The
-------
-8-
second approach may be undesirably by virtue o£ nothing being truly conserva-
tive in this seme. Substances which may be conservative in the water column
may be reactive with respect: to sediment or biota. The third approach may
have merit if the effects of the reaction are generally known and particularly
if they are monotonic. For example} it may be sufficient to know if the
model results are always conservative and if so, than to what extent are-
they conservative.
A similar approach may be taken regarding tie assumption, of
instantaneous and perfect mixing. It may be adequate to live with the assump-
tion with the understanding of which side the error lies and the extant of
the error.
These remarks on. the physical adequacy of the methodology must be
taken in the context of the function of the methodology. It is reiterated
that the physical adequacy cannot be judged in Isolation from die role of
the information produced by the aioctels in decision, making. Ie is my opinion
that the proposed probabilistic methodology Is appropriate, In terms of its
physical adequacy, for its intended purposes in this sense.
Mathenatieal/5 tatlatical Adequacy
The general questions concerning the mathematical/statistical
adequacy of the probabilistic methodology are as follows:
i) are the derivations correct in a mathematical sense?
ii) are the distribution assumptions of the input (Q »C ,Q .C^) and $
appropriate?
iii) are there drawbacks associated with the numerical solution requirements
of part of the methodology?
iv) are there other model outputs required (such as confidence llmts on
predicted MRl's as suggested in the Kahn memorandum)?
-------
-9-
These questions are addressed in the same order below.
1 have found the derivations to be correct except for the deduc-
tion of equations (34) and (35) from equation (33) on page 3-13 as noted in
the review by Small. However, the estimation for MEI in equation (36) is
correct.
• The assumption of distribution function form is a constant
problem* Since no real empirical distribution obeys any theoretical dis-
tribution, the selection, of theoretical distribution form Is always a
compromise. Although the issues raised by Kahn are quite legitimate, there
are two-factors in favor of the log-normal distribution assumption:
(i) many natural processes are well-approximated by log-normal distributions
and (ii) the field data in the report strongly suggest log-normal distribu-
_tions. Since I would find it difficult to justify alternative distributions*
I would support tiie log-normality assumptions concerning Q ,C ,Q and C in
s s r r
lack of evidence to the contrary.
The assumption of the log-no reality of $ is more obscure. The
assumptions appear to have no significant disadvantage based on the results
presented in the report. However, I would agree that some experimentation
with other distribution function forms for $ would be useful,
The two computational techniques offered by the methodology
consist of an approximate method of moments which yields closed-form
analytical solutions and a numerical integration procedure which requires
fewer simplifying assumptions. I am confident that a user of the methodo-
logy could rapidly gain an appreciation for the limits of applicability of
the approximate method by running both procedures in tandem. I would think
that the closed-form solutions would be appropriate for stany applications.
Even for those applications where this is not the case, the numerical
-------
-10-
precedures are not overly complex. I would judge diem, to be an order of
magnitude less complex than operating simulation models as suggested in the
Roesner memorandum.
Since there is uncertainty in parameter estimates osed in the
methodology j additional model output reflecting this uncertainty would be
useful. The calculation of confidence intervals on ME1 as suggested by
Kahn would undoubtedly enhance the utilization of the methodology.
Functional Adequacy
The general questions concerning the functional adequacy of
the probabilistic methodology are as follows:
(i) does the information provided fay the probabilistic methodology strike
a good balance between an adequate representation of the processes
being modelled and an adequate scope for the problems being addressed,
and as such does the methodology lend itself to a good decision
support model?
(ii) will users adopt die methodology?
These questions are addressed in the same order below.
I believe that the methodology Is an appropriate model for
decision support. The results of the methodology are quite credible and
the validations contained in the report are conviiiciiig. The information
contained in Figures 4-6a & b is not only extremely useful but also easily
attained. I would think that information of this kind would go a long way
to -improving decision making in the practice of urban runoff control.
The probabilistic methodology is not as familiar to most
analysts as more conventional approaches such as simulation; however, I
believe that this should not act as a deterrent to the acceptance of the
-------
-il-
methodology. This process of user acceptance could be greatly enhanced with
"side-by-aide" comparisons with simulation model output. As the profession
adapted to simulation modelling from pencil and paper methods, so it should
adapt to analytical models,
ALT1BNATI7ES TO THE PROBABILISTIC METHODOLOGY
Hie only serious alternative to analytical models such as those
contained In the probabilistic methodology is continuous simulation, as
indicated in the Eoesner memorandum. • Soesner quite properly points out that
simulation models have the advantages of not requiring many o£ the simplifying
assumptions of analytical models and being able to use historical meteorc-
logic data directly. However, dies £ features also represent disadvantages.
Continuous simulation models require massive computer code and massive amounts
of meteorologic data. This has meant mainframe computers and magnetic tapes
of data ^shich act ma significant deterrents to continuous simulation* Models
such as STOSM have been available for about two decades, yet little vide-
spread practical use has been made of them. Although such models will become
more accessible when ported to microcomputer systems, analytical models will
still have advantages.-
In the future, analytical and simulation models should be used
together. For the present, although continuous simulation is not widely-
used, this form of analysis is essential for Intelligent decision making in
urban runoff control practice. Analytical models such as those proposed ia
tie probabilistic methodology can fill this void.
------- |