United States
Environmental Protection
Agency
Environmental Sciences Research
Laboratory
Research Triangle Park NC 27711
Research and Development
EPA-600/S3-83-108 Dec. 1983
&EPA Project Summary
Synthesis of the Rural
Model Reviews
D. G. Fox, D. Randerson, M. E. Smith, F. D. White, and J. C. Wyngaard
The U.S. Environmental Protection
Agency (EPA) has undertaken an inde-
pendent review of eight rural diffusion
models, two of which were developed
by the EPA; the remaining six were
submitted to the EPA for approval by
outside agencies and consulting firms.
In the first phase of the review process,
EPA arranged with an outside contrac-
tor to calculate and tabulate a uniform
set of statistics for the eight models to
provide reviewers with a consistent set
of measures for evaluating model per-
formance.
Under a cooperative agreement with
the EPA, the American Meteorological
Society (AMS) conducted the scientific
review of the rural diffusion models.
Seven independent reviewers evaluated
each model using scientific and tech-
nical information obtained from User's
Guides and the statistical performance
data developed for the EPA. This report
presents the results of the scientific
review as summarized by the AMS
Steering Committee, and contains some
of the views and recommendations of
the AMS Committee based on the
review process and performance evalu-
ations.
This Project Summary was developed
by EPA's Environmental Sciences Re-
search Laboratory, Research Triangle
Park. NC. to announce key findings of
the research project that is fully docu-
mented in a separate report of the same
title (see Project Report ordering in-
formation at back).
Introduction
In carrying out its regulatory responsi-
bilities, the EPA over the past decade has
used mathematical models extensively to
establish compliance (or non-compliance)
with air quality rules. This has led to
debate about the credibility of EPA-
approved models. Acting as an impartial
expert, the American Meteorological
Society (AMS) has provided technical
evaluationsof airquality modeling. In the
1981 report, Air Quality Modeling and the
Clean Air Act, the AMS recommended
that EPA conduct a scientific review of all
models listed in their modeling Guideline
and of those models under consideration.
After preliminary negotiations, the AMS
entered into a cooperative agreement
with the EPA in August 1981 to scientifi-
cally review 10 "rural" point source
models being considered by the EPA.
Since three of these models produce
identical results, the number was reduced
to eight. The purpose of the AMS report
was to summarize the reviews and to
describe how they were accomplished.
The AMS selected a steering committee
of five (the authors of this report), who
developed a set of ground rules and a list
of questions for the reviewers. Seven
reviewers, all experts in "rural" modeling,
were chosen. They were provided user's
guides for the models and independently-
derived performance measure statistics
comparing modeling calculations with
field data. These statistics were intended
to duplicate those recommended in D. G.
Fox's 1981 report, "Judging Air Quality
Model Performance,"(Bull. Am. Meteorol.
Soc., 62:599-609). The models were
applied using 2 years of data to calculate
1-, 3-, and 24-hour average SOaconcen-
trations for the Chfty Creek Power Plant,
located along the Ohio River in southern
Indiana, a region of low ridges and rolling
hills. The plant used three 208-meter
stacks. Emission rates of SOa were calcu-
-------
lated on an hourly basis. Hourly averages
of SOz concentrations were available for
six monitors located 3-15 kilometers from
the plant. Onsite winds were measured at
60 meters and temperatures, at 10
meters. Stability was calculated by
Turner's method, using National Weather
Service (NWS) surface observations at
Cincinnati (20 kilometers away). Mixing
heights were calculated and interpolated
accordi ng to the CRSTR model preproces-
sor, using NWS radiosonde measure-
ments at Dayton (160 kilometers away)
and surface observations at Cincinnati.
Results and Recommendations
The summary covers the scientific
evaluation but does not include the sta-
tistical performance measures. The latter
will be published separately.
Before listing the main points of the
reviewers' comments, a brief discussion
of reasonable expectations with respect
to model performance is in order. This is
particularly appropriate, since both the
statistics and the reviews provide a
distinctly negative tone. Dispersion
models predict the ensemble average
(i.e., the most likely) dispersion. However,
dispersion measurements, repeated
under the same mean conditions, differ
from one realization to another because
of inevitable and unresolvable differences
in the details of both initial conditions and
the dispersion process itself. Predicted
and measured dispersion will therefore
necessarily be different. The difference or
scatter will never vanish, but presumably
it can be decreased by better input data,
better model physics and better calcula-
tion techniques. Even a perfect model
with ideal input data will not agree with
data in individual realizations. The re-
viewers for the most part took a more
fundamental position, namely that a good
model should show a good correlation
with observations paired in space and
time, even though this expectation would
probably be somewhat unreasonable
even with good input data. It is certainly
unreasonable with the inadequate input
data available for this study. Therefore,
the AMS Committee does not believe that
the scientific community has necessarily
failed to provide suitable models to repre-
sent rural dispersion, but rather that it is
impossible to determine at present wheth-
er it has or has not.
The main points stressed by the re-
viewers may be summarized as follows:
A. Both in concept and performance the
models are quite similar to each
other. Furthermore, the options
chosen by the developers for this
evaluation tended to emphasize sim-
ilarity with the approved EPA models,
CRSTER and MPTER. The statistics
did reveal some differences, among
the models, but these variations are
neither consistent nor important
compared with the overall poor per-
formance.
B. The reviewers were nearly unani-
mous in their contention that the
models do not reflect the most mod-
ern and appropriate scientific think-
ing. The most serious criticisms
involve the following:
1. the Pasquill-Gifford diffusion
parameters used in all but one of
the models are inappropriate for
many applications, especially for
the tall stacks modeled in this
evaluation;
2. the failure to employ recent
developments in convective scal-
ing in unstable conditions;
3. the crude treatment of plume
behavior with respect to the
inversion capping the mixed
layer; and
4. the speculative adjustment of the
equations for source-terrain
height differences.
C. Despite the fact that the quality of the
data base chosen for the statistical
study was excellent, not all of the
relevant parameters were measured.
In fact, it is virtually certain that
deficiencies in the data would have
made it impossible to identify even a
perfect model. The main problems
involved the lack of suitable informa-
tion concerning the wind and turbu-
lence at and above stack height, and
the uncertainty regarding the depth
of the mixed layer. All of this informa-
tion was inferred from remote surface
and upper air measurements using
the CRSTER preprocessor system.
D. None of the eight models showed
much skill in predicting the measured
SOg concentrations at the same
location and time. On the contrary, in
the space-time pairing the predic-
tions explained only about 10% of the
variance of the observations on 1-,
3-, and 24-hour time scales.* Five of
'The AMS Committee members noted, however, that
the comparison between observed and predicted
frequency distributions compared well in the upper
percenti/es, and that good space-time comparisons
should not necessarily be expected.
the seven reviewers felt that this lack
of fundamental correspondence be-
tween predictions and measured
concentrations rendered the remain-
der of the statistical package almost
meaningless.
The AMS Committee concludes that:
A. The models performed similarly, and
there is no reasonable basis for
choice among them.
B. It is apparent, as it has been in past
attempts to validate models of this
type, that comprehensive data are
not available. EPA should devote
vigorous effort to foster the develop-
ment of such data, both within its
own program and by the encourage-
ment of others. It should be noted
that it is unlikely that a single study
will provide sufficient information.
C. Whether justified or not, there is a
strong and pervasive impression
among the reviewers that EPA tends
to discourage models that do not bear
a very close resemblance to the
CRSTER - MPTER systems. Modeling
innovations should be actively en-
couraged by all concerned.
D. Future reviews should be encour-
aged. A detailed list of recommenda-
tions for future reviews appears at
the end of the full report, but the
Committee feels that future reviews
will be more meaningful only if:
1. they can be based on suitable
meteorological and air quality
data;
2. the statistical evaluation can be
reduced to a reasonably digest-
ible package; and
3. the available models can be
tested fully enough to determine
whether available modeling op-
tions and innovations have merit.
-------
D. G. Fox. D. Randerson, M. E. Smith, F. D. White, andJ. C. Wyngaardare with the
American Meteorological Society, Boston, MA 02108.
K. L. Demerjian is the EPA Project Officer (see below).
The complete report, entitled "Synthesis of the Rural Model Reviews,, "(Order No.
PB 84-121 037; Cost: $10.00, subject to change) will be available only from:
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone: 703-487-4650
The EPA Project Officer can be contacted at:
Environmental Sciences Research Laboratory
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
-------
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
BULK RAT
U.S. POSTA
PAID
Cincinnati, 0
Permit No. G
Official Business
Penalty for Private Use $300
------- |