United States
Environmental Protection
Agency
Atmospheric Sciences \
Research Laboratory ~-
Research Triangle Park NC 27711 ^
Research and Development
EPA/600/S3-85/060 Sept. 1985 '/»
&EPA Project Summary
Summary of Complex Terrain
Model Evaluation
Fred D. White, Jason K. S. Ching,
Robin L. Dennis, and William H. Snyder
The U.S. Environmental Protection
Agency (EPA) has undertaken an inde-
pendent review of eight complex ter-
rain dispersion models, including some
developed by the EPA and others sub-
mitted to the EPA for approval by out-
side agencies and consulting firms. In
the first phase of the review process,
EPA arranged with an outside contrac-
tor to calculate and tabulate a uniform
set of statistics for the eight models to
provide reviewers with a consistent set
of measures for evaluating model per-
formance. Three EPA staff reviewers
then evaluated each model using scien-
tific and technical information obtained
from the User's Guides and the statisti-
cal performance data developed for the
EPA.
Under a cooperative agreement with
the EPA, the American Meteorological
Society (AMS) summarized the scien-
tific reviews of the complex terrain dis-
persion models. This report presents
the results of the three EPA scientific
reviews and the summary by the AMS
Steering Committee, and contains
some of the views and recommenda-
tions of the AMS Committee based on
the review process and performance
evaluations.
This Project Summary was devel-
oped by EPA's Atmospheric Sciences
Research Laboratory, Research Triangle
Park, NC, to announce key findings of
the research project that is fully docu-
mented in a separate report of the same
title (see Project Report ordering infor-
mation at back).
Introduction
In carrying out its regulatory respon-
sibilities, the EPA over the past decade
has used mathematical models exten-
sively to establish compliance (or non-
compliance) with air quality regula-
tions. This has led to debate about the
credibility of EPA-approved models.
Acting as an impartial expert, the Amer-
ican Meteorological Society (AMS) has
provided technical evaluations of air
quality modeling. In the 1981 report. Air
Quality Modeling and the Clear Air Act,
the AMS recommended that EPA con-
duct scientific reviews of all models
listed in their modeling Guideline and of
those models under consideration.
This complex terrain model evalua-
tion is the third in the series conducted
under the cooperative agreement be-
tween the AMS and the EPA. Like the
earlier reviews of rural and urban dis-
persion models, this evaluation in-
cluded three distinct tasks, a perform-
ance evaluation in which model
predictions and field data were com-
pared, a set of independent peer re-
views, and a brief summary of the eval-
uation composed by the AMS members
of the Steering Committee.
Eight models were included in the
evaluation: COMPLEX I, COMPLEX II,
4141, PLUME 5, RTDM, SHORTZ, COM-
PLEX/PFM, and IMPACT. Five of the
models are basically simple Gaussian
models, using minor adaptations to
deal with some aspects of complex ter-
rain flow and dispersion. These adapta-
tions include modified terrain receptor
height adjustments to account for verti-
cal plume motion over obstacles, en-
hanced lateral dispersion under certain
circumstances, adjustments to account
for wind direction shear, and limitation
of plume approach height on impaction
to some minimum value. RTDM and
COMPLEX/PFM are more contemporary
in a scientific sense, the latter using po-
-------
tential flow adjustments in the wind
field for part of the calculations. IMPACT
is the only model to simulate the three-
dimensional transport and diffusion
field.
Two data sets were used for the
model evaluation. The 1980 Cinder
Cone Butte data were obtained during
the EPA Complex Terrain Model Devel-
opment program. The key terrain fea-
ture is a nearly axisymmetric cone ris-
ing from a nearly flat plain, and the
concentration data and meteorological
measurements were quite detailed. The
sources, however, consisted of passive
releases of tracer gases and smoke
which, while they provided very reliable
concentration data, did not represent
typical industrial sources in which large
buoyant plumes are common. Thus,
this set of data is unrepresentative in an
important way.
The second data set, obtained at the
Westvaco plant site between 1979 and
1981, represented a much more typical
industrial problem, but the concentra-
tion and meteorological data obtained
were limited. Furthermore, one could
hardly describe this particular site as be-
ing ideal from the standpoint of a vali-
dation study. The plant is situated in a
rather winding valley, a configuration
which distorted both the flow trajecto-
ries and the diffusion from an idealized
complex terrain problem.
Results and Recommendations
The AMS summary covers the scien-
tific evaluation but does not include the
statistical performance measures. The
latter have been published separately.*
In the opinion of the AMS Steering
Committee, the three EPA reviewers
presented valuable, objective studies of
the models and their performance. They
mirrored the reactions of the reviewers
of the rural and urban models and the
AMS Committee members in being dis-
appointed in the technical quality of the
models, and in concluding that a mas-
sive statistical analysis is not the best
way to analyze model performance.
This group of reviewers and the Com-
mittee members believe that future
evaluations should stress understand-
ing of the physics of the models and
examination of their performance in
specific case studies.
The main points stressed by the re-
viewers may be summarized as follows:
A. None of the models can be de-
scribed as up-to-date scientifically.
Seven of the eight models still utilize
the Gaussian diffusion kernel with
simple modifications to account for
the terrain effects.
B. This performance evaluation was
unlike either the rural or the urban
reviews in that one model, RTDM,
exhibited much better performance
statistics than the others. In both
data sets, this model predicted con-
centrations that were quite close to
the observed, on average, although
there were indeed large deviations
between individual observations
and predictions.
C. Overprediction is the most common
problem with the models, and fre-
quently the overprediction is suffi-
ciently large to make regulatory
application difficult. This overpredic-
tion seems to be associated largely
with plume impingement on terrain
being predicted to occur where it
does not, or in calculating excessive
concentrations when it does im-
pinge.
Conclusions
The AMS members of the Committee
feel that the following are the key find-
ing of the evaluation.
A. The models and the data sets
against which they were compared pro-
vide a very limited representation of the
full variety and range of the flow pat-
terns and diffusion situations actually
occurring in complex terrain. The evalu-
ation has provided limited data on pre-
dicted and observed concentrations
that might be expected on nearby
slopes and crests.
B. The voluminous performance
measures developed in this study are
difficult to digest. Both the reviewers
and the Committee would prefer the
comparison against the data to have
provided more information about what
the models were doing in a limited
number of specific situations.
C. It is highly unlikely that complex
terrain problems can ever be treated in-
telligently with a set of "cookbook"
models. Flow and diffusion in complex
terrain, are, and will continue to be,
complex.
'Evaluation of Complex Terrain Air Quality Simula-
tion Models, EPA-450/4-84-017, June 1984).
-------
Frank D. White is with American Meteorological Society. Boston. MA 02108;
and the EPA authors. Jason K. S. Ch/ng, Robin L. Dennis, and William H.
Snyder, are with Atmospheric Sciences Research Laboratory, Research
Triangle Park, NC 27711.
Francis A. Schiermeier is the EPA Project Officer (see below).
The complete report, entitled "Summary of Complex Terrain Model Evaluation,"
(Order No. PB 85-236 891 /AS; Cost: $14.50. subject to change) will be
available only from:
National Technical Information Service
5285 Port Royal Road
Springfield. VA 22161
Telephone: 703-487-4650
The EPA Project Officer can be contacted at:
Atmospheric Sciences Research Laboratory
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
Official Business
Penalty for Private Use $300
EPA/600/S3-85/060
OC00329 PS
U S ENVIR PROTECTION AGENCY
REGION 5 LIBRARY
230 S DEARBORN STREET in,
CHICAGO IL 60604
------- |