* OAQPS
'• Office of Air Quality Planning and Standards
"t i" u s Erivirorimerital Protection. Agency'
COMPENDIUM OF REPORTS
PEER REVIEW PROCESS
For
AERMOD
February 2002
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards
Emissions, Monitoring and Analysis Division
Research Triangle Park, North Carolina 27711
From the
-------
Table of Contents
I. BACKGROUND
II. THE EPA'S INSTRUCTIONS TO THE PEER REVIEW COMMITTEE.
IE.. SUMMARY OF THE PEER REVIEW COMMITTEE FINAL COMMENTS.
REPORT I. THE FIRST PEER REVIEW REPORT.
REPORT H. AERMIC' S RESPONSE TO THE FIRST PEER REVIEW REPORT.
REPORT HI. THE FINAL PEER REVIEW REPORT.
REPORT IV. THE FINAL RESPONSES FROM AERMIC.
-------
COMPENDIUM OF REPORTS
From the
PEER REVIEW PROCESS
For
AERMOD
I. BACKGROUND
In 1991, the American Meteorological Society (AMS) and the U.S. Environmental
Protection Agency (EPA) initiated a joint effort to develop a vastly improved air quality model. A
committee was formed (AERMIC [the AMS/EPA Regulatory Model Improvement Committee] )
to upgrade the current models which were developed nearly two decades ago. Much progress has
occurred in the scientific knowledge of atmospheric turbulence and dispersion and so a need had
been recognized to update the regulatory air quality models based on more up-to-date science.
The goal of such an update would be to improve the accuracy of these models.
AERMIC chose to focus on the development of a new model, AERMOD (AERMIC's
Dispersion Model), for estimating the near-field concentrations from a variety of stationary
sources. That is, AERMOD is designed to handle the same source types currently addressed with
the EPA recommended Industrial Source Complex Model (ISC3), including sources located in
various terrain settings.
AERMOD, along with its associated preprocessors (AERMET - the meteorological data
preprocessor, AERMAP - the terrain data preprocessor), was submitted to the EPA's Office of Air
Quality Planning and Standards for consideration as a regulatory dispersion model. Part of this
process was the technical review of the model by experts outside of the Agency and separate from
AERMIC, i.e. a peer review of the new air dispersion modeling system. This compendium
provides the documentation of that peer review process and the conclusions drawn from the
external review of the model.
This peer review process of the AERMOD modeling system had 5 distinct steps. Each
step generated a corresponding document as listed below (in chronological order):
1. The scope of work (instructions) given to a contractor who, in turn, selected and
managed the peer review committee (this step started the peer review process and defined
the charge or mission to the peer review committee);
-------
2. The first report from the peer review committee;
3. The response from AERMIC to the peer review committee's comments;
4. The final report from the peer review committee after reviewing the updated
documents; and,
5. The final responses from AERMIC to the peer review committee's final report.
All five documents are bundled together (and provided in a compressed format) for the reader's
convenience. For brevity, only the conclusions and the general comments are included in the
reports in this compendium. The detailed or line-by-line comments and the draft versions of the
reports that were reviewed by the peer-review committee are provided in the docket which
supports the EPA proposing to use AERMOD for regulatory applications (docket number A-99-
05).
In addition, the Peer Review Committee's final summary and conclusions were copied from
Report HI and included immediately following the section listing of instructions given to the
contractor and the Peer Review Committee. This section was created for the reader's
convenience so that this important information could be easily found up front in the
compendium.
-------
II. mi EPA'S INSTRUCTIONS TO Till PEER REVIEW COMMITTEE
CONTRACTOR.
Task 1. Establish a list of qualified peer review candidates.
The contractor shall identify a list of qualified candidates to provide a technical review of
the AERMOD air dispersion. The candidates qualifications shall including the following:
1. Considerable experience and understanding in the formulation, application,
performance of air quality dispersion models for short-range (near-field) situations (i.e.
distances from sources generally less than 30 to 50 km).
2. Knowledge of the current state-of-the-art scientific understanding of turbulence and
dispersion in the planetary boundary layer for flow in both flat and complex terrain
settings.
3. Familiarity and experience with existing regulatory dispersion models and their
applications and the regulatory framework under which they are commonly used.
Examples of scientists that AERMIC believe meet the above qualifications include: Bruce Turner
(Trinity Consultants), Steve Hanna (Chief Editor, Journal of Applied Meteorology), Helga
Oleson (Danish Air Pollution Laboratory), Roger Brower (Versar), Bruce Egan (Woodward and
Clyde), Mark Garrison (Engineering Associates), Dick McNider (University of Alabama), Robert
L. Miller (Oak Ridge National Laboratory). Steve Hanna recently chaired the committee for the
Atmospheric Modeling Division program peer review for EPA/ORD.
Task 2. Select the peer review committee.
The contractor shall set up a peer review committee of 3 individuals from the list of
qualified candidates. The contractor shall call the members of the list of qualified candidates and
determine their availability and their interest in participating in the peer review process. The
contractor shall continue to call list members until the committee of 3 members is set. If the
contractor cannot find 3 qualified people to serve on the committee within the schedule alloted
for this activity, the Work Assignment Manager (WAM) shall be contacted immediately and be
informed of the problem and determining the next steps to complete the selection process.
The contractor shall establish one peer review committee member as the chairperson.
This chairperson shall coordinate and run the peer review meeting describe below in Task 4.
Task 3. Provide the necessary documents to the peer review committee.
-------
The contractor shall provide to the peer review committee, copies of all the necessary
background information as supplied by the WAM. The purpose of the peer review is to obtain
the committee's comments on the formulation, documentation, evaluation results, and user
friendliness of AERMOD and its companion preprocessor programs: AERMET, the
meteorological preprocessor, and AERMAP, the terrain and receptor preprocessor. The
contractor shall contact the WAM to obtain a complete listing of this background information;
but, it shall contain, at a minimum, the following documents:
1. 2 AWMA papers describing the earlier versions of the AERMOD;
2. The Phase I and Phase II test reports;
3. The user guides (3: [AERMET, AERMAP, and AERMOD]);
4. The model formulation document;
5. A computer copy of the AERMOD, AERMET, and AERMAP models (the
contractor shall identify the preferred method of computer model transfer
for each committee memeber—the WAM will provide assistance to the
contractor, if necessary, to complete this part of the materials transfer);
and,
6. A detailed list of questions that the peer review committee shall address
during this process.
A copy of each of these documents will be supplied by the WAM. Based on current planning,
these background documents are scheduled to be completed by March 1, 1998 (the AWMA
papers are currently available). The contractor shall instruct the peer review committee to read
all the supplied background documents and focus their review on responding to a list of specific
questions. The list of questions provided to the peer review committee shall include, at a
minimum, the following:
1. Model formulation
1.1. As a steady-state, plume-based, regulatory model, does AERMOD represent
the state-of-the-art in its handling of boundary layer turbulence and dispersion?
1.2. Within the context of regulatory dispersion models in the US, does
AERMOD represent significant scientific advances over ISC3?
1.3. What do you think are the most important scientific advancements of
AERMOD?
1.4. Are there any areas or features of AERMOD in which an improved
formulation or treatment would be desired? If so, please discuss whether you
think the revised treatment would lead to better performance and how much.
2. Documentation
2.1. Is the current organization of the Model Formulation Document and User's
Guides appropriate or would an alternative be desired?
2.2. Is the presentation of the model clear and explanatory? Please note any
specific sections of the documentation that were unclear or confusing.
-------
2.3. Is the documentation sufficient for an typical ISC-type user to guide them in
the use of the model and its preprocessors. Do you think training sessions would
be particularly useful?
3. Evaluation and Performance
3.1. How do you rate the performance of AERMOD relative to ISC3 and the other
models included in the evaluation exercises?
3.2. From a model design, scientific, and performance perspective, what
comments do you have on the replacement of ISC3 with AERMOD for regulatory
applications?
3.3 When considering the eight data bases used to evaluated the model, would
additional evaluation of AERMOD be desirable?
4. User Friendliness of Entire Package
4.1. Please comment on the user friendliness of the entire AERMOD package, in
terms of the documents and/or the model codes.
4.2. Please list any areas where you feel improved user friendliness is needed.
(Although the AERMIC committee has focused its resources on development of
the model algorithms themselves, the committee recognizes the usefulness of
graphical user interfaces and is aware that several private contractors are
developing ones for the AERMOD system. These will clearly enhance the user
friendliness of the model).
[Later, two more questions were forwarded to the peer review contractor]
Additional EPA Question 4.1 Dated 3/18/98 - Is the AERMOD approach to modeling urban
sources scientifically sound and state of the art?
Additional EPA Question 4.2 Dated 3/24/98 - Do the building downwash algorithms within
AERMOD represent the current state of science and are these algorithms appropriate to
regulatory applications?
-------
III. SUMMARY OF THE PEER REVIEW COMMITTEE FINAL COMMENTS.
For the reader's convenience, the section below was copied directly from Report HI. This
important information has been included near the beginning of the compendium to help locate
and read the final conclusions of the Peer Review Committee.
"We believe that AERMOD represents a significant improvement over ISC3.
There are many new state-of-the-science concepts and approaches in AERMOD, including
a unique "interface" that creates complete temperature, wind and turbulence profiles from
even the barest minimum input data. However, we feel that, because of these new
approaches, AERMOD is more likely than simpler models like ISC3 to need an extended
break-in period when its use in routine applications can be thoroughly tested. It is worth
noting in this regard that all of the AERMOD evaluation data bases (except for Prairie
Grass) involved tall, non-downwashed, highly buoyant power plant stacks (the shortest
stack in the group was 84 meters in Indianapolis). The vast majority of ISC3 applications
involve modest stacks with modest buoyancy flux values, most of which are subject to
some degree of aerodynamic downwash, as well as area and volume source configurations
which were not evaluated or tested (at least not in the documentation provided) to
determine how AERMOD predictions compare to predictions using ISC3.
"We are concerned about the accuracy of the concentrations predicted by the
downwash algorithm in AERMOD, since the same downwash algorithm is used in both
ISC3 and AERMOD. Considering that something has to be incorporated, it seems alright
for the present to include the convoluted H-S and S-S procedures that are currently
incorporated into the ISC3 models with the simplification that variances will be added
rather than using virtual sources. Then if the PRIME research project makes an
appropriate case for alternate procedures, these can be included in the next AERMOD.
Hopefully, that would be a substitution, not a melding of H-S, S-S and PRIME.
"In the course of applying AERMOD to many different sources, in many different
settings, with many different meteorological data bases, users may discover aspects of
AERMOD that would be desirable to change. It is possible to minimize the number of
situations where this would occur by conducting a thorough, exhaustive, independent set
of sensitivity tests aimed at understanding the underlying reasons for model performance
and the interrelationship of model components, rather than just the end results of the
model. Such a thorough analysis is not likely to happen in the near future, however, and it
therefore might be appropriate to allow for an interim period when AERMOD can be
accepted as a "refined" model but that its use would not be required. This would be
especially appropriate since some applications cannot be correctly handled by AERMOD
(e.g., those involving deposition) and some algorithms (e.g., downwash) are likely to
experience additional changes in the near future.
-------
"Our basic conclusion is that AERMOD is ready to be proposed as a replacement
to ISC3. However, it would be a mistake to treat AERMOD as "finished" and not
needing any further evaluation or dialogue regarding its performance and its use as a
routine model. It is acknowledged by AERMIC that model changes, involving downwash
and deposition in particular, are going to be made in the next year or two. We suggest that
the period during which these additional changes are being evaluated and implemented
could be regarded as an interim period where AERMOD can be used as a "refined" model.
The use of AERMOD could be conditioned on developing site-specific information
regarding its performance, possibly relative to ISC3 and the performance of the model's
interface in terms of generating meteorological profiles. After an interim approval period,
the information and model improvements (if any) generated through these comparisons
and evaluations could be assessed. An advantage of the interim approval period would be
the generation of a wealth of information about the model and its performance and use in
the real world."
-------
REPORT I. HI! FIRST PEER REVIEW REPORT (dated 4/23/98)
1. Introduction
A Peer Review Panel has been assembled by the EPA in order to review the various documents
produced by the AMS/EPA AERMIC group on their new AERMOD dispersion model. The Peer
Review Panel consists of Dr. Steven Hanna (chairman) of George Mason University, Mr. Mark
Garrison of ERM, Inc., and Mr. Bruce Turner of Trinity Consultants, Inc. Administrative
support to the Peer Review Panel is provided by Mr. Edward Carr and his colleagues at
SAI/ICFKAISER. The EPA is proposing to possibly replace the ISCST3 regulatory model with
the new AERMOD dispersion model, and asked to panel to provide specific technical comments
on the documents as well as to provide answers to a set of questions concerning whether
AERMOD is ready for use as a regulatory model.
After the Peer Review Panel was set up in mid-March 1998, a group of AERMOD documents
was delivered to each member. Page-by-page specific comments on seven of these documents
are given in Appendices I through VII of this report. These seven documents included one report
describing the technical formulation, three reports describing model evaluations, and three user's
guides. It should be noted that several "older" conference papers were included in the group of
documents sent to the Panel but were not reviewed because they describe previous versions of
the model and outdated evaluations.
The Panel was also asked to address a set of 12 general questions related to whether AERMOD
was ready for use as a regulatory model. Section 2 contains our responses to these questions.
Two conference calls were held on 2 and 3 April between the Panel, the AERMIC group, and
SAI staff. About 12 persons participated in these calls. The 2 April call lasted about three hours
and the 3 April call lasted about 1 V2 hours. The group discussed the 12 general questions plus a
set of specific questions provided by the Panel to the EPA before the calls. In addition, we
discussed several details concerning the seven individual reports.
During April, the panel has used e-mail communications in order to generate the current report.
Several draft versions of sections have been prepared, reviewed by the Panel members, and
revised. The comments in this report represent a unanimous viewpoint.
On 17 April, the EPA asked that the Panel provide a two to four page summary of our
conclusions and recommendations. That information is provided in Section 3.
2. Answers to General Questions asked by EPA
-------
REPORT I l lll FIRST PEER REVIEW REPORT
The EPA asked the panel to address 12 general questions related to whether AERMOD was
ready for use as a regulatory model. Our responses to these 12 questions are given below.
EPA General Topic Area 1. Model Formulation
EPA General Question 1.1 - As a steady-state, plume-based, regulatory model, does AERMOD
represent the state-of-the-science in its handling of boundary layer turbulence and dispersion?
Panel Response to EPA General Question 1.1- Within the resources available to AERMIC and
the data available with which to evaluate the model, the AERMOD represents a very good
attempt to incorporate additional knowledge of boundary layer turbulence and dispersion into a
model framework. In addition, instead of just having a number of disconnected algorithms that
independently address different source situations, terrain types, and meteorological conditions,
there has been a significant effort to merge calculations continuously from one situation to the
next. As a result, there will not be large jumps in calculated values in going from one source
scenario, terrain type, or meteorological condition to the next.
EPA General Question 1.2 - Within the context of regulatory dispersion models in the US, does
AERMOD represent significant scientific advances over ISC3?
Panel Response to EPA General Question 1.2 - If the model is run with the minimum of input
meteorological data, the results may or may not be better than the models currently being used
such as ISC3. If, instead, the model is run with a set of input meteorological data that represents
nearly the full complement of data that can be accepted by AERMOD, then one would expect
results to be better. However, at this point no data set that represents a nearly complete data set
has been used to independently evaluate AERMOD. The developmental model testing exercise
included some high quality data sets such as Kincaid, Indianapolis, and Prairie Grass, but a
number of adjustments were made to AERMOD to better fit the observed concentrations. It is,
of course, unknown whether these adjustments, or some of them, will better fit the universe of all
situations or whether these are adjusting away from most situations to just fit the peculiarities of
a specific data set. Since AERMOD is able to include more boundary layer observations than the
ISC3 regulatory model, it would seem that it would represent better science.
The Panel feels that the plume rise and dispersion algorithms in AERMOD represent the state-of
-the-art for plume modeling. Of course, a wealth of Eulerian grid models exists (K-models,
Large Eddy Simulation Models, and Computational Fluid Dynamic models), but these advanced
models require much computer time and therefore are not suitable for calculating concentrations
for the thousands of hours required by the EPA. AERMOD accounts for "continuous stability
classes " via its use of the Monin-Obukhov length, for non-Gaussian vertical velocity
distributions in the convective boundary layer, for lofting and partial penetration of plume, for
-------
REPORT I l lll FIRST PEER REVIEW REPORT
similarity theory in the near-surface stable layer, and for terrain effects through an elegant
simplified approach.
However, the Panel points out that there are several existing models (e.g., OML, ADMS, HPDM,
and SCIPUFF) that also account for the scientific advances mentioned in the previous paragraph.
We feel that the EPA should have looked more thoroughly at these available models and
demonstrated where AERMOD would represent an advance.
EPA General Question 1.3 - What do you think are the most scientific advancements in
AERMOD?
Panel Response to General Question 1.3 - The rational inclusion of straightforward procedures to
handle terrain influences would appear to be an advancement. This is easy to expect as the
treatment of terrain effects in ISC3 is very rudimentary. In contrast, the treatment of terrain in
CTDMPLUS is much more complex and required simplification.
Another scientific advance is the elimination of the use of the six discrete Pasquill stability
classes and its associated complete linkage of vertical and horizontal dispersion. AERMOD
replaces this method with the continuous stability class method related to the use of the Monin-
Obukhov length.
The treatment of strongly convective conditions by the pdf approach, which allows vertical
distributions to be non-Gaussian, should be more realistic. Furthermore, the plume lofting and
penetration model represents a significant advancement.
Since surface releases disperse differently than releases above the ground, the fact that these are
handled differently in the model (via similarity theory) should be a step in the right direction.
As pointed out in our answer to the previous question, we feel that there are other models
available (e.g., OML, ADMS, HPDM, SCIPUFF) which can be considered "scientifically
advanced" and which should have been more thoroughly investigated.
EPA General Question 1.4 - Are there any areas or features of AERMOD in which an improved
formulation or treatment would be desired? If so, please discuss whether you think the revised
treatment would lead to better performance and how much.
Panel Response to General Question 1.4- The development of an advanced building downwash
module by the PRIME project should be followed closely to see if the outputs of that program are
representing improved treatment of building downwash. The current treatment of downwash in
ISC3 (which we believe has been transferred with almost no change to AERMOD) considers
zones of turbulence behind structures to be of the same size for all wind speeds. One would
-------
REPORT I l lll FIRST PEER REVIEW REPORT
expect these zones, which are largely caused by the mechanical turbulence of wind moving past
structures, to be highly dependent upon wind speed. Furthermore, the current treatment in ISC3
allows highest concentrations involving building downwash to frequently be calculated to occur
during stable conditions with light wind speeds, conditions when structure downwash would
actually be expected to be at a minimum. Also, the ISC downwash module does not properly
treat strongly buoyant plumes from short stacks near buildings. The panel recommends that the
results of the PRIME project be considered, as well as the current algorithm in ADMS, which is
based on Britter's studies of building downwash.
The panel believes that the AERMOD approach to urban dispersion requires further development
and evaluation. The current approach produces unstable boundary layers over urban areas at
night. The formulas should be tested with urban boundary layer data, as well as concentration
data.
As stated in our specific review comments, the Panel believes that many of the scientific modules
in AERMOD have not yet been adequately justified using comparisons with data. For example,
the Technical Formulation report (see Appendix I) contains no tables or figures demonstrating
the degree of agreement of the various parameterizations with meteorological or concentration
observations. These new algorithms may indeed represent scientific advancements, but they
have not been demonstrated to the satisfaction of normal peer review criteria.
2. Documentation
EPA General Question 2.1 - Is the current organization of the model formulation document and
User's Guide appropriate or would an alternative be desired?
Panel Response to EPA General Question 2.1 - The organization of the material into the separate
documents for Formulation and for User Guides seems appropriate. In addition, the structure of
the material within each document seems reasonable. Users will be grateful that the familiar
structure of the ISC3 user's guide has been retained wherever possible. It would be useful to
have a clearer recommendation concerning the use of detailed on-site meteorological data such as
vertical profiles of wind velocity, turbulence, and temperature. It would also help to add some
schematic diagrams to explain difficult concepts such as the specification of terrain heights.
EPA General Question 2.2 - Is the presentation of the model clear and explanatory? Please note
any specific sections of the documentation that were unclear or confusing.
Panel Response to EPA General Question 2.2 - Specific comments on each document are given
in the seven appendices.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
Formulation - The document consists of lists of equations with brief discussions of each
equation given in the text. Although the new algorithms appear to represent significant technical
improvements over ISC3, the document would be greatly enhanced by adding justifications for
the many parameterizations (including supporting figures and table), by adding schematic
diagrams to better explain the methodology, and by including discussions of comparisons with
similar state-of-the-art models such as OML, ADMS, HPDM, and SCIPUFF. Additionally, it
would be helpful to have, with most equations, the units of the variables in the equation and the
numerical values of constants such as cp.
User's Guides - There are numerous places in the user's guides where it is indicated that
adjustments can be made to accommodate larger numbers of sources, receptors, etc. It is
indicated that the program then has to be compiled. The panel is not sure all users understand the
process of compiling a program. This is well-explained in Chapter 4 of the AERMOD User's
Guide. We suggest that where compiling is mentioned a reference should be made to this chapter
where the full explanation is given.
There seems to be some minor inconsistencies between the various programs in the AERMOD
series. Redirect symbols are used with the various programs of AERMET but are not used with
AERMOD or AERMAP. This is not terribly significant but causes users to have to be quite
careful (which, of course, is not a bad thing).
The User's Guides seem to explain things well and have just about the right amount of
redundancy so that users can look up information that they need quite easily.
It is suggested that schematic diagrams be added to aid understanding of topics such as terrain
height specification and requirements for meteorological inputs.
EPA General Question 2.3 - Is the documentation sufficient for a typical ISC-type user to guide
them in the use of the model and its preprocessors? Do you think training sessions would be
particularly useful?
Panel Response to EPA General Question 2.3 - This question can best be answered by actually
running the model and seeing if there are areas of difficulty. Presumably this exercise was
carried out by the so-called "beta-testers". There were insufficient funds for the Peer Review
Panel to run the model execution and experiment with various scenarios. Therefore, it is not
possible to answer this question in a complete way. As mentioned above, we expect some
confusion concerning the use of meteorological data when extensive on-site data are available.
We also expect some users to experience problems with the terrain processor due to a lack of
schematic diagrams.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
We would expect that the instructions in the user's guide should be sufficient for most users to be
able to run the models. If it appears that there are places in the user's guides where further
information should be given, a revision to the user's guide (in the way of page changes) should
correct the problem. Therefore, we would not expect it necessary to have training sessions to
teach running the model.
The area where some training may be required in the future is in the sensible use of the model.
With some experience it will be found that increased efforts should be made to provide the
proper meteorological information in order to achieve the best performance of the model. As
experience is gained through both applications of the model and sensitivity testing, it may be
desirable to have training sessions that discuss the importance of the various data inputs and
suggest good ways of obtaining these data.
3. Evaluation and Performance
EPA General Question 3.1 - How do you rate the performance of AERMOD relative to ISC3 and
the other models included in the evaluation exercises?
Panel Response to EPA General Question 3.1- Any model evaluation exercise is limited by the
availability of appropriate data and manpower and funds. Since several components of the model
were "tuned" or calibrated using the five data sets reported as the Developmental Evaluation
Report, the "evaluation" results using these data must be set aside in answering this question. Of
course it is a positive result that AERMOD agrees with these five data sets with reasonable mean
bias. However, it is true that the performance of ISC3 and other models could have been
improved if they were also tuned with these five data sets. The panel recommends that a measure
of model scatter (e.g., geometric variance, VG, or Normalized Root Mean Square Error, NMSE)
be included in the set of performance measures. The panel also feels that a "scientific"
evaluation should include a technical peer review of the model components. Documentation of
the changes that were made to the model during and following the developmental evaluation
should appear in a separate section with a summary table.
The three data sets used in the Independent Performance (Phase II) Evaluation all represent the
running of the model with what seems to be the barest minimum of input data and the evaluation
of the concentration predictions with only a few (6 to 10) routine monitors. The Panel feels that
it is short-sighted to anticipate that most users will run the model with a bare minimum of data.
From the model description, AERMOD would be expected to perform best when furnished data
inputs that have not been routinely available for running the current regulatory models. If an
evaluation with a more complete data set could be included (i.e., one that has vertical profiles of
temperature, wind, horizontal fluctuation and vertical fluctuation), and the statistical results
demonstrated improved estimates from the model, then there would be some incentive for
obtaining improved data inputs even for relatively routine model application.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
Two areas of application that need to be further evaluated are 1) situations involving significant
topography and 2) situations involving a considerable amount of building downwash. Many
future applications of the model, if it is adopted for regulatory use, would be expected to involve
at least one of these two situations. The Cinder Cone Butte, Hogback, and Tracy tracer studies
should be used to further evaluate the performance of AERMOD in terrain (all three data sets
contain extensive meteorological observations and sampling networks. The PRIME data set
should be used to test the building downwash component of AERMOD.
The model has not been independently tested in a scenario with a near-surface source. Several
such tracer data sets from NOAA/ARL could be used for these tests. Furthermore, the detailed
Bull Run power plant tracer data set should be used for further evaluations for tall stack plumes.
The evaluations should be expanded to include other similar models such as OML, ADMS,
HPDM, and SCIPUFF.
EPA General Question 3.2 - From a model design, scientific, and performance perspective, what
comments do you have on the replacement of ISC3 with AERMOD for regulatory applications?
Panel Response to EPA General Question 3.2 - The model design is reasonable, although more
guidance is needed on the use of meteorological input data. The science in the model appears to
represent state-of the-art concepts, but as mentioned above, there is a need to justify the various
constants of parameterization and explain the model components by use of schematic diagrams.
The panel feels that AERMOD has not been adequately evaluated with independent data, since
the three data sets in the Independent Performance Evaluation Report are all "routine" data sets
that do not include extensive onsite meteorological observations or spatial coverage of
concentration monitors. The sources in these studies were not straightforward, either, since all
involved multiple stacks and some involved sources widely spread over the domain.
The panel recognizes that there are other state-of-the-art models already available, such as OML,
ADMS, HPDM, and SCIPUFF. At least one of these models should be included in each of the
new evaluations. For example, the few evaluations of HPDM that were carried out in
conjunction with the Independent Performance Evaluation report suggest that HPDM is
performing as well as AERMOD and therefore should also be considered as an alternate
replacement model for ISC3.
EPA General Question 3.3 - When considering the eight data bases used to evaluate the model,
would additional evaluation of AERMOD be desirable?
Panel Response to EPA General Question 3.3 - The Panel feels that, in reality, only three data
bases were used to evaluate the model. It is necessary to make clear to the readers of the so-
called "Development Evaluation" document that the algorithms in AERMOD were changed
-------
REPORT I l lll FIRST PEER REVIEW REPORT
based upon the initial results obtained with these five data bases. The model predictions were
then compared against the predictions of other models (e.g., ISC3) which did not have the benefit
of any tuning. Therefore, the results with these five data bases must be interpreted in a different
light than the results for the three truly independent data sets reported in the Independent
Performance Evaluation document. And these three independent data sets are similar in that they
all involve routine data bases. No independent evaluations were carried out with detailed tracer
data bases or with source releases near the ground.
Yes, it would certainly be desirable to perform additional evaluation of the model. Comments to
this effect have been made in the Panel's answers to questions above. The areas most in need of
further evaluations are: 1) The use of meteorological data that represent a more complete set
than those for the Phase II Evaluation; that is, profiles of wind, temperature, and vertical and
horizontal fluctuations (the Bull Run data base would be very useful); 2) The use of detailed
tracer data in situations involving complex terrain (the Cinder Cone Butte, Hogback, and Tracy
data bases should be used); 3) The use of observations in situations involving structure
downwash (the PRIME data base should be tested), and 4) The use of observations from sources
near the ground (NOAA/ARL and DOD have several tracer data bases available from Dugway
Proving Ground and the Nevada Test Site).
Additional EPA Question 4.1 Dated 3/18/98 - Is the AERMOD approach to modeling urban
sources scientifically sound and state of the art?
Panel Response to EPA Question 4.1 - The Panel understands that the primary way in which
urban areas influence AERMOD is through the assumption that a surface source of
anthropogenic heat based upon population will cause the nighttime urban boundary layer to be
unstable. Also an increase of the nighttime mixing height dependent upon population is
calculated. Furthermore, roughness lengths will be assigned appropriate to the urban
environment. It is unsettling that these procedures were devised through calibration of the model
concentration predictions with observations at Indianapolis. There was no attempt made to
evaluate the fundamental predictions of heat flux and mixing depth at Indianapolis, even though
those data are available. The panel notes that there are no urban data sets in the three
independent data sets used for the Phase II evaluation. Consequently, there is no independent
evidence that the urban algorithms in AERMOD will match field data.
Additional EPA Question 4.2 Dated 3/24/98 - Do the building downwash algorithms within
AERMOD represent the current state of science and are these algorithms appropriate to
regulatory applications?
Panel Response to EPA Question 4.2 - It is the panel's opinion that the building downwash
algorithms that are currently in ISC3, which were moved intact to AERMOD, have some
deficiencies. For example, these algorithms have problems with light-wind stable conditions and
with strongly-buoyant plumes from short stacks. There are indications that some of these
deficiencies may be overcome by the results of the PRIME project. I think that rather than
-------
REPORT I l lll FIRST PEER REVIEW REPORT
placing any burdens on AERMIC at this time, it would be best to wait and see what results from
PRIME and then consider what changes might be appropriate to improve AERMOD.
Furthermore, the Panel understands that the ADMS model contains advanced building downwash
algorithms that should be considered for inclusion in AERMOD.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
3. Conclusions and Recommendations
AERMOD Development and Evaluation Process - The Panel notes that, up until CTDMPLUS,
models (such as ISC3ST) proposed for regulatory use by the EPA were generally imposed on
users with a minimum of technical justification and evaluation. The process for proposing
CTDMPLUS in the late 1980's was greatly improved, since extensive technical justifications and
evaluations with detailed tracer data sets were included. The process for submittal of AERMOD
is another step in the right direction, as demonstrated by the very fact that the Peer Review Panel
has been formed and asked to prepare the current report. The Panel applauds this trend.
However, the development of AERMOD by the AERMIC committee (composed of EPA and
AMS scientists) has proceeded over the past several years with minimal funding. The scientists
who composed the primary boundary layer and dispersion algorithms have done so on nearly a
volunteer basis. The budget for evaluations and documentation was relatively small. Thus the
Panel finds a model which appears on the surface to represent significant scientific advancements
over ISC3, but which is not adequately documented, including detailed justifications for new
parameterizations, and which is not thoroughly evaluated with a wide range of tracer studies and
routine data sets. The comments by the Panel are intended to be at the level of the peer review of
a journal article, and therefore naturally do not take into account the shortfall in funding
necessary to do a thorough job. Perhaps the EPA/AMS AERMIC committee can carry out the
necessary further evaluations by "leveraging" funds obtained from a variety of interested parties,
such as DOD, API, and EPRI.
Discussion of Availability of Similar Models and Evaluations with the Same Data Sets as
AERMOD - While the Panel recognizes that the technical algorithms in AERMOD represent the
state-of-the-art, we are concerned that the impression is given that AERMOD is unique in this
regard. In fact, there are several models available which also contain similar state-of-the-art
technical algorithms (e.g., OML, ADMS, HPDM, and SGPUFF). For example, all these models
account for the non-Gaussian nature of the vertical velocity probability density function in the
convective boundary layer. And these models have been available (software, technical
documents, evaluations, experience of many users) for several years. The only one of these
models that is mentioned in the AERMOD reports is HPDM, which was compared with
AERMOD and ISC3 in some of the evaluation exercises (and which did quite well in these
comparisons). A natural question that should be addressed is why AERMIC did not simply
adopt one of these similar models? The public will ask why the EPA doesn't simply adopt, say,
ADMS? What advantages does AERMOD have over, say, ADMS or OML or HPDM or
SCIPUFF? These issues should be addressed in the model formulation document and in the
evaluation documents, which should include comparisons of statistical results for all relevant
models.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
Desired Additions to AERMOD Description of Model Formulation Document - The Panel
recommends that, besides the addition of discussions of available alternate models discussed
above, the Model Formulation document should be enhanced to include justifications for new
parameterizations. Supporting figures and tables should be added, including comparisons of
"intermediate" variables such as net radiation, heat flux, turbulence, plume rise, and plume
spread. There are several new "constants" and power law coefficients proposed with no
supporting evidence as would be required in any peer-reviewed journal article. Furthermore,
because of the use of many confusing variables (there are over a dozen sigmas and z's used in the
document) it would be a great help to include schematic diagrams illustrating these definitions
and approaches (e.g., the method for estimating the terrain height around a receptor).
Approaches to Modifications to Account for Urban Dispersion and Building Downwash Need
Further Study - The Panel feels that the modifications in AERMOD to account for urban
dispersion are preliminary and may need revision. It is troubling that the current methodology
assumes that the boundary layer is always unstable at night in an urban area. This assumption
should be tested with boundary layer observations in urban areas, instead of relying simply on
tuning the concentration predictions with concentration observations at Indianapolis.
Furthermore, the building downwash algorithms in AERMOD are identical to those in ISC3.
The community has identified a few problems with these algorithms (e.g., light-wind stable
conditions and strongly-buoyant plumes from short stacks), and these problems are being
addressed by the PRIME program. In addition, the AERMOD developers should investigate the
state-of-the-art building downwash algorithms that were developed by Britter and that are already
in ADMS
Specific Recommendations for Input Data - The Panel believes that, while most of the
information in the User's Guides is excellent, there will be confusion concerning which
meteorological input data are best for a specific scenario. This problem is seen especially for
situations where data are available from several on-site locations and from vertical profiles
measured by sounders. What exactly should a user do? The predicted concentrations can vary
by a factor of two or more depending on whether the modeler uses the tower next to the stack, the
tower on the hill, the tower on the top of a tall building, the rawinsonde sounding, or the 915
MHz sounding. Also, the Panel would like to see a demonstration that the new terrain algorithm
will produce repeatable results when applied to the same scenario by several independent
modelers.
Additional Data Sets for Evaluation -
Complex Terrain - As important as terrain is in many source impact assessments that are carried
out, it would seem to the Panel to be important to execute AERMOD for the three data bases that
were used to develop CTDMPLUS - Cinder Cone Butte, Hogback Ridge, and Tracy. These data
sets are desirable because they involved detailed monitoring networks and extensive
meteorological observations. If possible, useful information might result if the evaluations were
carried out with two options for input data: first, using all available data; and secondly, using a
-------
REPORT I l lll FIRST PEER REVIEW REPORT
degraded data base that might more closely resemble the type of data frequently available to
users. This evaluation exercise should also include ADMS, which contains the Hunt-Britter
algorithms.
Flat Terrain Site with Detailed Meteorological Inputs - Using an independent data base such as
Bull Run, it would be helpful to see how the model performs when complete vertical profiles of
temperature, wind, horizontal fluctuations and vertical fluctuations are available. This data base
could also be "degraded" to compare results when less data are available. The evaluation exercise
should also include OML, ADMS, HPDM, and SCIPUFF.
Building Downwash - As stated above, the results that come from the PRIME project should be
carefully reviewed. If these results are credible and indicate that useful changes should be made
to the building downwash algorithm in AERMOD, then recommendations should be made and
follow-up action taken. In addition the building downwash module in ADMS should be
reviewed and considered for implementation. The revised model should be evaluated with the
building downwash data (field and laboratory) collected during PRIME. It would also be useful
to carry out sensitivity studies to be sure that the combination of the new AERMOD model with
the old ISC3 building downwash modules yields results that correspond to what one might
expect. The panel would like to be assured that the AERMOD dispersion calculations do not
link with the building downwash algorithms to produce some totally unexpected result.
Further Evaluations with Ground-Level Sources - Since AERMOD's revised similarity theory
approach for ground-level sources was strongly calibrated to the Prairie Grass data, it is desirable
to evaluate the similarity model against an independent data base. The most promising
candidates would be the recent tracer studies carried out by Dugway Proving Ground.
-------
REPORT I l lll FIRST PEER REVIEW REPORT
Appendix I Comments on AERMOD Description of Model Formulation
Comments by Peer Review Panel on document by Cimorelli et al., entitled AERMOD
DESCRIPTION OF MODEL FORMULATION, dated 10 March 1998
General
G-l The review panel understands that this 81-page document was written with the
intent of providing some brief technical discussions of the model equations and
assumptions. However, in order to meet the standards of a peer-reviewed technical
document, it is necessary to much more fully explain the background, justification, and
testing of the model components.
G-2 At present, aside from the flowchart in Figure 1, there is not a single explanatory
figure or table in the document. Because of the use of many complicated definitions of
variables and the presence of many logical steps and decision points in the model, it is
necessary to provide figures that explain these complicated variables and procedures. As
it is, the review panel is lost in many places (e.g., the definition of terrain heights).
G-3 More background information should be given concerning related models (OML,
HPDM, ADMS, SCIPUFF). There are many models already available that include the
scientific advances in AERMOD.
G-4 Attribution should be given to the original references rather than intermediate
references (e.g., the wind profile equations were originally given by Businger, not
Wyngaard; most of the plume rise equations were originally suggested by Briggs).
G-5 The Section numbering convention led to major confusion for the three reviewers.
Instead of simply putting 2, the full 6.b.2 number should be given. The caps vs lower
case vs bold heirarchy should be modified so that it is more rational (e.g., lower case
applies to a lower-level section, etc.). The section headings should be expanded to be
more descriptive (e.g., add in CBL or in SBL).
G-6 The ordering of the sections should be switched around so that topics follow in
orderly progression. Now there are many many forward references to equations that
don't appear until many pages later. This is because, for example, the source and plume
rise sections are at the end rather than at the beginning. The entire document should be
reorganized so it flows from one topic to the next and so there are no forward
references.
G-l A clear recommendation is needed concerning the use of on-site meteorological
data. Should all available on-site meteorological data (e.g., surface fluxes, detailed
-------
REPORT I l lll FIRST PEER REVIEW REPORT
vertical profiles of winds, temperatures and turbulence) be used in AERMOD? Or is it
better to use parameterized meteorological variables in some cases? What do we do if
there are data available from more than one on-site tower (e.g., one at stack base, another
5 km to the west, and another on a hill 2 km to the east)
[Detailed, line-by-line comments for Appendix I in the original report are removed for brevity]
Appendix II Comments on Overview of AERMOD Evaluation Studies
Comments by Peer Review Panel on document by AERMIC entitled Overview of
AERMOD Evaluation Studies, dated March 1998.
This six-page document provides an overview of the two more detailed documents that
describe the developmental evaluation and the independent evaluation.
General Comments
G-l The review panel recommends that the so-called development evaluation be
downplayed and perhaps renamed (results of model calibration with developmental data
sets). It is not fair to evaluate AERMOD versus ISC when AERMOD has had its
parameters tuned with the data set while ISC has not had the benefit of this tuning.
G-2 The review panel finds that the so-called independent evaluations was interesting,
but was limited to only three sites where only routine data were available. No near-
surface sources were tested. It is not possible to conclude that AERMOD is a significant
improvement over ISC and other model until a comprehensive evaluation is carried out
using a variety of scenarios, including some sites with high-quality on-site meteorological
observations and tracer releases.
G-3 In most of the model-to-model comparisons, HPDM performed as well as
AERMOD. Why is HPDM not being recommended for adoption by the EPA, too?
G-4 The models included in the evaluations should be expanded to include OML,
ADMS, and SCIPUFF, which are all state-of-the-art models very similar to AERMOD.
[Detailed, line-by-line comments for Appendix II in the original report are removed for brevity]
-------
REPORT I l lll FIRST PEER REVIEW REPORT
Appendix III Comments on Developmental Evaluation
Comments by Peer Review Panel on document by Lee et al. entitled
DEVELOPMENTAL EVALUATION OF THE AERMOD DISPERSION MODEL
General
G-l As the review panel has mentioned elsewhere, this report should be renamed to
something like results of model calibration with developmental data sets. The
AERMOD Technical Document describes the intensive tuning and calibration that was
done with these data sets. Similar tuning was not done with the other models (ISC3,
HPDM, CTDMPLUS) that are compared with these data. It was therefore not a level
playing field.
G-2 A measure of the scatter is needed (e.g., VG or NMSE).
G-3 The meteorological data that were used from each site should be clearly explained.
G-4 The legends and captions of the figures should be expanded so that they can be
better understood by the readers.
G-4 It would help if a list could be given, in as brief a form as possible, of the changes
that were made to AERMOD as a result of the Developmental Evaluation.
[Detailed, line-by-line comments for Appendix HI in the original report are removed for brevity]
Appendix IV Comments on Performance (Phase II) Evaluation
Comments by Peer Review Panel on document by PES, Inc., entitled PERFORMANCE
(PHASE H) EVALUATION OF THE AERMOD DISPERSION MODEL dated March
1998
General
G-l - The terminology for this phase II study needs to be cleared up and a single definition
-------
REPORT I l lll FIRST PEER REVIEW REPORT
settled on. The Summary Report refers to this phase as an INDEPENDENT evaluation,
while the current document refers to it as a PERFORMANCE evaluation.
G-2 As mentioned in our review of the Developmental Evaluation report, the peer
review panel recommends that the Phase I developmental study be downplayed, since
much tuning was done with AERMOD. Emphasis should be on the current report, which
uses more independent data sets.
G-3 Despite the use of data from three sites, it should be stressed that the evaluation
exercise is very limited and should be greatly expanded. The three sites all involve
elevated stack plumes, routine S02 monitoring systems, and minimal on-site meteorology.
Before accepting the model, the EPA should evaluate it with additional data sets,
including ground level releases, sites with extensive on-site meteorological observations,
sites where tracer studies were carried out with dozens of monitors on well-designed arcs,
and sites where detailed complex terrain studies were made (e.g., Cinder Cone Butte,
Hogback, Tracy).
G-4 The report should acknowledge the existence of similar advanced models such as
OML and ADMS and SCIPUFF and explain why those models were not included in the
evaluations.
G-5 The rationale for accepting AERMOD would also lead to accepting HPDM. Does
the committee plan to recommend HPDM, too, on the basis of its performance in this
report?
G-6 Nearly all of the many figures need improvement. Figure numbers are hand-written
and captions are often non-existent. The captions should contain sufficient information
that the figure stands alone in the sense that the reader can understand it without
referring to the text.
G-7 The model evaluations should include a measure of the scatter, such as VG or
NMSE. The five-page discussion of the results should be greatly expanded to adequately
cover what is presented in 27 figures and 7 tables.
G-8 - The AERMOD interface is a key component of the model, providing complete
profiles of wind, temperature and turbulence from the surface up to 5000 meters above the
surface, extrapolating data up to that level from even the barest minimum (10 m)
measurement height. The Panel recommends that some evaluation of the performance of
the interface be provided as part of the overall evaluation of the model. Evaluation of the
interface can be accomplished by performing sensitivity tests and/or by comparing
predicted profiles against data. Comparisons against data can be accomplished by utilizing
meteorological measurements alone, which should increase the number of potential data
sets considerably over those that have concurrent measured pollutant concentrations. Of
-------
REPORT I l lll FIRST PEER REVIEW REPORT
particular interest is how the interface performs when driven by minimal data, and further
how sensitive it is to small changes in input parameters such as Zim.
G-9 - AERMOD has a tendency to underpredict the upper end of the frequency
distribution at Martins Creek even though the robust high concentration - RHC - is
overpredicted slightly for 24-hr averages. Marginal performance from a regulatory
perspective should trigger a more detailed review of the predictions to find evidence that
would (hopefully) support the model predictions. An examination of the emission rates -
total over all sources - associated with predicted and observed concentrations, however,
suggests that the model could underpredict substantially when considering the average
emission rate of the highest 25 predicted observed and modeled concentrations (Table
4.10 of the Phase II evaluation document; avg during obs is approx 1300, during pred.
2300 g/s). This tendency, coupled with an even more distinct tendency to underpredict
the upper end of the frequency distribution at Lovett where normalized concentrations
were reported, suggests that further model formulation changes may be called for. The
Panel believes that the tendency to underpredict at Martins Creek might be influenced by
the interface prediction of less stable temperature gradients at plume level than those
used by other models (thereby resulting in higher plume rise and lower Hcrit). Evaluation
of the interface, and further diagnostic analysis of the meteorological conditions associated
with high predicted and observed concentrations, could provide some insights into
whether this is the case and could potentially lead to changes in the interface as opposed
to changes in the terrain interaction formulation.
[Detailed, line-by-line comments for Appendix I in the original report are removed for brevity]
Appendix V - Comments on "User's Guide for AERMOD"
The AERMOD users guide is similar in organization to the ISC3 users guide. This structure
should help new users gain familiarity with the new model. Some general comments follow:
In past updates to the ISC3 model, the EVENT processor which makes it possible to identify
source-specific contributions and meteorological conditions associated with selected events, has
always lagged behind the appearance of the new model. Some consideration should be given to
including the EVENT processing directly in AERMOD so that this lag does not occur.
Given the importance of AERMOD's interface that generates profiles of winds, temperature, and
turbulence, it would be helpful to have an option that prints out the generated profile for selected
hours in a format that can be compared directly to the input data. Ultimately, the understanding,
acceptance, and use of AERMOD will be enhanced by the degree to which users (not all users, but
those who are involved with modeling routinely and who are committed to understanding the
tools that they use) can determine how the model is carrying out its calculations. Experience has
-------
REPORT I l lll FIRST PEER REVIEW REPORT
shown that the profiles generated by the interface are a key element in this understanding, and an
easy way to review the profiles for selected hours would be quite helpful.
-------
REPORT II. AERMIC'S RESPONSE TO THE FIRST PEER REVIEW REPORT
(Dated 12/98)
1. INTRODUCTION
In the following, AERMIC responds to comments provided by a Peer Review Panel
(Hanna et al., 1998) that assessed several documents describing the dispersion
model AERMOD. The Panel was established in mid-March 1998 at the request of
EPA and reviewed documents related to the AERMOD development:
1) a Model Formulation Document (MFD) describing the dispersion
model algorithms, 2) three reports summarizing model evaluations, and 3) three
user's guides. In addition, the Panel focused on twelve general questions
posed by EPA concerning the formulation and evaluation of AERMOD and its
potential for replacing ISC3 as a regulatory model for short-range
dispersion applications.
AERMIC is grateful to the Panel for their quick response and review of the
AERMOD documents, and their recommendations on this
model development activity. In Section 2, we give responses to Panel's
answers to the 12 general questions. Although we do not agree with all of
the comments raised by the Panel, we believe that their comments have played an important role
in: 1) producing a more complete MFD, 2) recommending some of the needs for further model
development (e.g., the downwash model), and 3) suggesting areas for further model evaluation
efforts.
-------
REPORT II. AERMIC' S FIRST RESPONSE
2. RESPONSE TO THE PANEL'S ANSWERS TO THE TWELVE GENERAL
QUESTIONS
General Question 1.1. As a steady-state, plume-based, regulatory model, does
AERMOD represent the state-of-the-science in its handling of boundary layer
turbulence and dispersion?
[Unfortunately, the Panel did not give a direct answer to this question.
Their answer is qualified by "Within the resources available to AERMIC and
the data available" "represents a very good attempt" and thus is
ill-defined. Does a "very good attempt" mean that we succeeded or failed?
There are some later hints that the Panel thought that AERMOD
features were state-of-the-science (e.g., answers to questions 1.2 (page 4),
1.3 (pages 4 and 5), 3.2 (top of page 10)). We hope that the peer review committee will draw a
conclusion on this question.]
General Question 1.2. Within the context of regulatory dispersion models in
the US, does AERMOD represent significant scientific advances over ISC3?
[Again, the Panel did not give a direct answer to this
question, i.e., the AERMOD state-of-the-art science relative to ISC3.
The first part of their answer refers to the type of meteorological
input and its effect on the quality of the predictions relative to
those of ISC3. They state: "Since AERMOD is able to include more
boundary layer observations than the ISC3 regulatory model, it would seem
that it represents better science." Again, this is a qualified remark.
The Panel states, however, that AERMOD's plume rise and dispersion algorithms
represent the state-of-the-art in plume modeling, but do not state whether or
not this is an advance over ISC3.]
[In the final paragraph, the panel comments about the lack of consideration given to several other
models that include scientific advances (OML, ADMS, HPDM, and SCIPUFF) and that we
should demonstrate where AERMOD represents an advance over these models. (AERMIC
in fact did consider two of the models as discussed below.) However, the real question asked was
whether or not AERMOD represented a significant advance over ISC3. Unfortunately, this
question was not answered.]
With regard to OML, ADMS, HPDM, and SCIPUFF,
none of these models was formally submitted to the EPA
for consideration, although HPDM came close. In 1991, when
AERMIC was formed, none of these models was available to EPA,
with the possible exception of OML. AERMIC invited Drs. Helge
-------
REPORT II. AERMIC' S FIRST RESPONSE
Olesen and Ruwim Berkowicz, the developers of OML, to one of its early
meetings. We obtained users guides and formulation
documents for OML. Like HPDM, OML does not treat terrain with significant
elevations (above stack height),
although it has simple algorithms to describe dispersion over
rolling terrain. Due to the terrain handling limitation, further
consideration was not given to HPDM or OML, since extensive modifications
would be necessary to include terrain. We note, however, that a number of
ideas are borrowed from or in common with these models; this commonality
will be clarified in the final MFD.
ADMS is a proprietary model that was not available when the AERMIC activity
began. Technical information about the model appears to be
difficult to obtain without paying licensing fees. SCIPUFFs
public availability as an operational model with full documentation also
was limited when our development process began. In addition, SCIPUFF was not
developed to the point where it is today.
General Question 1.3. What do you think are the most scientific advancements
in AERMOD?
The Panel lists the following: 1) inclusion of straightforward procedures to
handle terrain influences, 2) elimination of the discrete Pasquill stability
classes and replacement of them by continuous stability classes using the
Monin-Obukhov length (and we add planetary boundary layer variables),
3) treatment of strongly convective conditions by the PDF approach,
4) adoption of a plume lofting and penetration model, and 5) inclusion of
a different dispersion treatment for surface sources than for elevated
releases.
The Panel again states that there are other models (OML, ADMS, HPDM, and
SCIPUFF), which should have been investigated more thoroughly. As noted
above (our response to question 1.2), we did indeed review OML and HPDM and
have a number of treatments (e.g., PDF approach with HPDM, convective scaling
and PBL parameterization, etc.) that are similar to theirs. We believe that
all major technical features included in these "other models," relative to ISC3, are also included in
AERMOD (although the details do differ). These relationships have been clarified in the latest
version of the MFD.
General Question 1.4. Are there any areas or features of AERMOD in which
an improved formulation or treatment would be desired? If so, please discuss
whether you think the revised treatment would lead to better performance
and how much.
-------
REPORT II. AERMIC' S FIRST RESPONSE
AERMIC is aware of the deficiencies of the ISC3 downwash approach as
expressed by the Panel and has considered the PRIME model as an alternative.
However, there is no thorough documentation of PRIME to permit a scientific
evaluation of the model, and there has been no peer review of that model.
Furthermore, there was insufficient funding to pursue an alternative
downwash approach (PRIME, ADMS, or some other) as an improved treatment
for downwash. We agree with the Panel that an alternative downwash approach
should be considered.
We agree with the Panel that further development and testing of the urban
dispersion model would be desirable, but the resources for this are not
available. For example, considerable time will be necessary to prepare
well-organized urban data sets that are similar in quality to other rural data sets.
We also note that the MFD (March 1998) submitted for the Peer Review incorrectly stated
that the model assumes an unstable planetary boundary layer (PBL) over the
urban area at night. In fact, the PBL is modeled as stable, consistent with
the surrounding rural area, but has enhanced convectively-generated
turbulence due to the urban-rural temperature difference. The enhanced
turbulence velocities are proportional to the urban nocturnal convective
velocity scale, which depends on the urban mixed-layer height and the
upward heat flux.
The Panel also states that many of the algorithms in AERMOD have not been
adequately justified by comparison with data. We agree that more justification
of some algorithms is necessary. However, in some cases, algorithms have
been borrowed from other dispersion models (e.g., the PDF approach from
Weil et al., 1997), and we rely on this earlier work for justification.
In other cases (e.g., surface layer sources in the CBL), the model has been taken
as an interpolation fit between two well-defined limits (a surface source
and an elevated source) and dispersion data for intermediate source heights
are lacking. Further justification is being provided in the MFD where necessary,
and figures showing the variation of meteorological variables with height and
other parameters have been added.
General Question 2.1. Is the current organization of the model formulation
document and User's Guide appropriate or would an alternative be desired?
The Panel suggested that a clear recommendation should be made concerning
the use and need for on-site meteorological data including vertical profiles.
Recommendations and guidance of this will be developed from the
meteorological degradation analysis, which has just been completed. These
recommendations should be available for the next EPA Conference
-------
REPORT II. AERMIC' S FIRST RESPONSE
on Air Quality Modeling. The degradation analysis consists of evaluating
the model against several air quality data sets using different levels or
degrees of meteorological inputs—1) off-site data only (e.g., National
Weather Service), 2) on-site data with only 10-m height tower and off-site temperature profiles,
and 3) on-site data with meteorological profiles (and extrapolated profiles using PBL scaling),
etc.
The Panel also recommended that schematic diagrams be added to explain some
of the concepts. We agree with this and have added a number of such diagrams
to the current version of the MFD.
General Question 2.2. Is the presentation of the model clear and explanatory?
Please note any specific sections of the documentation that were unclear or
confusing.
The Panel again recommends that the MFD be enhanced by adding justifications
for parameterizations included in AERMOD and adding explanatory diagrams.
We agree with this and have pursued it in the new version of the MFD as noted
above (Question 2.1).
The Panel also recommends that discussions of comparisons with similar
state-of-the-art models (OML, HPDM, ADMS, and SCIPUFF) be included. AERMIC
will include general and some specific comments about these
models. However, the real issue before the peer-review panel is the following: Is
AERMOD a state-of-the-science model that is ready for regulatory use (for replacing ISC)?
As another suggestion, the Panel recommended that the units of
variables be included with the equations in the MFD; we have added a full List of Symbols
with units as another section to the MFD.
General Question 2.3. Is the documentation sufficient for a typical ISC-type
user to guide them in the use of the model and its preprocessors? Do you
think training sessions would be particularly useful?
Overall, the Panel seemed to think that the documentation would be sufficient
for typical users. The area where they felt some confusion may exist and
where training may be necessary is in the use of on-site meteorological
data, particularly when extensive data (multiple-level towers, sodars, surface
data, etc.) are available. AERMIC believes that results from the (recently
completed) meteorological degradation analysis will be useful here, and guidance from this
analysis will be available in the near future. [A draft of this document is now available
for the Panel.]
-------
REPORT II. AERMIC' S FIRST RESPONSE
General Question 3.1. How do you rate the performance of AERMOD relative to
ISC3 and the other models included in the evaluation exercises?
The Panel states that the five developmental data sets (Kincaid S02 and
SF6, Indianapolis, Lovett, and Prairie Grass) should be "set aside" when
addressing this question because model components were "tuned" or calibrated
using this data. AERMIC disagrees with this "set aside" philosophy. Clearly,
some tuning was performed using these data sets which were quite different
in the nature of the sources and terrain: 1) tall stack, buoyant release in
flat terrain (Kincaid), 2) tall stack, buoyant release near elevated terrain
(Lovett), 3) tall stack, buoyant release in an urban area (Indianapolis),
and 4) nonbuoyant surface release (Prairie Grass). However, since the core
model algorithms apply to such a diverse range of conditions, we believe
that there is an underlying model generality that is exemplified by
these comparisons and should be noted.
We agree that ISC3 and other models could be tuned and improved using these same data sets.
HPDM was tuned and improved with the Kincaid and Indianapolis data. In essence,
one could say that ISC was the model chosen for the AERMIC development effort
since we adopted that model's computer architecture. Early in our development effort (1992, 1993;
e.g., see Weil, 1992), we referred to the model (which has evolved into AERMOD) as an updated
or new version of ISC and called it ISC3! However, EPA recommended
that we adopt a new name to avoid confusion with the PGT-based ISC Model
familiar to the user community; hence, we adopted the name AERMOD.
We agree with the Panel that one should expect AERMOD to
perform as well as or better than the PGT-based ISC3, as it does. This
should be taken in a positive light showing the applicability of AERMOD
to a broader range of conditions.
We note that when CTDM was submitted to EPA in 1987 for
regulatory consideration, the three developmental tracer data bases
(Cinder Cone Butte, Hogback, and Tracy) were given equal weight with the "independent" data
bases in judging model performance. These developmental data bases were not set aside, and,
in fact, were used in assigning a "skill score" to the model.
The Panel "feels that a scientific evaluation should include a technical
peer review of the model components." AERMIC believed that
such an evaluation was being generated by the Peer Review just completed.
At any rate, AERMOD will receive such additional scientific evaluation when the model is
submitted for journal publication in 1999.
The Panel also states that changes made to the model following the
developmental evaluation should appear in a separate section with a summary
-------
REPORT II. AERMIC' S FIRST RESPONSE
table. AERMIC feels that this is an unwieldy and unreasonable request, and
we do not have the records to construct this table. We are unaware
of other development efforts where a log of model changes was recorded as
the development progressed.
The Panel states that the three data sets used in the Phase II Evaluation
represent running of the model with the "barest minimum of input data."
The AERMIC disagrees with this view. Each of the three source sites had
meteorological towers with multiple levels of data, and the Martin's Creek
plant had tower and sodar data (for winds) at multiple levels. We would
interpret the "barest minimum" of meteorology to include only one level
of on-site tower data and probably at the 10-m level.
The Panel suggests that most users would run the model with more than the
bare minimum of data. Experience would suggest otherwise because the bare
minimum would be the least expensive; of course, there are exceptions.
Most users would go beyond the bare minimum only
if: 1) compelled to do so by a regulatory agency, or 2) they felt there was a
good chance of obtaining lower predicted concentrations and hence a higher
emission limit. In fact, there are examples where users have argued
against their own on-site, multiple-level tower data versus airport data collected some 20 km
away; the on-site meteorology happened to yield higher ground-level
concentrations!
The Panel discusses the value of running the model with more detailed on-site
meteorology with vertical profiles of winds, temperature, and turbulence
and comparing model results using this input against those with less-detailed
meteorology. Such a study is being conducted under our meteorological
degradation analysis and results of that will be reported at the next
EPA Conference on Air Quality Modeling. From this analysis, recommendations
and guidance will be developed on the AERMOD meteorological inputs.
The Panel recommends that AERMOD be evaluated further for situations of
complex terrain and building downwash. AERMIC agrees with this recommendation
and has followed it for complex terrain. Since the Peer Review, AERMOD
has been evaluated further using 1) the Tracy tracer data set on which
it performed as well as CTDM+, and 2) the Westvaco data set where AERMOD outperformed
CTDM+ and ISC3. AERMIC also would like to test the downwash algorithm with the
PRIME data set but currently does not have the resources to do this.
The Panel recommends further independent testing of AERMOD against data from a
near-surface source and the Bull Run power plant—a tall stack buoyant plume.
Of the two, we believe that the surface release is the more important, and
-------
REPORT II. AERMIC' S FIRST RESPONSE
we will address this as resources permit. We note that as of now AERMOD
has been evaluated with 10 substantial data bases, which is comparable to
or more than that used for any other model proposed for short-range
regulatory applications.
Finally, the Panel recommended that other similar models (OML, HPDM, ADMS,
and SCIPUFF) be included in the model evaluation exercise. As noted earlier,
we did include HPDM in the Phase II evaluation using the Baldwin and
Clifty Creek data bases. In evaluation, our first priority is comparing AERMOD to other models
currently in use for regulatory applications, i.e., to determine
AERMOD's use or fit as a replacement regulatory model.
General Question 3.2. From a model design, scientific, and performance
perspective, what comments do you have on the replacement of ISC3 with
AERMOD for regulatory applications?
Several panel comments here were raised previously and addressed under our
earlier responses. A new comment is that the Panel "feels
that AERMOD has not been adequately evaluated with independent data, since
the three data sets in the Independent Performance Evaluation Report are all
"routine" data sets that do not include extensive onsite meteorological
observations..." AERMIC believes that these data sets are typical or better
(Martin's Creek) in terms of on-site meteorology for routine, year-long
air quality data sets. In the Phase II Evaluation, our emphasis is on such
data sets because they are typical of the types of regulatory applications
to which the model will be subjected in the future.
The Panel also points out that HPDM performs as well as AERMOD in the two
independent data sets for which it was evaluated. AERMIC agrees with this and
also notes the high-wind algorithm of HPDM was tuned in part using these
two data sets—Baldwin and Clifty Creek. The Panel states that HPDM also
should be considered as a replacement for ISC3. However, when HPDM was
submitted earlier to EPA for consideration as a regulatory model, it was
classified as simple terrain elevated-source model. Given the lack of a
significant complex terrain algorithm, HPDM could not be considered as a
viable replacement for ISC3. Of most importance, it is not within AERMIC's purview
to decide on the submission of HPDM as a regulatory model or to
select it for regulatory consideration (this is the responsibility of
the EPA OAQPS).
General Question 3.2. When considering the eight data bases used to evaluate
the model, would additional evaluation of AERMOD be desirable?
-------
REPORT II. AERMIC' S FIRST RESPONSE
The Panel's comments here were raised earlier and addressed in our previous
responses. We agree that further model evaluations would be desirable,
but only can be addressed as resources permit. Of the four recommended areas
for further evaluation, the one involving tracer releases in complex terrain
has been addressed since completion of the Peer Review—the
Tracy and Westvaco data sets were added to the evaluation. Of the three
remaining recommendations, AERMIC believes that the priorities (highest first) are: 1) downwash
conditions, 2) surface releases, and 3) data sets with more complete meteorology (more
vertical profiles, etc.) than those used in the Phase II Evaluation. The AERMIC
would also add urban sources to this list as a priority 2.
Additional EPA Question 4.1. Dated 3/18/98. Is the AERMOD approach to
modeling urban sources scientifically sound and state-of-the-art?
The Panel states that the urban modeling procedures were developed through
calibration of predicted concentrations with observations at the
Indianapolis site. It is true that one dimensionless coefficient
(alpha = 0.03) was adjusted by comparing predictions and observations.
However, the basic expressions relating heat flux and urban mixed-layer
height to city size (using the urban-rural temperature difference) were
obtained from data given by Oke (1973, 1982). The latter studies covered
Canadian cities with sizes ranging from populations P of 1.0E+03 to 2.0E+06;
Indianapolis with a P = 700,000 falls within this range. We agree that
comparisons of the heat flux and mixed-layer height predictions to
observations at Indianapolis would be desirable.
The Panel also notes that there were no independent urban
data sets in the Phase II Evaluation, which is true. Again, available resources
were insufficient to include such data sets. However, we are unaware of other tracer data sets
for tracking or tracing individual sources in urban areas that would permit
unambigous evaluation of AERMOD for a single source. We recognize that
multiple source and concentration data sets (e.g., S02) exist for
urban areas, but they would not permit evaluation of single-source
dispersion within the urban environment. Furthermore, given the size of such
data sets and the number of sources, use of them in model evaluation would
require substantial resources.
Additional EPA Question 4.2. Dated 3/24/98. Do the building downwash algorithms
within AERMOD represent the current state of the science and are these
algorithms appropriate to regulatory applications?
As stated earlier, AERMIC agrees that there are deficiencies in the
-------
REPORT II. AERMIC' S FIRST RESPONSE
ISC3 downwash algorithms as used in AERMOD. We are considering PRIME and other
downwash treatments as a potential future replacement for the current
treatment in AERMOD. This is one of our highest priorities for future work and will be
addressed as resources permit.
-------
REPORT II. AERMIC' S FIRST RESPONSE
3. RESPONSE TO THE PANEL'S CONCLUSIONS AND RECOMMENDATIONS
AERMOD Development and Evaluation Process — Here, the Panel's two major
criticisms are those that have appeared earlier in their responses to the
twelve questions: 1) inadequate documentation and justification for the new
parameterizations in AERMOD, and 2) insufficient AERMOD evaluation over a
wide range of tracer studies and routine data sets. We have addressed
these comments in our earlier responses. However, in summary, AERMIC has
added justification and further description of algorithms to the most recent
version of the MFD.
Concerning the model evaluation, we reiterate that {\bf AERMOD has been
evaluated against 10 substantial data bases} including: 1) four data sets
for tall stack buoyant plumes in flat terrain (Kincaid S02, Kincaid SF6,
Baldwin, and Clifty Creek), 2) four data sets for tall stacks in complex
terrain or near elevated terrain (Lovett, Martins Creek, Tracy, and Westvaco),
3) a buoyant elevated release in an urban environment (Indianapolis),
and 4) a nonbuoyant surface release (Prairie Grass). We agree that more
evaluation would be desirable (as always) especially for downwash conditions,
urban sources, and surface releases. However, there is a key question
to the AERMOD development process: Has there been enough evaluation already to justify
replacing ISC3 by AERMOD? AERMIC believes that there has been.
Discussion of Availability of Similar Models and Evaluations with the Same
Data Sets as AERMOD — Here, it is stated that "While the Panel recognizes
that the technical algorithms in AERMOD represent the state-of-the-art,
we are concerned that the impression is given that AERMOD is unique in this
regard." The impression that AERMOD is unique, as the Panel surmises from
the AERMOD documentation, certainly was not intended. This impression will
be removed by adding to the MFD a discussion of the general attributes of
these other models. Clearly, AERMIC recognizes that other models contain
similar state-of-the-art algorithms for dispersion; several AERMIC members
participated in the development of other such models. We indicated in our
earlier responses why OML and HPDM were not chosen by AERMIC for
development—the absence of treatment for dispersion about significant
elevated terrain. Concerning ADMS and SCIPUFF, we note that
widely-available documentation for these models and/or a demonstrated
applicability to the range of conditions required by a new US regulatory dispersion model
(for full elevated terrain, downwash, etc) did not exist when the AERMIC
activity began in 1991.
The Panel posed a question concerning adoption of one of the other models
for submission to the EPA. However, another natural and relevant question
-------
REPORT II. AERMIC' S FIRST RESPONSE
is: Why not start with the ISC Model, which is quite familiar to the US-based
user community, and adopt/modify it to overcome its limitations (lack
of treatments for PBL scaling and state-of-the-art dispersion, full
elevated terrain, vertical inhomogeneity in the PBL, etc.)? The latter
approach is of course the path followed by AERMIC in developing AERMOD.
Desired Additions to AERMOD Description of Model Formulation Document — We
agree that the MFD needs to be enhanced with justifications for new
parameterizations, etc. and have addressed this under our earlier responses.
Approaches to Modifications to Account for Urban Dispersion and Building
Downwash Need Further Study — As noted earlier, the MFD was in error in
stating that the boundary layer is always unstable in an urban area (see
general question 1.4). The other comments concerning downwash were addressed
in our earlier responses.
Specific Recommendations for Input Data — As stated earlier (general question
2.1), recommendations and guidance on meteorological input data will be
developed from the meteorological degradation analysis, which has just been
completed. This analysis used several of the data sets which had multiple
levels of tower data.
Additional Data Sets for Evaluation
Complex Terrain — As noted in our earlier responses, we added an evaluation
of AERMOD with the Tracy data set, as recommended by the Panel, and also the
Westvaco data set, which represents a more operational-type situation.
Flat Terrain Site with Detailed Meteorological Inputs — We agree that it
would be nice to evaluate the model against another tall stack, buoyant
plume situation in flat terrain, but AERMOD already has been evaluated with
four flat-terrain data sets. The recommended degraded meteorology exercise
has been carried out with the Kincaid data set, which offers the same
type of meteorological data as Bull Run.
Building Downwash — The Panel's comments here were addressed under our
responses to the twelve general questions.
Further Evaluations with Ground-Level Sources — The Panel's comments here
also were addressed under our responses to the twelve general questions.
We certainly will consider the Dugway Proving Ground data for evaluation
of surface-source dispersion, as resources become available.
-------
REPORT II. AERMIC' S FIRST RESPONSE
4. RESPONSE TO APPENDIX I COMMENTS
Seven general comments and a voluminous number of specific ones were made
in Appendix I on the MFD. Four of the general comments (Gl, G2, G3, G7) were
addressed earlier in this response (Section 2). Two of the general comments
and many of the specific ones concerned either the format of the MFD or
were editorial in nature. Most of them have been addressed in an updated
and expanded version of the MFD; e.g., we have added the complete section
number to all of the section and subsection headings in the document. There
were recurring requests and comments concerning the referencing to earlier
models—HPDM, OML, ADMS, and SCIPUFF; a general discussion of these models
is given in the background section of the MFD.
A number of other substantive comments were included in Appendix I and many of them were
dealt with in our earlier remarks (Section 2) or are being handled in the updated MFD.
For example, we have added about 16 figures to the MFD with 3 or 4 pertaining
to elevated terrain. However, there were also a number of comments with
which we disagreed—e.g., the data base "set aside" philosophy, the request
to evaluate four other models (HPDM, OML, ADMS, and SCIPUFF), including
background on the AMS - EPA working group back to 1979, etc. Some comments
also were unjustified. This includes the comment on why the beta test was
not open and why Earth Tech did not participate in it. In fact,
the beta test was open with AERMOD being placed on the EPA SCRAM Bulletin
Board at the same time AERMIC asked specific individuals to review the
model. Earth Tech and in particular Joseph Scire was invited to participate
in both of the beta tests. Unfortunately, Joe was unable to do so due to
a heavy work commitment at the time, but AERMIC did receive some comments
from David Strimaitis.
-------
REPORT II. AERMIC' S FIRST RESPONSE
5. RESPONSE TO APPENDIX II COMMENTS
[Comments by Peer Review Panel on document by AERMIC entitled "Overview of AERMOD
Evaluation Studies", dated March 1998.]
General Comments -
G-l - The review panel recommends that the so-called "development evaluation" be
downplayed and perhaps renamed (results of model calibration with developmental data
sets). It is not fair to "evaluate" AERMOD versus ISC when AERMOD has had its
parameters tuned with the data set while ISC has not had the benefit of this tuning.
The large number of data sets involved in the developmental evaluation suggests that AERMOD
has considerable skill in performing well for a diverse selection of sites. In addition, the
expression that "parameters were tuned" is not an accurate one. The AERMOD formulation is
based upon sound physical principals, rather than highly statistical fits to data. The diverse data
bases confirm the selection and use of the model algorithms rather than drive their formulation.
In terms of "fairness" in not tuning ISC, it should be noted that AERMOD represents a significant
advance in U.S. regulatory model development. The purpose of the evaluation exercise is to
verify that this new model performs at least as well as or better than an existing regulatory model,
ISC. It should also be noted that ISC was developed, in part, as a result of the use of data from the
Prairie Grass experiment. ISC has also been previously evaluated with several of the Phase I and
II data bases, with fairly good results for the Kincaid, Indianapolis, Baldwin, and Clifty Creek data
bases. Therefore, AERMOD was confronted with a significant challenge in performing better
than ISCST3 on several of these data sets.
G-2 - The review panel finds that the so-called "independent evaluation" was interesting,
but was limited to only three sites where only routine data were available. No near-surface
sources were tested. It is not possible to conclude that AERMOD is a significant
improvement over ISC and other model until a comprehensive evaluation is carried out
using a variety of scenarios, including some sites with high-quality on-site meteorological
observations and tracer releases.
The use of three data bases for an independent evaluation is a significant task that required
considerable resources. In response to the peer review panel comments, two additional data bases
(both involving complex terrain) were added to the list (Westvaco and Tracy). In addition, the
American Petroleum Institute has carried out an independent series of evaluations of AERMOD,
ISCST3, and ADMS on a tracer experiment conducted at a petrochemical complex, in which near-
surface area and volume source types were tested.
G-3 - In most of the model-to-model comparisons, HPDM performed as well as AERMOD.
Why is HPDM not being recommended for adoption by the EPA, too?
-------
REPORT II. AERMIC' S FIRST RESPONSE
The submittal of HPDM to EPA for consideration as a guideline model is not the responsibility of
AERMIC. In 1991, when AERMIC was considering the basis for a new model platform, HPDM
had been withdrawn from consideration as a guideline model. HPDM did not address complex
terrain effects, and this was seen as a major shortcoming of the model. However, many of the best
features of HPDM, including several of the meteorological pre-processor formulations and a
convective PDF algorithm, are incorporated in AERMOD.
G-4 - The models included in the evaluations should be expanded to include OML, ADMS,
and SCIPUFF, which are all state-of-the-art models very similar to AERMOD.
The primary purpose of the AERMOD evaluation effort, as directed by the U.S. EPA, is to
determine whether AERMOD performs as well as, or better than, the existing regulatory models.
Evaluations including the OML, ADMS, and SCIPUFF models would be nice to do, but would
consume valuable resources that are better directed toward more pressing priorities. It should also
be noted that there are drawbacks to each of these models. OML has been developed for
Denmark, and is not strongly oriented to address complex terrain sites. The ADMS model is a
proprietary code that is not readily available. Evaluations involving ADMS are being separately
conducted by API, and preliminary indications are that the ADMS formulations are not as
advanced as those of AERMOD, and that the AERMOD evaluation performance is somewhat
better than that of ADMS. In recent conversations with Dr. Ian Sykes, Mr. Robert Paine has
learned that SCIPUFF is not quite ready for public release.
[Detailed responses to line-by-line comments in the original report are removed for brevity]
6. RESPONSE TO APPENDIX III COMMENTS
Comments by Peer Review Panel on document by Lee et al. entitled
"DEVELOPMENTAL EVALUATION OF THE AERMOD DISPERSION MODEL"
(undated but received March 1998)
General comments
G-l - As the review panel has mentioned elsewhere, this report should be renamed to
something like "results of model calibration with developmental data sets". The AERMOD
Technical Document describes the intensive tuning and calibration that was done with these
data sets. Similar tuning was not done with the other models (ISC3, HPDM, CTDMPLUS)
that are compared with these data. It was therefore not a "level playing field".
The large number of data sets involved in the developmental evaluation suggest that AERMOD
has considerable skill in performing well for a diverse selection of sites. In addition, the
expression that "parameters were tuned" is not an accurate one. The AERMOD formulation is
-------
REPORT II. AERMIC' S FIRST RESPONSE
based upon sound physical principals, rather than highly statistical fits to data. The diverse data
bases confirm the selection and use of the model algorithms rather than drive their formulation.
In terms of "fairness" in not tuning ISC, HPDM, or CTDMPLUS, it should be noted that
AERMOD represents a significant advance in U.S. regulatory model development. AERMIC
knows of no precedent in which multiple models were "tuned" in preparation for a model
evaluation study. The purpose of the evaluation exercise is to verify that this new model performs
at least as well as or better than the existing regulatory models, ISC and CTDMPLUS. HPDM
was included with two databases (Baldwin and Clifty Creek) because it was felt that HPDM
should do well with these data sets, and AERMOD should have at least a comparable
performance.
G-2 - A measure of the scatter is needed (e.g., VG or NMSE).
For the tracer data sets, the box and whisker plots provide a visual means to assess the prediction
scatter. For full-year data sets, the sparse coverage of the monitors does not lend itself to
meaningful scatter analyses.
G-3 - The meteorological data that were used from each site should be clearly explained.
This document has been absorbed into a more complete report that covers all of the major
evaluations of AERMOD. Some expansion of the discussion of the meteorological data for the
developmental evaluation was added.
G-4 - The legends and captions of the figures should be expanded so that they can be better
understood by the readers.
OK.
G-4 - It would help if a list could be given, in as brief a form as possible, of the changes that
were made to AERMOD as a result of the Developmental Evaluation.
A list of the changes desired are not available and would take considerable resources to
reconstruct, if even possible. AERMIC is not aware of a precedent in which such information was
provided.
[Detailed responses to line-by-line comments in the original report are removed for brevity]
-------
REPORT II. AERMIC' S FIRST RESPONSE
7. RESPONSE TO COMMENTS APPENDIX IV
Comments by Peer Review Panel on document by PES, Inc., entitled "PERFORMANCE
(PHASE II) EVALUATION OF THE AERMOD DISPERSION MODEL" dated March
1998 (about 200 pages of text including many figures)
General comments
G-l - The terminology for this phase II study needs to be cleared up and a single definition
settled on. The Summary Report refers to this phase as an "INDEPENDENT" evaluation,
while the current document refers to it as a "PERFORMANCE" evaluation.
AERMIC internally referred to the "Phase II" evaluation as the independent evaluation. The
terminology will be clarified in the general evaluation report that covers both the developmental
and independent evaluations. The report that is the subject of these comments will remain as a
resource, but will not be updated or placed in the EPA docket for the 7th Modeling Conference.
G-2 - As mentioned in our review of the "Developmental Evaluation" report, the peer
review panel recommends that the Phase I developmental study be downplayed, since much
tuning was done with AERMOD. Emphasis should be on the current report, which uses
more independent data sets.
The large number of data sets involved in the developmental evaluation suggest that AERMOD
has considerable skill in performing well for a diverse selection of sites. In addition, the
expression that "parameters were tuned" is not an accurate one. The AERMOD formulation is
based upon sound physical principals, rather than highly statistical fits to data. The diverse
databases confirm the selection and use of the model algorithms rather than drive their
formulation. The consistently good results obtained with AERMOD over the combination of the
developmental and the independent evaluation databases provides a confirmation of the broad
applicability of the model.
G-3 - Despite the use of data from three sites, it should be stressed that the evaluation
exercise is very limited and should be greatly expanded. The three sites all involve elevated
stack plumes, routine S02 monitoring systems, and minimal on-site meteorology. Before
accepting the model, the EPA should evaluate it with additional data sets, including ground
level releases, sites with extensive on-site meteorological observations,
sites where tracer studies were carried out with dozens of monitors on well-designed arcs,
and sites where detailed complex terrain studies were made (e.g., Cinder Cone Butte,
Hogback, Tracy).
-------
REPORT II. AERMIC' S FIRST RESPONSE
The use of three data bases for an independent evaluation is a significant task that required
significant resources. In response to the peer review panel comments, two additional data bases
(both involving complex terrain) were added to the list (Westvaco and Tracy). In addition, the
American Petroleum Institute has carried out an independent series of evaluations of AERMOD,
ISCST3, and ADMS on a tracer experiment conducted at a petrochemical complex, in which near-
surface area and volume source types were tested.
AERMIC disagrees with the comment that the three independent databases described in this
report involved "mininal" meteorological data. Each had a tall tower instrumented at multiple
levels, and one site, Martins Creek, also had a sodar.
G-4 - The report should acknowledge the existence of similar advanced models such as
OML and ADMS and SCIPUFF and explain why those models were not included in the
evaluations.
The primary purpose of the AERMOD evaluation effort, as directed by the U.S. EPA, is to
determine whether AERMOD performs as well as, or better than, the existing regulatory models.
Evaluations including the OML, ADMS, and SCIPUFF models would be nice to do, but would
consume valuable resources that are better directed toward more pressing priorities. It should also
be noted that there are drawbacks to each of these models. OML has been developed for
Denmark, and is not strongly oriented to address complex terrain sites. The ADMS model is a
proprietary code that is not readily available. Evaluations involving ADMS are being separately
conducted by API, and preliminary indications are that the ADMS formulations are not as
advanced as those of AERMOD, and that the AERMOD evaluation performance is somewhat
better than that of ADMS. In recent conversations with Dr. Ian Sykes, Mr. Robert Paine has
learned that SCIPUFF is not quite ready for public release.
G-5 - The rationale for accepting AERMOD would also lead to accepting HPDM. Does the
committee plan to recommend HPDM, too, on the basis of its performance in this report?
The submittal of HPDM to EPA for consideration as a guideline model is not the responsibility of
AERMIC. In 1991, when AERMIC was considering the basis for a new model platform, HPDM
had been withdrawn from consideration as a guideline model. HPDM did not address complex
terrain effects, and this was seen as a major shortcoming of the model. However, many of the best
features of HPDM, including several of the meteorological pre-processor formulations and a
convective PDF algorithm, are incorporated in AERMOD.
G-6 - Nearly all of the many figures need improvement. Figure numbers are hand-written
and captions are often non-existent. The captions should contain sufficient information that
the figure "stands alone" in the sense that the reader can understand it without referring to
the text.
-------
REPORT II. AERMIC' S FIRST RESPONSE
The final evaluation document will attempt to incorporate improvements suggested in this
comment.
G-7 - The model evaluations should include a measure of the scatter, such as VG or NMSE.
The five-page discussion of the results should be greatly expanded to adequately cover what
is presented in 27 figures and 7 tables.
For tracer data sets, the box and whisker plots provide a visual means to assess the prediction
scatter. For full-year data sets, the sparse coverage of the monitors does not lend itself to
meaningful scatter analyses. In terms of the discussion of results, the report that covers all of the
evaluations will not be significantly increased in size, so as not to overly burden the reader. The
figures and tables speak for themselves.
G-8 - The AERMOD interface is a key component of the model, providing complete profiles
of wind, temperature and turbulence from the surface up to 5000 meters above the surface,
extrapolating data up to that level from even the barest minimum (10 m) measurement
height. The Panel recommends that some evaluation of the performance of the interface be
provided as part of the overall evaluation of the model. Evaluation of the interface can be
accomplished by performing sensitivity tests and/or by comparing predicted profiles against
data. Comparisons against data can be accomplished by utilizing meteorological
measurements alone, which should increase the number of potential data sets considerably
over those that have concurrent measured pollutant concentrations. Of particular interest
is how the interface performs when driven by minimal data, and further how sensitive it is
to small changes in input parameters such as Zim.
PES has conducted a limited number of sensitivity runs with degraded meteorological data sets to
attempt to address the concerns stated in this comment. The results of the analysis will be part of
the EPA modeling conference docket.
G-9 - AERMOD has a tendency to underpredict the upper end of the frequency distribution
at Martins Creek even though the robust high concentration - RHC - is overpredicted
slightly for 24-hr averages. Marginal performance from a regulatory perspective should
trigger a more detailed review of the predictions to find evidence that would (hopefully)
support the model predictions. An examination of the emission rates - total over all sources
- associated with predicted and observed concentrations, however,
suggests that the model could underpredict substantially when considering the average
emission rate of the highest 25 predicted observed and modeled concentrations (Table 4.10
of the Phase II evaluation document; avg during obs is approx 1300, during pred.
2300 g/s). This tendency, coupled with an even more distinct tendency to underpredict the
upper end of the frequency distribution at Lovett where normalized concentrations were
reported, suggests that further model formulation changes may be called for. The Panel
-------
REPORT II. AERMIC' S FIRST RESPONSE
believes that the tendency to underpredict at Martins Creek might be influenced by the
interface's prediction of less stable temperature gradients at plume level than those
used by other models (thereby resulting in higher plume rise and lower Hcrit). Evaluation of
the interface, and further diagnostic analysis of the meteorological conditions associated
with high predicted and observed concentrations, could provide some insights into whether
this is the case and could potentially lead to changes in the interface as opposed to changes
in the terrain interaction formulation.
In response to the peer review panel comments, the AERMOD terrain treatment was modified to
incorporate additional CTDMPLUS-like features, and two additional data bases (both involving
complex terrain) were added to the list (Westvaco and Tracy). The performance results of the
new AERMOD model address the concerns discussed in this comment.
[Detailed responses to line-by-line comments in the original report are removed for brevity]
-------
REPORT III. HI! FINAL PEER REVIEW REPORT (Dated Mar 3, 1999)
1 Background
The EPA has assembled a peer review panel in order to review the various documents produced
by the AMS/EPA AERMIC group on the new AERMOD dispersion model. The EPA is
proposing to possibly replace the ISCST3 regulatory model with the new AERMOD dispersion
model, and asked the panel of experts in air dispersion modeling to provide specific technical
comments on the documents as well as to provide answers to a set of questions concerning
whether AERMOD is ready for use as a regulatory model.
After the Peer Review Panel was set up in mid-March 1998, a group of AERMOD documents was
delivered to each panel member. A total of seven documents were reviewed which included one
report describing the technical formulation, three reports describing model evaluations, and three
user's guides. The Panel was also asked to address a set of twelve general questions related to
whether AERMOD was ready for use as a regulatory model. Two conference calls were held on
2 and 3 April discussing the between the Panel, the AERMIC group, and ICF Kaiser. A draft
report was prepared by the panel and delivered to EPA on 23 April.
During the intervening period, AERMIC has taken our comments into consideration and has
revised the "Model Formulation" and "Model Evaluation" documents, carried out evaluations
with additional field data sets, and prepared a document containing point-by-point responses to
our comments. This information was provided to the peer review panel on 23 December 1998 for
additional review. This report contains comments on the revised "Model Formulation" and
"Model Evaluation" documents and the AERMIC response document. In addition, we have
revised our responses to the set of questions posed by the EPA to the Peer Review Committee.
-------
REPORT III FINAL PEER REVIEW REPORT
2 INTRODUCTION AND GENERAL COMMENTS
We believe that AERMOD represents a significant improvement over ISC3. There are many new
state-of-the-science concepts and approaches in AERMOD, including a unique "interface" that
creates complete temperature, wind and turbulence profiles from even the barest minimum input
data. However, we feel that, because of these new approaches, AERMOD is more likely than
simpler models like ISC3 to need an extended break-in period when its use in routine applications
can be thoroughly tested. It is worth noting in this regard that all of the AERMOD evaluation data
bases (except for Prairie Grass) involved tall, non-downwashed, highly buoyant power plant
stacks (the shortest stack in the group was 84 meters in Indianapolis). The vast majority of ISC3
applications involve modest stacks with modest buoyancy flux values, most of which are subject
to some degree of aerodynamic downwash, as well as area and volume source configurations
which were not evaluated or tested (at least not in the documentation provided) to determine how
AERMOD predictions compare to predictions using ISC3.
We are concerned about the accuracy of the concentrations predicted by the downwash algorithm
in AERMOD, since the same downwash algorithm is used in both ISC3 and AERMOD.
Considering that something has to be incorporated, it seems alright for the present to include the
convoluted H-S and S-S procedures that are currently incorporated into the ISC3 models with the
simplification that variances will be added rather than using virtual sources. Then if the PRIME
research project makes an appropriate case for alternate procedures, these can be included in the
next AERMOD. Hopefully, that would be a substitution, not a melding of H-S, S-S and PRIME.
In the course of applying AERMOD to many different sources, in many different settings, with
many different meteorological data bases, users may discover aspects of AERMOD that would be
desirable to change. It is possible to minimize the number of situations where this would occur by
conducting a thorough, exhaustive, independent set of sensitivity tests aimed at understanding the
underlying reasons for model performance and the interrelationship of model components, rather
than just the end results of the model. Such a thorough analysis is not likely to happen in the near
future, however, and it therefore might be appropriate to allow for an interim period when
AERMOD can be accepted as a "refined" model but that its use would not be required. This
would be especially appropriate since some applications cannot be correctly handled by
AERMOD (e.g., those involving deposition) and some algorithms (e.g., downwash) are likely to
experience additional changes in the near future.
Our basic conclusion is that AERMOD is ready to be proposed as a replacement to ISC3.
However, it would be a mistake to treat AERMOD as "finished" and not needing any further
evaluation or dialogue regarding its performance and its use as a routine model. It is
acknowledged by AERMIC that model changes, involving downwash and deposition in particular,
are going to be made in the next year or two. We suggest that the period during which these
additional changes are being evaluated and implemented could be regarded as an interim period
-------
REPORT III FINAL PEER REVIEW REPORT
where AERMOD can be used as a "refined" model. The use of AERMOD could be conditioned
on developing site-specific information regarding its performance, possibly relative to ISC3 and
the performance of the model's interface in terms of generating meteorological profiles. After an
interim approval period, the information and model improvements (if any) generated through
these comparisons and evaluations could be assessed. An advantage of the interim approval
period would be the generation of a wealth of information about the model and its performance
and use in the real world.
-------
REPORT III FINAL PEER REVIEW REPORT
3 General Comments on AERMIC Responses
to Peer Review Committee Report
The document was prepared by the AERMIC committee in response to our April 1998 AERMOD
Peer Review Report. In addition, AERMIC has provided a revised Model Formulation document
and Model Evaluation document. Over the past month, we have read the three documents and
feel that the Peer Review Committee should be pleased that so many of our suggestions have been
seriously considered and satisfactorily addressed by the AERMIC committee.
The AERMIC responses suggest that they were disappointed that we did not provide more
definite answers to their 12 general questions. However, we believe that we were not provided
sufficient information to allow us to reach definite answers. The latest documents have improved
the situation. Yet, even though AERMOD represents the state-of-the-art better than ISC3,
AERMIC has not seriously considered similar models such as SCIPUFF or OML. On page 4 of
their response, AERMIC states that"SCIPUFF's public availability as an operational model with
full documentation was limited when our development process began". But that was 8 or 9 years
ago. Since then SCIPUFF has been further developed and has been in the public domain with
comprehensive documentation for several years.
Despite these reservations, our peer-review committee endorses the substitution of AERMOD for
ISC, since AERMOD does represent the modern class of dispersion models and does provide an
improvement to the current EPA models. Furthermore AERMOD does demonstrate satisfactory
agreement with the various field data bases used for its evaluation. However, as mentioned in
Section 1, none of these field data bases involved downwash.
AERMIC and the EPA still need to provide better guidance on the use of meteorological data for
inputs. Also, they should carry out more comprehensive sensitivity tests in order to be sure that
the model does not produce odd predictions under certain combinations of inputs. These
sensitivity tests should include many types of real-world scenarios where downwash occurs.
-------
REPORT III FINAL PEER REVIEW REPORT
Comments on Vertical profiling of dq/dz: We were interested in how dq/dz is estimated,
especially when no measurements are available, as with Martins Creek. We interpreted Figure 4
as a representation of what AERMOD generates (through its interface) when measurements are
not available, given assumed micrometeorological parameters that represent a "strongly stable
boundary layer" (MFD, p. 29). This figure raises several issues. For example, we were not able
to reproduce Figure 4, given the parameters stated in the text and using equation (31). This may
be simply our misunderstanding of how the figure was developed, but it may be a good candidate
for an example illustration in an AERMOD training course.
If it is true that this curve (Figure 4) represents a typical profile of dq/dz as generated by the
interface when there are no measurements available, then it is worth noting what the implications
are for Martins Creek. At 183 meters (the height of the MC stacks), dq/dz essentially disappears,
subject to the minimum value of 0.002 K/m. A sense of what this means in terms of plume rise
can be gained by examining the above table, which represents calculated stable case plume rise for
a highly buoyant stack, based on AERMOD formulas (version 98314).
These results show that, if lower potential temperature gradients are frequently predicted by the
interface, then the difference in model performance between AERMOD and the other models
(where low wind-speed, stable conditions will result in a much lower plume rise) is not hard to
understand. In practice, however, shorter stacks may experience temperature gradients that are
actually more conservative than the ISC3 defaults of 0.02 and 0.035 and the relative performance
picture may be very different.
Example of using AERMOD plume rise for a highly buoyant stack
Plume rise calculations
wind speed final stable neutral length neutral limit calm rise (eq distance to unstable p.r.
Inputs
plume rise scale (eq. 124 - (eq. 123)
125)
final rise (eq. 116) @
(eq 122)
see note)
(stable)
dfr (stable)
0.02
0.002
dtheta/dz
0.0259
0.0082
BVF
Using dtheta/ dz - 0.02
0.0181
0.0057
BVFPrime
1
276
48722
62968
324
167
454
293
293
TA
2.5
203
19489
25156
324
417
335
450
450
TS
5
161
9744
12578
324
834
266
5
5
DS
35
35
VS
Using dtheta/dz = 0.002
9.816
9.816
G
1
595
48722
62890
769
542
979
0.6
0.6
Betal
2.5
438
19489
25156
769
1354
721
0.124
0.124
u*
5
348
9744
12578
769
2708
573
749.2
749.2
FB
4985.1
4985.1
FM
Note: equation 124 in model code has ustar*ustar
in the denominator
1680.3
1680.3
Distance to final rise (uns)
183
183
stack height
-------
REPORT III FINAL PEER REVIEW REPORT
Some natural questions arise from this, including 1) how good a representation of vertical dq/dz is
the interface's predictions, and 2) how sensitive are the predictions to the mechanical mixing
height, a parameter that is not well defined and extremely difficult to measure/verify? These
questions should be part of the ongoing dialogue as AERMOD continues to be evaluated and
improved.
Other model formulation comments: Concerning the development of horizontal and vertical
dispersion parameters, equation (96) does not appear to be correct, either as the result of making
the indicated substitutions or in comparison to the formulation appearing in the model code
(subroutine SIGZ). 2) The discussion on the contribution of plume meander was difficult to
follow in the MFD and the model code. A sketch would be useful, 3) The plume rise discussion
was relatively easy to follow, except that the issue of the effect of random vertical motions in the
CBL was taken out of the plume rise section altogether. It would help to have a reference to eq.
64 in the section on plume rise in the CBL (Section 6.5.1), just to tie things together a little more
neatly. Furthermore, if it is intentional not to calculate a "final rise" in the CBL, it would be
useful to say so in section 6.5.1 - especially since, in the discussion on vertical inhomogeneity
(section 4.2), reference is made to the distance to final rise and the calculation of the
homogeneous layer limits is based in part on the "final" plume height.
Comments on AERMOD Interface: The interface contained in AERMOD that provides complete
profiles to the model is based on approaches that are consistent with the current state-of-the-
science regarding profiles of wind, temperature, and turbulence in the boundary layer and their
relationship to micrometeorological variables. There was little attempt in the evaluation of the
model, however, to address the question of how well the interface itself works, whether it is
sensitive to parameters that are poorly understood and difficult to measure, and whether the entire
interface approach is internally consistent.
Comments on Minimum Meteorological Data Requirements: As we understand the
recommendation in this document, there is no reason to require that any data be collected beyond
what is now routinely available (i.e., NWS data). The implication of this is that the interface
works well enough to eliminate the need for collecting profiles of wind, temperature, and
turbulence values. However, an unambiguous link has not been established between improved
inputs and better model performance, i.e. the modeled atmosphere seems to do as well as the
measured atmosphere.
We do not think that AERMOD should be held back because of these concerns, but we also
believe that its implementation should be accomplished in a way that the evaluation and the
dialogue concerning the profiling of meteorological variables would continue. In an interim
implementation period, this dialogue could be continued by requiring that the use of AERMOD be
accompanied by an analysis of the performance of the interface at the site in question - a
statistical summary of profiled values, for example, could be generated and presented as part of a
representativeness analysis. The performance of the interface for many different settings could be
generated quickly, and an evaluation after the end of the interim period could provide additional
-------
REPORT III FINAL PEER REVIEW REPORT
evidence of the performance of the interface and possibly point the way to improvements - or, if
the interface evaluations reveal nothing worthwhile, the requirement could be dropped.
[Sections 4 and 5 contained the detailed, line-by-line comments and are omitted for brevity. In the
last report, the AERMIC final responses, there is a general discussion response to these two
omitted sections and provide a sense of the overall detailed comments]
-------
REPORT III FINAL PEER REVIEW REPORT
6 Comments on AERMOD Model Performance
Document Dated 15 Dec. 1998
On balance, the evaluations demonstrate relatively unbiased performance over a wide range of
source and receptor relationships reflected in a large number of data bases. Additionally,
AERMOD tends to do better over most of the frequency distribution than most of the other
models tested. However, at least part of AERMOD's good performance is due to adjusting the
model to fit the data (e.g., complex terrain changes were made that affected model performance at
Martins Creek and Lovett, but these changes were apparently made before the evaluations for
Tracy and Westvaco). However, even if this is true, good performance over a wide range of data
bases is still significant in terms of the likely performance of the model in the real world.
We have some remaining questions about model performance in complex terrain. The comments
that we presented in our draft report were first, that the model performance at Lovett showed a
distinct tendency towards underprediction at the upper end of the frequency distribution for all
averaging periods, and second, that a slight tendency towards underprediction at Martins Creek
did not stand up to further investigation when considering emission rates for predicted values vs.
emission rates for observed values. AERMIC responded to these comments by saying that first,
the incorrect plots had been provided for Lovett (the correct plots involved concentration
comparisons, not normalized C/Q comparisons). Furthermore, some modifications to the
complex terrain algorithms were made that improved model performance. And finally, AERMIC
stated that the use of C/Q statistics is not warranted since the "low emission hours for Martins
Creek are probably associated with full-load emissions from one or more of the shorter stacks,
which were likely responsible for the bulk of the observed impacts".
The model changes had a noticeable effect on the upper end of the frequency distribution,
particularly for Martins Creek. In the latest evaluation document, concentration Q-Q plots are
provided for Lovett for 1-hr, 3-hr and 24-hr averaging periods and C/Q Q-Q plots are provided
stratified by stable/unstable conditions. It is unclear from this whether the C/Q or the
concentration comparisons are being suggested as the more correct comparisons. When impacts
are clearly associated with a single source, C/Q comparisons are the most appropriate to use and
therefore there is still some question as to AERMOD's performance at Lovett. We agree with
AERMIC that at Martin's Creek it would probably be impossible to separate out the effects of
different sources. However, if what AERMIC contends is true (that full-load emissions from
short stacks cause high observations during low emission hours for Martins Creek), then
AERMOD must be under-predicting the impacts from the shorter stacks since the shorter stacks
are modeled at close to full load for most of the data base.
These concerns are not enough to conclude that AERMOD is so flawed that it cannot be
recommended (particularly since AERMOD performed so well with the Tracy SF6 data base). It
is clear from the evaluations, however, that CTDMPLUS has a distinct tendency to overpredict by
-------
REPORT III FINAL PEER REVIEW REPORT
a substantial margin in situations (Lovett and Martins Creek - and to a lesser extent Westvaco)
where emissions from a highly buoyant source are at an elevation just above terrain where
monitors are located. AERMOD has a tendency to predict much lower values for these situations,
and if chi/Q statistics are considered - or at the very least, emissions are taken into consideration
qualitatively - performance becomes questionable. These results should be further analyzed and
explained, in order to add a comfort level that the AERMOD predictions are more believable.
It is possible, additionally, to question the performance of AERMOD for surface releases and in
urban settings, since for Prairie Grass and Indianapolis there is a slight tendency for
underprediction relative to observations and relative to ISC3 (for Prairie Grass, the
underprediction tendency appears both in the comparison of model predictions to RHC and in the
1-hr Q-Q plots, while for Indianapolis the tendency appears only in the Q-Q plots). Whether a
regulatory model can be accepted for general use, even if it displays some tendency - albeit slight -
to underpredict under some conditions, is a question that is important to consider. It is much
easier to accept a model like ISC3 for regulatory applications based on fewer and less extensive
evaluations, with its clear tendency to overpredict. AERMOD's performance has many desirable
features, but its predictions are occasionally less than observations. The EPA should be open to
further evaluations aimed at bolstering the current set of results - and making improvements if
necessary - during an interim model approval period.
Although the authors make a point when they say that this model has used more evaluation data
sets than previous models, we are concerned that only one evaluation data set related to surface
releases (Prairie Grass) has been used. Because it has been found that the S02 tracer was
actually being deposited to some extent with downwind distance, we wonder if AERMIC has
accounted for this mass loss in their evaluations. Furthermore, the results of the evaluation shown
in Figures A-3 and A-5 indicate that the model tends to underestimate concentrations. Also see
Figures A-4 and A-6. Is this a desirable attribute for a regulatory model?
-------
REPORT III FINAL PEER REVIEW REPORT
7. Revised Answers to EPA Questions Posed in
Original Charge to Reviewers
EPA General Topic Area 1. Model Formulation
EPA General Question 1.1 - As a steady-state, plume-based, regulatory model, does AERMOD
represent the state-of-the-science in its handling of boundary layer turbulence and dispersion?
AERMOD embodies state-of-the-science approaches to boundary layer turbulence and dispersion.
EPA General Question 1.2 - Within the context of regulatory dispersion models in the US,
does AERMOD represent significant scientific advances over ISC3?
AERMOD represents significant scientific advances over ISC3. However, AERMOD is similar
to other available state-of-the-art models such as SCIPUFF, OML, ADMS, CTDM, and HPDM.
EPA General Question 1.3 - What do you think are the most important scientific advancements in
AERMOD?
AERMOD incorporates several scientific advancements, including the use of convective scaling
and non-Gaussian pdfs of vertical velocity in convective conditions. Also, dividing streamlines
are used for complex terrain. The vertical profiles of meteorological variables are developed
based on state-of-the-art methods. However, the peer review committee mentions that such
advancements are available is other models as well, such as SCIPUFF, CTDM, and HPDM.
EPA General Question 1.4- Are there any areas or features of AERMOD in which an improved
formulation or treatment would be desired? If so, please discuss whether you think the revised
treatment would lead to better performance and how much.
The urban dispersion algorithms need more development and justification. An updated
downwash algorithm is desired. It would be useful to remove the tendency towards
underpredictions. The specification of meteorological inputs should be clarified.
2. Documentation
EPA General Question 2.1 - Is the current organization of the model formulation document
and User's Guide appropriate or would an alternative be desired?
The separation of the technical details into a model formulation document (particularly the much-
improved, current version of the MFD) is fine. The User's Guide is well-organized and useful.
-------
REPORT III FINAL PEER REVIEW REPORT
EPA General Question 2.2 - Is the presentation of the model clear and explanatory? Please
note any specific sections of the documentation that were unclear or confusing.
The peer review panel had numerous concerns regarding the original March 1998 documentation,
mostly regarding the MFD. The current version of the MFD is an improvement over the last
version, and will provide users with a good description of the model. Further justification of
algorithms is needed.
EPA General Question 2.3 - Is the documentation sufficient for a typical ISC-type user to guide
them in the use of the model and its preprocessors? Do you think training sessions would be
particularly useful?
The documentation is sufficient. Because the model is not simple, training sessions would be
helpful for most users.
3. Evaluation and Performance
EPA General Question 3.1 - How do you rate the performance of AERMOD relative to ISC3
and the other models included in the evaluation exercises?
Overall, the performance of AERMOD is good over a wide range of scenarios. Some of the
evaluations of AERMOD show a slight tendency for AERMOD to underpredict near the upper
end of the frequency distribution. The model needs more evaluation for downwash scenarios.
EPA General Question 3.2 - From a model design, scientific, and performance perspective,
what comments do you have on the replacement of ISC3 with AERMOD for regulatory
applications?
The replacement of ISC3 with AERMOD for regulatory applications is appropriate. We
recommend that this be done in a staged process so as to allow for testing of the new algorithms
and the development of guidance for optimum meteorological inputs. All of the AERMOD
evaluation data bases (except for Prairie Grass) involved tall stacks with buoyant plumes (the
shortest stack in the group was 84 meters in Indianapolis), with little probability of downwash
occurring. The vast majority of ISC3 applications involve modest stacks with modest buoyancy
flux values, most of which are subject to some degree of aerodynamic downwash, as well as area
and volume source configurations which were not evaluated or tested (at least not in the
documentation provided) to determine how AERMOD predictions compare to predictions using
ISC3.
EPA General Question 3.3 - When considering the eight data bases used to evaluate the
model, would additional evaluation of AERMOD be desirable?
The number of data bases is actually ten, with the addition of Tracy and Westvaco. Additional
evaluations of AERMOD in urban settings, for surface releases, and for downwash impacts would
-------
REPORT III FINAL PEER REVIEW REPORT
be desirable, as would further diagnostic evaluation of model performance in complex terrain.
But use of the model should not be delayed pending the completion of these evaluations. An
"interim" approval status might be appropriate for AERMOD while further evaluations are
conducted, in particular while the downwash and deposition algorithms are being subjected to
further testing and modification.
Additional EPA Question 4.1 Dated 3/18/98 - Is the AERMOD approach to modeling urban
sources scientifically sound and state of the art?
The new urban formulation of AERMOD embodies many parts of a state-of-the-science approach.
There are still issues remaining concerning the method of estimating stability in urban areas.
Additional EPA Question 4.2 Dated 3/24/98 - Do the building downwash algorithms within
AERMOD represent the current state of science and are these algorithms appropriate to
regulatory applications?
The building downwash algorithms in AERMOD are essentially the same as those currently in
ISC3. Consequently, they should be suitable for regulatory applications. However, there is no
information in the documentation provided as to how these algorithms work within AERMOD,
and none of the evaluation data sets concerned downwash.
-------
REPORT IV. THE FINAL RESPONSES FROM AERMIC (Dated June 7, 1999)
1. Introduction
In the following, AERMIC responds to comments provided in the AERMOD Peer Review Final
Report (Hanna et al., 1999) that assessed several documents describing the dispersion model
AERMOD. In March 1998, a Peer Review Panel was established by EPA to review AERMOD, a
new short-range dispersion model for industrial source applications and a potential replacement
for the ISC3 Model. The Panel reviewed numerous documents pertaining to the formulation, user
guidance, and evaluation of AERMOD and produced a Peer Review Report (Hanna et al., April
1998). As a result of the Panel's comments, AERMIC has: 1) revised and expanded the
AERMOD Model Formulation Description (MFD, Cimorelli et al., 1998), 2) modified the
AERMOD code accordingly, 3) evaluated the complex terrain portion of the model with two
additional data bases (the Tracy and Westvaco data sets), and 4) revised and clarified the model
evaluation documentation (Paine et al., 1998). The Panel has reviewed these revised documents,
provided comments on them, and modified their responses to a set of twelve general questions
posed by the EPA (Hanna et al., 1999).
As expressed in our earlier response (AERMIC, December 1998), AERMIC is grateful to the Peer
Review Panel for their time, effort, and completeness in examining the revised AERMOD
documents and providing their assessment. The basic conclusion of the Panel is that AERMOD is
ready to be proposed as a replacement for the ISC3 Model for regulatory air quality
applications,but they have some comments on further AERMOD development and evaluation
work. In the following, we summarize and respond to what we perceive as the major issues or
questions by the Panel concerning AERMOD, its evaluation and implementation. This is done by
section of the final report (Hanna et al., 1999), but omitting Section 1 on "Background."
2. Response to Section 2: Introduction and General Comments
The Panel states that they believe AERMOD to be a significant improvement over ISC3 and that
it contains many new state-of-the-science concepts and approaches including a unique
meteorological interface. Their basic conclusion is that AERMOD is ready to be proposed as a
replacement for ISC3. They raise four issues or concerns discussed below: 1) limitations of
source types in the AERMOD evaluation, 2) the downwash algorithm, 3) the need for more
experience with AERMOD results and sensitivity tests, and 4) the need for a "break-in" period.
2.1. Source Types in AERMOD Evaluation
The Panel noted that with the exception of Prairie Grass (surface release), all of the AERMOD
evaluation data bases involved buoyant releases from tall stacks at power plants. They also noted
that the majority of ISC3 applications pertain to "modest" buoyancy fluxes and "modest" stacks,
many of which are subject to aerodynamic downwash.
AERMIC agrees that further testing and evaluation with "intermediate stack height" data bases
would be desirable and would merit attention in the future given appropriate data sets and the
resources to analyze them. We note two factors to reduce this concern. First, AERMOD has been
-------
REPORT IV. FINAL AERMIC RESPONSE
formulated as a continuous function of source height (z_s) and buoyancy flux and tested in the
limits of: 1) a nonbuoyant surface release (Prairie Grass), and 2) buoyant releases in the upper part
of the atmospheric surface layer (lowest 10% of the boundary layer) or in the mixed-layer (upper
90%) of the convective boundary layer (CBL). Simple interpolation formulas are used that give a
continuous variation of dispersion (oy, oz) with zs between these limits. Thus, we believe that the
parameterization of dispersion from intermediate stack heights should be reasonable.
Second, AERMIC has conducted a consequence analysis in which AERMOD and ISC were run
for a large number (72) of source, meteorological, terrain, and urban/rural combinations. Thus, the
difference between the AERMOD and ISC results or their similarity can be determined for the
type of source, source height, etc. of interest.
2.2. The Downwash Algorithm
AERMIC plans to introduce an improved building downwash algorithm into AERMOD as
resources permit. This could include the EPRI-sponsored PRIME algorithm or another alternative
that overcomes the deficiencies of the existing ISC3 downwash model. AERMIC notes that the
existing AERMOD downwash algorithm may be an improvement over that of ISCST3 because of
the associated advances in the treatment of meteorological profiles in AERMOD. During the
period in which the downwash model is incomplete, AERMOD can be used for other types of
applications or for mixed applications in which the downwash algorithm does not dominate the
highest concentration predictions. If the use of the current AERMOD with downwashed sources
shows that other effects, such as terrain impacts, dominate, then AERMIC suggests that
AERMOD can be used to assess compliance with ambient standards. Alternatively, another
model such as ISC-PRIME could be used to assess a limited receptor area where downwash
effects dominate. Since near-field downwash effects are likely to be influenced mainly by source
effects rather than atmospheric turbulence, the use of ISC-PRIME in the near field and AERMOD
everywhere else should be considered as a possible interim solution.
2.3. Additional AERMOD Experience
The Panel recommends that AERMOD be applied to a wide variety of sources, terrain settings,
and meteorological data bases to learn more about the model behavior, i.e., conduct a large
number of sensitivity tests. Exercises of this type have been conducted under two activities: 1) a
Technology Transfer Workgroup (TTW) during 1998, and 2) a consequence analysis to compare
AERMOD and ISC results.
2.3.1. TTW Activity
In 1998, EPA under the leadership of Mr. Robert Wilson of Region 10, assembled a TTW, to
conduct a variety of test runs of AERMOD. The TTW had several participants from a number of
states, EPA Regions, and even a representative from British Columbia. The numerous trial runs
of AERMOD conducted by the TTW on a variety of source types provided valuable feedback to
AERMIC and resulted in the elimination of some "odd predictions under certain combinations of
-------
REPORT IV. FINAL AERMIC RESPONSE
inputs" referred to by the Panel. This is not to say that our work is complete in this area.
AERMIC welcomes results from other work similar to that conducted by the 1998 AERMOD
TTW, either by organized groups or individuals who are exercising the model.
We note that ISCST3 itself is not immune to unusual predictions caused by discontinuities in its
formulation, or by known problems that have gone uncorrected due to their low priority. This has
not prevented ISCST3 from being promulgated and maintained as a guideline model. AERMOD
has been formulated and tested to eliminate, to the extent possible, any problems. However,
AERMIC acknowledges that with testing conducted over time, adjustments in certain algorithms
may be necessary.
2.3.2. Consequence Analysis
The consequence analysis is designed to provide regulatory design-concentration comparisons
between old (ISC3) and new (AERMOD) models for a number of typical source scenarios. The
purpose of such a study is to give the modeling community a sense of the regulatory impacts of
using the new model for source types that they have or may evaluate. Although not designed to be
an exhaustive sensitivity study, the consequence analysis for AERMOD and ISC3 included 72
combinations of source, meteorological, terrain, and urban/rural situations.
Although similar to the earlier consequence analyses in that 2 meteorological data bases and 2
different land classifications were used (urban and rural), the AERMOD consequence analysis
evaluated all three types of sources (point, volume and area) in three types of terrain (flat, simple
and complex). There were 24 flat terrain point source combinations (6 stack heights, 2 met sites,
2 land classifications); there were 8 simple terrain point source combinations (2 stack heights, 2
met sites, 2 land classifications); there were 8 volume source combinations (2 stack heights, 2
met sites , 2 land classifications); there were 4 area source combinations (1 release height, 2 met
sites, 2 land classifications); and there were 32 point source combination for the complex terrain
(2 stack heights, 2 buoyancy types [medium and high], 2 distances to the hills, 4 hill types).
We believe that this consequence analysis is comprehensive in that most reasonably expected
source combinations have been evaluated by the latest version of AERMOD and successfully
completed. Of course, other analyses such as the evaluation -data- base reruns became part of the
testing of this last version of the model; thus, the testing of the model did not end with the
consequence analysis. These many computer runs have satisfied AERMIC that the current
version of the model is stable and does not provide erroneous results for the large majority of
source combinations.
2.4. Break-In Period
The panel recommends that in view of the new approaches integrated into the AERMOD
modeling system, there should be a time period during which the public gains experience with this
new model, i.e., a break-in period. During this period, the panel notes that some enhancements to
AERMOD (e.g., downwash, wet and dry deposition) also could be added as well as further model
-------
REPORT IV. FINAL AERMIC RESPONSE
evaluations. AERMIC believes that the EPA OAQPS will establish something akin to a break-in
period during which both AERMOD and ISC3 may be used.
3. Response to Sections 3 and 4: General and Specific Comments on AERMIC Response
Document (December 15, 1998)
The comments in these two sections deal primarily with the AERMIC response (December 1998)
to the Panel Peer Review Report of April 1998. As such they pertain to the earlier version of the
AERMOD MFD (March 1998). The focus of the current response document is the Panel review
of the most recent versions (December 1998) of the MFD and the model evaluation document.
Thus, with the exception of the AERMOD development process discussed below, we do not
address the discussion of the earlier AERMOD documents.
In a number of places, the Panel questions AERMIC's consideration of submodels and algorithms
(dispersion, etc) from other models—ADMS, HPDM, OML, and particularly SCIPUFF—in
developing AERMOD. We point out, however, that there are important links of AERMOD to
earlier models. For example, the PDF model for buoyant plume dispersion in the CBL (Weil et al.,
1986) was adopted in HPDM (Hanna and Paine, 1989; Hanna and Chang, 1993). This model has
been modified since the HPDM version to deal more effectively with highly-buoyant plumes and
CBL turbulence, including near-neutral conditions (Weil et al., 1997). This modification was
made to provide a continuous variation of the modeled concentration field with source buoyancy
and stability. The main point, however, is that the PDF model has an HPDM connection.
Likewise, the preprocessing of meteorological data in AERMET is similar to that done for HPDM
(Hanna and Paine, 1989; Hanna and Chang, 1993) and CTDMPLUS (Perry, 1992) as stated in the
most recent AERMOD MFD (December 1998); several of the processing schemes were borrowed
from these earlier models. In addition, most or many of the profiling expressions for winds,
temperature, and turbulence are similar to or are borrowed directly from earlier profile models as
referenced in the Interface section of the MFD.
For terrain effects, much of our initial thinking of the problem was guided by concepts in
CTDMPLUS, as indicated in the most recent MFD, e.g., the dividing streamline height and its
dependence on wind speed, stratification, and hill height. However, our intent was to simplify
considerably the concentration calculations for terrain effects by comparison to the treatment in
CTDMPLUS, where subjectively-defined, idealized hill shapes and considerable terrain data are
required to define them. In addition, the AERMOD vertical dispersion (oz) model for elevated
plumes in stable conditions has its origin in the early CTDMPLUS developments.
For AERMOD, our main aim was to develop a simple plume model for routine air quality
predictions much in the spirit of ISC3, but with dispersion and other processes based on
state-of-the-art understanding. Three of our key design goals were to: 1) provide reasonable
concentration estimates over a wide variety of conditions with minimal discontinuities, 2) be user
friendly and require reasonable input data and computer resources as in the current ISC3 model,
and 3) capture the essential physical processes while remaining simple.
-------
REPORT IV. FINAL AERMIC RESPONSE
In considering SCIPUFF, part of our rationale for not pursuing it was: 1) the lack of complete
documentation for the model when the AERMIC activity began, and 2) the absence of treatment
for a number of conditions (full elevated terrain, downwash, etc) required by a new US regulatory
model. SCIPUFF treatments for several of these conditions have been added since the AERMIC
activity began. However, another major reason for not pursuing SCIPUFF was its basis—as a puff
or integrated puff (for plumes) model and the detailed, numerically-intensive calculations and
resources required for its operation. This contrasts with the relative simplicity of the plume
models such as AERMOD and ISC3. Note that our goal was for the new model to be run with a
year or perhaps five years of meteorological data to identify the worst-case concentrations for
comparison with air quality standards. Our understanding is that SCIPUFF is not intended nor
well-suited for operation in this mode. Instead, it is intended for modeling shorter time periods
(e.g., 24 hours or less) in a more intensive and detailed manner (Sykes, 1999; private
communication to R. Paine).
AERMIC is very interested in the numerous field data sets with which SCIPUFF has been
evaluated as reported in the Panel final report (Hanna et al., 1999). We are especially interested in
those data sets that fill some of the evaluation needs of AERMOD—continuous surface sources,
short stacks in the planetary boundary layer, downwash situations, etc. We would be grateful for
references to all of these evaluations, including those in the peer-reviewed literature.
In summary, we believe that AERMOD incorporates some of the best features of the other models
mentioned by the Peer Review Panel. AERMIC is pleased that the Panel endorses the substitution
of AERMOD for ISC.
4. Response to Section 5: Comments on AERMOD Description of Model Formulation: Dated 15
December 1998
In the following, we respond to the discussion of: 1) specific comments on the MFD, 2) vertical
profiling of meteorological data, 3) other model formulation comments, and 4) minimum
meteorological data requirements.
4.1. Specific Comments on the MFD
The Panel lists a number of specific comments by page (in the MFD) such as justification for
certain assumptions, editorial comments, etc. AERMIC intends to update the MFD with further
clarifications, corrections, etc. and these comments will be addressed in a future version of the
MFD. Furthermore, the MFD will be updated on a continuing basis in the future as algorithms are
modified and/or improved. In addition, AERMIC intends to publish several journal articles
describing AERMOD and its evaluation. Many of the specific comments listed on pages 7 and 8
of the Panel Report also will be dealt with in these articles.
With respect to the urban boundary layer, further discussion of the model treatment of the
nighttime situation will be given in an updated MFD. We note that in Eq. (129), a coefficient of
0.1 preceding the In (P/P0) term was omitted, but this was included in the AERMOD code. With
-------
REPORT IV. FINAL AERMIC RESPONSE
this coefficient included, the urban - rural temperature difference is positive for population P >
100; a simple modification is being added to maintain a positive (or zero) temperature difference
for all P.
4.2. Vertical Profiling of Meteorological Data
The Panel comments about the lack of an evaluation of the AERMOD Interface and its potential
sensitivity to poorly-understood or difficult-to-measure parameters. The Interface uses
parameterized profiles of wind, temperature, and turbulence that are generally consistent with
profiles obtained by other investigators and that have been compared with observations (e.g., see
the Paine and Kendall (1993) reference cited in the MFD as well as the Stull (1988) textbook and
other references in the MFD). We have relied on these earlier model/observation comparisons for
Interface support since there has been a fair amount of testing in this area, and we did not wish to
repeat the same comparisons. However, this is not to say that the algorithms will always work
perfectly, or not be overly sensitive to variations in certain input data. AERMIC welcomes
reports of the modeling community's experience with the model in this regard.
A specific mention is made by the Panel of the temperature gradient profiling algorithm. The
panel notes that the expected drop-off of the temperature gradient with height in stable conditions
results in better AERMOD model performance for tall stack sources. It notes that the performance
could be worse than ISC3 for short stacks since the AERMOD temperature gradients could be
greater than the ISC default values. However, we note that the plume rise depends on the
temperature gradient to the 1/3 power. Thus, for gradients a factor of 1.5 or 2 greater than the ISC
values, the plume rise would be only 13% and 20% lower, respectively, than the ISC plume rise.
As noted earlier, a number of AERMOD/ISC3 comparisons have been carried out in the
Consequence Analysis for different stack heights, buoyancy fluxes, stabilities, and downwind
distances. One can consult this document to gain an idea of the AERMOD/ISC concentration
differences for a particular scenario of concern.
4.3. Other Model Formulation Comments
These comments are generally of an editorial nature or require further explanation in the MFD and
will be addressed in an updated MFD. We note that Eq. (96) does have an error; the 2 owT term in
the denominator should be replaced by owT (t/2). The correct expression is used in the AERMOD
code.
4.4. Minimum Meteorological Data Requirements
The AERMIC recommendations regarding minimum meteorological data for AERMOD are now
available for review on EPA's SCRAM web site. It is not AERMIC's intent nor recommendation
that only National Weather Service (NWS) data be used in, or sufficient for, all applications. The
NWS data may be acceptable for some sources in simple terrain settings. However, in complex
terrain, one may need a tall tower and sodar to provide the meteorological input if representative
-------
REPORT IV. FINAL AERMIC RESPONSE
data from NWS sources are not available. Consideration of meteorological data
representativeness on a case-by-case basis is an important aspect of the recommendations.
5. Response to Section 6: Comments on AERMOD Model Performance Document Dated 15
December 1998
The Panel notes that even if AERMOD's formulation were adjusted to improve its performance on
some of the databases, its consistently good performance on a large number of databases is
significant. The adjustment that improved the performance at the Lovett and Martins Creek plants
was the oz formulation for stable conditions, Eq. (96) with the correction noted above. This
formulation adopts the same Lagrangian time-scale model used in CTDMPLUS (see Venkatram
et al., 1984); it results in a smaller oz than in the earlier AERMOD version and hence higher
concentrations on elevated terrain.
5.1. Performance at Lovett
The Panel notes some confusion in the choice of concentration C or C/Q, where Q is the emission
rate, in forming the quantile - quantile (q-q) plots for Lovett and Martins Creek. At Lovett, the
concentration q-q plots were examined for the 1-, 3-, and 24-hr average concentrations, whereas
C/Q plots were presented for the 1-hr averages segregated by stability, i.e., convective and stable.
For averaging times greater than 1 hr, we adopted the C q-q plots because Q can vary over the 3-
or 24-hr period, and there is an issue of the representative Q for normalizing C. For steady
operation of a single source, the 1-hr average q-q plots could be presented either as C or C/Q. The
choice of the C q-q plots was done primarily for consistency with the 3- and 24-hr averages and
because the data record contained some low emission hours with highly uncertain Q values as
discussed below.
The use of C/Q plots for the data separated by stability was a carryover from our earlier analysis
and was not intended to confuse. However, we can understand the potential for confusion because
of the underpredicted C/Q values at the upper end of the q-q plots. The plots (Figs. A-27, A-28) in
the most recent AERMOD evaluation document (December 1998) contain data with quite low and
uncertain Q values at the upper end. For the convective periods (Fig. A-27), the mean and
maximum SO_2 emission rates for the data record were 160 g/s and 360 g/s, respectively. The
highest observed C/Q value (5.87) was obtained for a Q = 2.2 g/s, which is only 1.4% of the mean
Q, and the observed C was only 12.9 |i g/m3. In contrast, the highest observed C during convective
conditions was 442 |i g/m3 with a Q = 129 g/s, and yields a C/Q = 3.43. Thus, one can see that the
C/Q ratios for the very low Q hours can give a misleading impression about performance, i.e., the
highest observed C/Q values are associated with relatively low concentrations. We believe that the
cases with extremely low Q's should be de-emphasized; hence, we chose to focus on the C q-q
plots for assessing performance. Similar findings occurred for the stable hours (Fig. A-28), where
the Q's for the highest two observed C/Q values were only 4 and 5.4 g/s.
In hindsight, it would have been better to have eliminated the C/Q q-q plots or to have restricted
the analyzed cases to some minimum Q as had been done in the earlier CTDMPLUS work, where
-------
REPORT IV. FINAL AERMIC RESPONSE
a minimum Q = 40 g/s was adopted. In summary, we believe that focusing attention on the C q-q
plots (Figs. A-24 to A-26) is the appropriate course forjudging the AERMOD performance.
5.2. Performance at Martins Creek
For the Martins Creek plant, there are three reasons for choosing the C q-q plots for assessing
model performance. The first two are the same as above for Lovett: 1) variation of Q over the 3-
or 24-hr period and selecting a representative Q, and 2) de-emphasizing any cases with low and
uncertain Q values. The third reason is that other sources in the Martins Creek area contribute to
the concentrations; thus, there is again the issue of a representative Q for normalizing the
concentrations. In addition, it should be noted that the modeling of the concentrations from the
other sources is less than ideal because the Martins Creek meteorological measurements were
used for all sources. The other sources—Hoffman-LaRoche, Warren County Resource Recovery
Facility, and Metropolitan Ed Portland Station—are approximately 9, 12, and 15 km from the
Martins Creek sodar site (see Fig. 10 in the model evaluation document). Probably the most
questionable extrapolated variable is wind direction, and hence there is the problem of getting the
plumes from the different sources properly "aimed." This is always a problem in air quality
modeling. However, it is particularly exacerbated in the situation of a winding river valley
surrounded by elevated terrain with non-colocated sources and only one meteorological site.
5.3. Overall Performance at a Number of Sites
The Panel questions the AERMOD performance for surface releases and the Indianapolis (urban)
power plant because there is a slight tendency for underprediction relative to observations and to
ISC3. They go on to state: "It is much easier to accept a model like ISC3 for regulatory
applications based on fewer and less extensive evaluations, with its clear tendency to overpredict."
However, the last statement is not true. ISC3 underpredicts the highest concentrations for the
Kincaid SF6 1-hr concentrations and the Kincaid S02 concentrations for 1-, 3-, and 24-hr averages
based on the robust highest concentration (RHC, Table 1). Underpredictions also are found in the
q-q plots for the same cases (Figs. A-8, and A-19 to A-21). We note that the ISC3
underpredictions for these cases are more significant than those of AERMOD for the Prairie Grass
and Indianapolis data.
It is probably fair to say that a single model will not perform uniformly well or the same at a large
number of sites due to a variety of factors: 1) random or stochastic variability in the observed
concentrations (Note that models, as used here, are intended to predict ensemble mean or average
concentrations over a large number of repetitions, whereas the observed concentrations are single
realizations obtained from such an ensemble, i.e., from a statistical or probability distribution of
concentration.), 2) uncertainty or errors in the meteorology and/or unrepresentative meteorological
inputs, 3) errors in the model physics (e.g., unaccounted-for site features), etc. For regulators and
regulatory applications, a key question is: Within what tolerance or range of the peak observed
concentrations, especially on the low side, are the model predictions acceptable? This is especially
an issue when considering a large number of data bases and comparisons with varying degrees of
performance. An example of the performance range can be found in Table 1 of the model
-------
REPORT IV. FINAL AERMIC RESPONSE
evaluation document. For 1- to 24-hr average concentrations, the lowest and highest ratios of
modeled/observed RHC's are 0.72 and 1.72 for AERMOD, and 0.45 and 9.11 for ISC3; for
CTDMPLUS and the complex terrain sites only, the lowest and highest ratios are 0.77 and 5.56.
Thus, one can see that AERMOD has the narrowest range of ratios centered about the ideal value
of 1. This is not to say that the model is perfect, but it does have the smallest range of variation.
We believe that model evaluation and model performance over a wide range of data bases are
important topics and require further analysis and discussion.
6. References
AERMIC, 1998: Response to AERMOD peer review report. Prepared by the AMS/EPA
Regulatory Model Improvement Committee for the Office of Air Quality Planning and Standards,
U.S. Environmental Protection Agency, Research Triangle Park, NC.
Cimorelli, A.J., et al., 1998: AERMOD: Description of model formulation. Prepared by the
AMS/EPA Regulatory Model Improvement Committee for the Office of Air Quality Planning and
Standards, U.S. Environmental Protection Agency, Research Triangle Park, NC.
Hanna, S.R., and R.J. Paine, 1989: Hybrid plume dispersion model (HPDM) development and
evaluation. J. Appl. Meteor., 28, 206—224.
Hanna, S.R., and J.C. Chang, 1993: Hybrid plume dispersion model (HPDM) improvements and
testing at three field sites. Atmos. Environ., 27A,1491 — 1508.
Hanna, S.R., M. Garrison, and B. Turner, 1998: AERMOD peer review report. Prepared for
Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, Research
Triangle Park, NC.
Hanna, S.R., M. Garrison, and B. Turner, 1998: AERMOD peer review final report. Prepared for
Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, Research
Triangle Park, NC.
Paine, R.J., and S.B. Kendall, 1993: Comparison of observed profiles of winds, temperature, and
turbulence with theoretical results. Proceedings of an International Specialty Conference on the
Role of Meteorology in Managing the Environment in the 90s, 395—413.
Paine, R.J., et al., 1998: Model evaluation results for AERMOD, Prepared by the AMS/EPA
Regulatory Model Improvement Committee for the Office of Air Quality Planning and Standards,
U.S. Environmental Protection Agency, Research Triangle Park, NC.
Perry, S.G., 1992: CTDMPLUS: A dispersion model for sources in complex topography. Part I:
Technical formulations. J. Appl. Meteor., 31, 633—645.
-------
REPORT IV. FINAL AERMIC RESPONSE
Stull, R.B., 1988: An Introduction to Boundary Layer Meteorology. Kluwer Academic Publishers,
666 pp.
Venkatram, A., D. Strimaitis, and D. Dicristofaro, 1984: A semiempirical model to estimate
vertical dispersion of elevated releases in the stable boundary layer. Atmos. Environ., 18,
923-928.
Weil, J.C., L.A. Corio, and R.P. Brower, 1986: Dispersion of buoyant plumes in the convective
boundary layer. Preprints 5th Joint Conference on Applications of Air Pollution Meteorology,
Amer. Meteor. Soc., Boston, 335—338.
Weil, J.C., L.A. Corio, and R.P. Brower, 1997: A PDF dispersion model for buoyant plumes in
the convective boundary layer. J. Appl. Meteor., 36, 982—1003.
------- |