W**"*"*" Ei'titon
Evaluation of Prognostic Meteorological Data in
AERMOD Applications
-------
-------
EPA-454/R-15-004
July 2015
Evaluation of Prognostic Meteorological Data in AERMOD Applications
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards
Air Quality Assessment Division
Air Quality Modeling Group
Research Triangle Park, North Carolina
-------
Preface
This document provides an evaluation of prognostic meteorological data in AERMOD. Included
in this document are descriptions of the inputs, evaluation of the meteorological data, and
evaluation of AERMOD results using observed meteorological data and prognostic
meteorological data. EPA will respond to specific requests for subsets of the data or for specific
additional inputs and outputs of the modeling process, depending on the availability of the data.
Requests for electronic copies of the air quality modeling data used for this rule should be sent to
James Thurman (Thurman.james@epa.gov).
11
-------
Acknowledgements
This report was developed as part of the 2015 proposal of The Guideline on Air Quality Models,
Appendix W with input from the meteorological data workgroup comprised of staff from EPA's
Office of Air Quality Planning and Standards and Regions 5, 7, and 8. WRF and MMIF
processing for the Gibson, IN evaluation were processed by Computer Science Corporation.
WRF and MMIF processing for Martins Creek, PA and Herculaneum, MO were performed by
Andy Hawkins of EPA's Region 7. Evaluations discussed in Appendix B were performed by
Rebecca Matichuk of EPA's Region 8. The workgroup acknowledges Kali Frost of the Indiana
Department of Environmental Management for providing AERMOD inputs for Gibson, IN.
in
-------
Contents
Preface ii
Acknowledgements iii
Figures vi
Tables x
1. Introduction 12
2. Methodology 13
2.1 Study areas 13
2.1.1 Gibson 14
2.1.2 Martins Creek 18
2.1.3 Herculaneum 21
2.2 Meteorological data evaluation 25
2.3 Model evaluation methodology 26
3. Results 28
3.1 Gibson 28
3.1.1 Meteorological data comparisons 28
3.1.2 AERMOD results 36
3.2 Martins Creek 44
3.2.1 Meteorological data comparisons 44
3.2.2 AERMOD results 51
3.3 Herculaneum 59
3.3.1 Meteorological data comparisons 59
3.3.2 AERMOD results 69
4. Summary and Conclusions 70
5. References 72
Appendix A. Meteorological data comparisons A-l
A.I Gibson A-l
A. 2 Martins Creek A-17
A.3 Herculaneum A-33
B.I Introduction B-l
B.2 Methodology B-l
IV
-------
B.3 Results/Summary B-5
B.3.1 Temperature B-5
B.3.2 Relative humidity B-6
B.3.3 Wind speed B-7
B.3.4 Wind displacement B-8
B.3.5 Mechanical mixing heights B-9
B.3.6 Convective mixing heights B-10
B.3.7 Surface friction velocity B-ll
B.3.8 Convective velocity scale B-12
-------
Figures
Figure 1. Locations of MMIF evaluation study areas 14
Figure 2. Location of Gibson facility, 862 monitors, and meteorological tower for Gibson 15
Figure 3. Locations of Gibson meteorological tower (green dot), Evansville (EVV) NWS station
(airplane) and 12 km WRF grid cells containing those stations. Panel a is a statewide view and
panel b is a closer view of southwest Indiana 16
Figure 4. Location of Martins Creek study area with meteorological sites, monitors, and
emission sources 18
Figure 5. Detailed view of Martins Creek meteorological sites, monitors, and emissions sources
with detailed terrain 19
Figure 6. Location of the Doe Run lead facility, NWS station (CPS), site-specific tower
(Herculaneum), three MMIF output locations (MMIF 4 KM, MMIF 12 KM, and MMIF 36 KM),
and lead monitors 22
Figure 7. As for Figure 6, with more detail around the facility 23
Figure 8. Detailed view of MMIF grid cell locations, Herculaneum site specific tower, monitors,
and Doe Run 24
Figure 9. Gibson 2010 wind roses for a) GIB OBS, b) GIB MMIF, c) EVV OBS, and d) EVV
MMIF 29
Figure 10. Wind displacement (km) among the Gibson meteorological scenarios 30
Figure 11. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a)
January, b) February, c) March, and d) April 31
Figure 12. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a) May,
b) June, c) July, and d) August 32
Figure 13. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a)
September, b) October, c) November, and d) December 33
Figure 14. Gibson hourly QQ plots. Concentrations are in |j,g/m3 37
Figure 15. Gibson 3-hour screening results 38
Figure 16. Gibson 24-hour screening results 39
Figure 17. Gibson fractional biases for a) 1-hour, b) 3-hour, c) 24-hour, and d) CPM based on
fractional biases 40
Figure 18. Gibson composite performance metric values with 5th and 95th percentiles of the
CPM values from the bootstrap results 41
Figure 19. Gibson MCM differences with a) 90th percentile and b) 95th confidence intervals. .. 43
Figure 20. May 1992 - May 1993 Martins Creek wind roses for a) Martins Creek, b) ABE, c)
MMIF 1 km, and d) MMIF 4 km 44
Figure 21. Wind displacement (km) among the Martins Creek meteorological scenarios 45
Figure 22. Martins Creek study monthly surface roughness lengths (m) by 10 degree sectors for
a) January, b) February, c) March, and d) April 46
Figure 23. Martins Creek study monthly surface roughness lengths (m) by 10 degree sectors for
a) May, b) June, c) July, and d) August 47
VI
-------
Figure 24. Martins Creek study monthly surface roughness lengths (m) by 10 degree sectors for
a) September, b) October, c) November, and d) December 48
Figure 25. Martins Creek QQ plots for a) 1-hour, b) 3-hour, c) 24-hour, and d) annual averages.
Concentrations are in |j,g/m3 52
Figure 26. Martins Creek 3-hour screening results 53
Figure 27. Martins Creek 24-hour screening results 54
Figure 28. Martins Creek fractional biases for a) 1-hour, b) 3-hour, c) 24-hour, and d) CPM
based on fractional biases 55
Figure 29. Martins Creek composite performance metric values with 5th and 95th percentiles of
the CPM values from the bootstrap results 57
Figure 30. Martins Creek MCM differences with a) 90th percentile and b) 95th confidence
intervals 58
Figure 31. 2009 wind roses for a) Herculaneum and b) CPS 59
Figure 32. 2009 wind roses for a) MMIF 4 km, b) MMIF 12 km, and c) MMIF 36 km 60
Figure 33. Wind displacement (km) among the Herculaneum meteorological scenarios 61
Figure 34. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors for
a) January, b) February, c) March, and d) April 62
Figure 35. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors for
a) May, b) June, c) July, and d) August 63
Figure 36. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors for
a) September, b) October, c) November, and d) December 64
Figure 37. Herculaneum 24-hour QQ plots. Concentrations are in |j,g/m3 69
Figure 38. Herculaneum 24-hour screening results 70
Figure A-l. Gibson wind speed (m/s): a) annual distributions and b) bias distributions A-3
Figure A-2. Gibson ambient temperature (K): a) annual distributions and b) bias distributions. A-
4
Figure A-3. Gibson station pressure (mb): a) annual distributions and b) bias distributions.... A-5
Figure A-4. Gibson relative humidity (percent): a) annual distributions and b) bias distributions.
A-6
Figure A-5. Gibson daytime albedo (fraction): a) annual distributions and b) bias distributions.
A-7
Figure A-6. Gibson Bowen ratio: a) annual distributions and b) bias distributions A-8
Figure A-7. Gibson heat flux (W/m2): a) annual distributions and b) bias distributions A-9
Figure A-8. Gibson surface friction velocity, u* (m/s): a) annual distributions and b) bias
distributions A-10
Figure A-9. Gibson convective velocity scale, w* (m/s): a) annual distributions and b) bias
distributions A-11
Figure A-10. Gibson Monin-Obukhov length (m): a) annual distributions and b) bias
distributions A-12
Figure A-l 1. Gibson convective mixing height (m): a) annual distributions and b) bias
distributions A-13
Figure A-12. Gibson mechanical mixing height (m): a) annual distributions and b) bias
distributions A-14
vii
-------
Figure A-13. Gibson potential temperature gradient (K/m) above Zic: a) annual distributions and
b)bias distributions A-15
Figure A-14. Gibson cloud cover (tenths* 10): a) annual distributions and b) bias distributions. A-
16
Figure A-15. Martins Creek wind speed (m/s): a) annual distributions and b) bias distributions.
A-19
Figure A-16. Martins Creek ambient temperature (K): a) annual distributions and b) bias
distributions A-20
Figure A-17. Martins Creek station pressure (mb): a) annual distributions and b) bias
distributions A-21
Figure A-18. Martins Creek relative humidity (percent): a) annual distributions and b) bias
distributions A-22
Figure A-19. Martins Creek daytime albedo (fraction): a) annual distributions and b) bias
distributions A-23
Figure A-20. Martins Creek Bowen ratio: a) annual distributions and b) bias distributions... A-24
Figure A-21. Martins Creek heat flux (W/m2): a) annual distributions and b) bias distributions.
A-25
Figure A-22. Martins Creek surface friction velocity, u* (m/s): a) annual distributions and b)
bias distributions A-26
Figure A-23. Martins Creek convective velocity scale, w* (m/s): a) annual distributions and b)
bias distributions A-27
Figure A-24. Martins Creek Monin-Obukhov length (m): a) annual distributions and b) bias
distributions A-28
Figure A-25. Martins Creek convective mixing height (m): a) annual distributions and b) bias
distributions A-29
Figure A-26. Martins Creek mechanical mixing height (m): a) annual distributions and b) bias
distributions A-30
Figure A-27. Martins Creek potential temperature gradient (K/m) above Zic: a) annual
distributions and b) bias distributions A-31
Figure A-28. Martins Creek cloud cover (tenths* 10): a) annual distributions and b) bias
distributions A-32
Figure 29. Herculaneum wind speed (m/s): a) annual distributions and b) bias distributions. A-35
Figure A-30. Herculaneum ambient temperature (K): a) annual distributions and b) bias
distributions A-36
Figure A-31. Herculaneum station pressure (mb): a) annual distributions and b) bias
distributions A-37
Figure A-32. Herculaneum relative humidity (percent): a) annual distributions and b) bias
distributions A-38
Figure A-33. Herculaneum daytime albedo (fraction): a) annual distributions and b) bias
distributions A-39
Figure A-34. Herculaneum Bowen ratio: a) annual distributions and b) bias distributions.... A-40
Figure A-35. Herculaneum heat flux (W/m2): a) annual distributions and b) bias distributions. A-
41
Vlll
-------
Figure A-36. Herculaneum surface friction velocity, u* (m/s): a) annual distributions and b) bias
distributions A-42
Figure A-37. Herculaneum convective velocity scale, w* (m/s): a) annual distributions and b)
bias distributions A-43
Figure A-38. Herculaneum Monin-Obukhov length (m): a) annual distributions and b) bias
distributions A-44
Figure A-39. Herculaneum convective mixing height (m): a) annual distributions and b) bias
distributions A-45
Figure A-40. Herculaneum mechanical mixing height (m): a) annual distributions and b) bias
distributions A-46
Figure A-41. Herculaneum potential temperature gradient (K/m) above Zic: a) annual
distributions and b) bias distributions A-47
Figure A-42. Herculaneum cloud cover (tenths* 10): a) annual distributions and b) bias
distributions A-48
Figure B-l. Map of sites analyzed for study B-2
IX
-------
Tables
Table 1. Gibson emissions 15
Table 2. Martins Creek study facility emissions (tons/year) and stack parameters 20
Table 3. Herculaneum point source emission rates and stack parameters 25
Table 4. Mean bias, fractional bias, root mean square error, and R2 for primary meteorological
variables 35
Table 5. Mean bias, fractional bias, root mean square error, and R2 for calculated meteorological
variables 35
Table 6. Mean bias, fractional bias, root mean square error, correlation, and index of agreement
for calculated meteorological variables 36
Table 7. 1-hour, 3-hour, 24-hour absolute fractional biases and composite performance measures
for Gibson meteorological scenarios 41
Table 8. Model comparison measures (MCM) for the four Gibson meteorological scenarios. .. 42
Table 9. Mean bias, fractional bias, root mean square error, and R2 for primary meteorological
variables 50
Table 10. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables 50
Table 11. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables 51
Table 12. 1-hour, 3-hour, 24-hour absolute fractional biases and composite performance
measures for Martins Creek meteorological scenarios 56
Table 13. Model comparison measures (MCM) for the four Gibson meteorological scenarios. 57
Table 14. Mean bias, fractional bias, root mean square error, and R2 for primary meteorological
variables 66
Table 15. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables 67
Table 16. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables 68
Table B-l. Meteorological sites analyzed for study B-2
Table B-2. Description of NWS/AERMET model cases B-3
Table B-3. Description of WRF/MMIF model cases B-4
Table B-4. Monthly averaged temperature (K) across all modeled cases at each site B-6
Table B-5. Monthly averaged relative humidity across all modeled cases at each site B-7
Table B-6. Monthly averaged wind speed (m/s) across all modeled cases at each site B-8
Table B-7. Monthly averaged wind displacement (km) across all modeled cases at each site. . B-9
Table B-8. Monthly averaged mechanical mixing heights (m) across all modeled cases at each
site B-10
Table B-9. Monthly averaged mechanical mixing heights (m) across all modeled cases at each
site B-ll
-------
Table B-10. Monthly averaged surface friction velocity (m/s) across all modeled cases at each
site B-12
Table B-l 1. Monthly averaged convective velocity scale (m/s) across all modeled cases at each
site B-13
XI
-------
1. Introduction
An important part of dispersion modeling applications is the selection of meteorological data
input into the model. The meteorological data input into models such as AERMOD should be
adequately representative of conditions of the modeling, as discussed in Section 8.4 of the
proposed version of EPA's Guideline on Air Quality Models (U. S. EPA 2015). Specifically,
Section 8.4.b states:
"The meteorological data used as input to a dispersion model should be selected on the
basis of spatial and climatological (temporal) representativeness as well as the ability of
the individual parameters selected to characterize the transport and dispersion conditions
in the area of concern. The representativeness of the measured data is dependent on
numerous factors including but not limited to: (1) The proximity of the meteorological
monitoring site to the area under consideration; (2) the complexity of the terrain; (3) the
exposure of the meteorological monitoring site; and (4) the period of time during which
data are collected. The spatial representativeness of the data can be adversely affected by
large distances between the source and receptors of interest and the complex topographic
characteristics of the area. Temporal representativeness is a function of the year-to-year
variations in weather conditions. Where appropriate, data representativeness should be
viewed in terms of the appropriateness of the data for constructing realistic boundary
layer profiles ..."
Meteorological data often comes from National Weather Service (NWS) or site-specific
meteorological monitoring programs. In recent years, interest has grown in the use of prognostic
meteorological data, such as the Weather Research and Forecasting (WRF) model to create
inputs for dispersion modeling with AERMOD. This is especially true in locations where it can
be difficult to find an adequately representative NWS station or cost-prohibitive or infeasible to
set up a site-specific meteorological monitoring tower. As part of the 2015 proposed update to
Appendix W, EPA has proposed the use of prognostic meteorological data for use in AERMOD
for areas where it is cost-prohibitive or not feasible to collect site-specific data and there is no
representative NWS or comparable station nearby. EPA has developed the Mesoscale Model
Interface Program, or MMIF, for processing prognostic meteorological data for AERMOD
(Environ, 2014). For more information on the use of prognostic data in AERMOD see Section
8.4.5 of the proposed version of Appendix W.
The purpose of the prognostic evaluation process was to determine if prognostic meteorological
data are suitable for input into AERMOD. The goal of the evaluation was to determine if such
prognostic data was comparable to NWS or site-specific meteorological data and, how
AERMOD performance to observed concentrations compares based on prognostic
meteorological data with simulations based on NWS data or site-specific data.
This report details the evaluation process used to determine the feasibility of prognostic
meteorological data for use in AERMOD. Section 2 discusses the methodology of the three case
studies: 1) Gibson, IN; 2) Martin's Creek, PA; and 3) Herculaneum, MO. This includes
12
-------
meteorological data processing (i.e., for both observed and prognostic modeling) and
development of AERMOD inputs. Section 3 is a review of the evaluation results, including
meteorological data evaluation and AERMOD concentration results for the three locations, while
Section 4 is a summary of conclusions and Section 5 provides the references. Appendix A gives
more results on the meteorological evaluations of the three case studies and Appendix B is an
evaluation of observed and modeled meteorological data for six sites in EPA's Region 8.
2. Methodology
2.1 Study areas
Three case study areas were considered for evaluation (Figure 1). Gibson (Frost, 2014) and
Martins Creek were both 862 releases while Herculaneum is a lead release. The Gibson
modeling presented in this document are based on data presented in Frost (2014). Martins Creek
is one of the databases used to evaluate AERMOD during its development (U.S. EPA, 2003;
Perry et al., 2005). All three case study area evaluations include site specific meteorological
data, a representative NWS station, and multiple prognostic model grid cells. More details about
each evaluation are below.
13
-------
Figure 1. Locations of MMIF evaluation study areas.
2.7.7 Gibson
As shown in Figure 1, the Gibson facility is located in southwest Indiana. The facility is
comprised of five units with a 3145 MW capacity (Frost, 2014). Table 1 lists the five units with
stack parameters, average annual hourly 862 emissions and maximum hourly 862 emissions for
2010 as described in Frost (2014).
14
-------
Table 1. Gibson emissions
Unit
1 and 2
3
4
5
Stack
height (m)
189.0
189.0
152.4
152.4
Stack
diameter
(m)
23.5
25.0
25.0
25.0
Average
annual
hourly
emissions
(g/s)
116.0
90.9
111.3
290.0
Maximum
hourly
emissions
(g/s)
462.2
794.1
460.3
819.8
Exit
temperature
(K)
327.0
327.0
322.0
328.0
Average
stack exit
velocity
(m/s)
15.6
18.5
17.4
12.5
Figure 2 shows the AERMOD study area with the location of stacks, four monitors, and the site-
specific meteorological tower.
0273X55 165 22
Figure 2. Location of Gibson facility, SOi monitors, and meteorological tower for Gibson.
The site-specific tower included multi-level measurements of winds, temperature, pressure, solar
radiation, and standard deviation of the horizontal wind direction (oฎ) at 10, 25, and 60 m. More
15
-------
details can be found in Frost (2014). Data from the National Weather Service (NWS) station at
Evansville, IN (EVV) were used for substitution of missing hours. Upper air data from Lincoln,
IL (ILN) was used for the morning soundings. Figure 3 shows the relationship between the site-
specific tower and EVV.
MICHIGAN
OHIO
J
ฎ
KENTUCKY
MISSOURI
Q
KENTUCKY
0 j(E,JITUC,i
-------
For the purposes of this study, 10m pressure and insolation were input into AERMET as well as
60 m winds and temperature. The a@ data was not used due to suspected problems with the data
(Frost, 2014b).
Four AERMET (version 14134) and AERMOD (version 14134) runs were performed using four
sources of meteorological data (meteorological and model run names in parentheses):
Site-specific data supplemented with EVV surface data (GIB OBS)
EVV data only (EVV OB S)
12 km WRF processed through MMIF and AERMET for the WRF grid cell containing
the facility (GIB MMIF)
12 km WRF processed through MMIF and AERMET for the WRF grid cell containing
EVV (EVV MMIF)
All four scenarios utilized a 0.5 m/s wind speed threshold. In addition to showing the locations
of the observed meteorological data sources, Figure 3 shows the locations of the two WRF grid
cells processed through MMIF and AERMET. While MMIF can process data for AERMOD
input directly, MMIF was run to process MMIF for AERMET in accordance with Section 8.4.2
of Appendix W and MMIF guidance (U.S. EPA, 2015b). Evaluation of the WRF data can be
found in U.S. EPA (2014). Note evaluations are for 2011 but are applicable to 2010 as well.
MMIF was processed to output 27 layers using the MID interpolation option. The output heights
corresponded to the vertical grid structure (25, 50, 75, 100, 125, 150, 175, 200, 250, 300, 350,
400, 450, 500, 600, 700, 800, 900, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, and 5000
m). Upper air data in the Forecast Systems Laboratory (FSL) format was output for every hour.
For GIB OBS and EVV OBS surface characteristics, AERSURFACE (U.S. EPA, 2013) was run
to determine albedo, Bowen ratio, and surface roughness length for 12 months and 1230ฐ surface
roughness sectors. Monthly Bowen ratios were adjusted using soil moisture and precipitation
data from the National Centers for Environmental Information (NCEI)1 for EVV. Adjustments
to albedo, Bowen ratio, and surface roughness length for winter months were determined using
snowfall data for EVV from NCEI and ratios of days with at least one inch of snow to days with
no snow cover were also used to adjust values (Frost, 2014).For GIB MMIF and EVV MMIF,
surface characteristics from the MMIF processor were used in accordance with guidance in
Section 8.4.2(b) of Appendix W (U.S, EPA, 2015a) and Section 3.3 of MMIF guidance (U.S.
EPA, 2015b). The grid cells for Gibson and Evansville were processed to output both AERMET
ready inputs and AERMOD ready inputs. Due to a coding error in the version of MMIF used for
this evaluation, when processing multiple grid cells for AERMET input, the surface
characteristics were output incorrectly for the AERMET ready files for Gibson and Evansville.
The surface characteristics were determined using the values from the AERMOD ready files.
1 Formerly the National Climatic Data Center (NCDC)
17
-------
Surface characteristics for 12 months and one 360ฐ sector were calculated. More information
about the meteorological data can be found in Section 3.1.1 of this document.
2.1.2 Martins Creek
As previously stated, Martins Creek was one of the evaluation databases used in AERMOD
development (U.S, EPA, 2003; Perry et al., 2005). In those evaluations, AERMOD performed
better than other models (ISCST3, CTDMPLUS, ISCST3, and RTDM). The Martins Creek
Steam Electric Station is located along the Delaware River on the Pennsylvania/New Jersey
border, approximately 30 km northeast of Allentown, PA and 95 km north of Philadelphia, PA
(U.S. EPA, 2003). See Figure 4 for the general location of Martins Creek and meteorological
sites.
0 1.5 3
I Kilo meters
Portland
Hoffman-LaRod
Warren
PENNSYLVANIA
Martins Creek MMIF 1 KM Martins Creek (emissions)
(meteorology)
Monitors
NEW JERSEY
Figure 4. Location of Martins Creek study area with meteorological sites, monitors, and
emission sources.
SO2 measurements were taken at seven monitors east of the facility (Figure 5) on Scott's
Mountain. Site-specific meteorology was recorded from May 1, 1992 to May 19, 1993.
18
-------
.egend
m High 648 8
^"- Low -8 83573
4 5
Portland
Kilometers
'
Martins Creek MMIF 1 KM
(meteorology)
Martins Creek (emissions)
11
12
10
Figure 5. Detailed view of Martins Creek meteorological sites, monitors, and emissions
sources with detailed terrain.
Data included temperature, wind speed, wind direction, and aฉ at 10m height west of the facility
(Figure 4). In addition, hourly multi-level wind measurements were taken by a sodar located
southwest of the facility. Upper air data was from Albany, NY with missing soundings
substituted from Sterling, VA.
In addition to emissions from Martins Creek, three other facilities were modeled: 1) Portland,
Hoffman-LaRoche, and Warren (Figure 5). Hourly emissions and hourly stack parameters
(temperature and exit velocity) at all four facilities were modeled in AERMOD.
Table 2 lists the units at each facility with stack parameters and annual emissions.
19
-------
Table 2. Martins Creek study facility emissions (tons/year) and stack parameters.
Facility
Martins
Creek
Portland
Hoffman-
LaRoche
Warren
Stack
MC12
MC3
MC4
EDI
ED2
HL2
WC1
WC2
Annual
SOi (tpy)
20272
2923
3395
5459
12939
837
0.2
0.2
Stack
height
(m)
182.9
182.9
182.9
121.9
121.9
59.4
76.2
76.2
Stack
diameter
(m)
5.3
6.9
6.9
3.1
3.6
2.7
1.9
1.9
Avg. exit
temperature
(K)1
400.8
403.9
403.0
395.0
400.0
451.5
404.6
410.3
Avg. exit
velocity
(m/s)1
17.1
17.5
18.7
33.3
26.4
6.8
3.3
3.4
1. Average temperature and exit velocity based on hours with non-zero emissions.
Four AERMET (version 14134) and AERMOD (version 14134) simulations were performed:
Site-specific data supplemented with Allentown/Bethlehem (ABE) data (Martins Creek)
ABE only data (ABE)
1 km WRF simulation of grid cell containing Martins Creek (MMIF 1 km)
4 km WRF simulation of grid cell containing Martins Creek (MMIF 4 km)
Figure 5 shows the locations of the 1 km and 4 km WRF grid cells processed through MMIF and
AERMET.
For the Martins Creek site-specific data, the surface characteristics supplied with the evaluation
database on SCRAM were used. For the ABE surface characteristics, AERSURFACE (U.S.
EPA, 2013) was run to determine albedo, Bowen ratio, and surface roughness length for 12
months and three surface roughness sectors (110ฐ-230ฐ, 230ฐ-330ฐ, and 330ฐ-110ฐ). Winter was
assumed to be "no snow" and moisture conditions were assumed to be dry based on
climatological data for ABE.
For the 1 km and 4 km MMIF generated outputs, surface characteristics from the MMIF
processor were used in accordance with guidance in Section 8.4.2(b) of Appendix W (U.S, EPA,
2015a) and Section 3.3 of MMIF guidance (U.S. EPA, 2015b). Surface characteristics for 12
months and one 360ฐ sector were calculated. MMIF was run to output 10 layers (17.13, 51.46,
85.91, 120.48, 207.7, 348.56, 564.66, 936.27, 1566.38, and 2851.29 m) using the TOP
interpolations. More information about the meteorological data can be found in Section 3.2.1 of
this document.
20
-------
2.1.3 Herculaneum
The third evaluation site is focused on the Herculaneum, MO area, specifically the Doe Run lead
facility (Figures 6 through 8). This evaluation allowed for the use of two observed datasets, site-
specific and NWS station and three MMIF outputs at three different horizontal grid resolutions, 4
km, 12 km, and 36 km. Figures 6 through show the spatial relationship between the Doe Run
facility, two lead monitors, and the various meteorological data locations. Table 3 lists the major
modeled point sources with emission rates. The five scenarios are:
Site specific data supplemented with St. Louis (CPS) meteorological data (Herculaneum)
CPS data only (CPS)
36 km WRF simulation of grid cell containing Doe Run (MMTF 36 km)
12 km WRF simulation of grid cell containing Doe Run (MMTF 12 km)
4 km WRF simulation of grid cell containing Doe Run (MMTF 4 km)
The Herculaneum site-specific tower and CPS were processed through AERMET using version
13350 with ILX upper air data. The WRF grid cells were processed through MMIF to generate
AERMOD and AERMET ready input files. MMIF was run using the same layer structure and
options and FSL formatted output frequency as for GIBSON. All five scenarios were processed
in version 13350 of AERMOD.
21
-------
iVf. fjfjj
MMIF 36 KM
MMIF 4 KMA j Monitors
Legend MMIF 12 KM
High 353 509
Low 68 1493
/
Herculaneum
Doe Run
M7
024
12
16
Kilometers
Figure 6. Location of the Doe Run lead facility, NWS station (CPS), site-specific tower
(Herculaneum), three MMIF output locations (MMIF 4 KM, MMIF 12 KM, and MMIF 36
KM), and lead monitors.
22
-------
-^
Herculaneum Doe Run
Figure 7. As for Figure 6, with more detail around the facility.
23
-------
Legend
High . 353 509
Low 68 1493
MMIF 4 KM
Mott St. Main St.
A "
Herculaneum Doe Run
MMIF 12 KM
0 02750.55 1 1 1 65 22
^^^^m =^^^^B Kilometers
Figure 8. Detailed view of MMIF grid cell locations, Herculaneum site specific tower,
monitors, and Doe Run.
24
-------
Table 3. Herculaneum point source emission rates and stack parameters.
Stack
30001
40004
40005
50007
50008
50011
50012
50013
50014
50015
50016
50017
50018
60001
60002
60003
60004
60005
60006
60007
60008
Emissions
(g/s)
4.17E+00
8.58E-04
8.58E-04
4.31E-02
2.97E-01
1.65E-03
1.65E-03
1.65E-03
1.65E-03
1.65E-03
1.65E-03
1.65E-03
1.65E-03
1.13E-04
1.13E-04
5.93E-06
1.80E-03
1.17E-03
1.17E-03
1.17E-03
1.17E-03
Stack height
(m)
100.75
21.3
21.3
45.72
45.72
18.8
18.8
18.8
18.8
18.8
18.8
18.8
18.8
21.3
21.3
7.6
6.1
16.8
16.8
16.8
16.8
Stack
diameter (m)
346.67
391.5
391.5
285.56
276.11
989.3
989.3
989.3
989.3
989.3
989.3
989.3
989.3
699.8
699.8
297
327.6
297
297
297
297
Avg. exit
temperature
(K)
5.81
0.69
0.69
7.13
34.57
5.96
5.96
5.96
5.96
5.96
5.96
5.96
5.96
2.73
2.73
7.7
17.5
5
5
5
5
Avg. exit
velocity (m/s)
10.31
0.76
0.76
2.59
3.05
0.61
0.61
0.61
0.61
0.61
0.61
0.61
0.61
0.56
0.56
1.08
0.25
0.56
0.56
0.56
0.56
2.2 Meteorological data evaluation
Evaluation among the various meteorological datasets for each study encompassed comparing
the hourly distributions of each dataset, comparing the distributions of the hourly differences
among pairs of datasets, and calculations of several statistics, including mean bias, fractional
bias, root mean square error, correlation, and index of agreement. For the wind direction
difference statistics, a difference called displacement, which is the difference in the U and V
vectors of the modeled and observed winds and was used. This was used in the assessment of
the 2011 12km WRF simulations over the U.S. (US EPA, 2014). The displacement can be
calculated as:
D = abs((UM -UO
- Vo) x (1 km/ WOO m) x (3600 s//ir) x 1/ir)
(1)
25
-------
Where D is the displacement in km, UM and VM are the u and v components respectively of the
modeled wind vector and Uo and Vo are the u and v components of the observed wind vector.
2.3 Model evaluation methodology
AERMOD output among the different meteorological datasets was evaluated using the EPA
Protocol for determining the best performing model, or Cox-Tikvart method (U.S. EPA, 1992;
Cox and Tikvart, 1990). The protocol uses a two-step process for determining the better
performing model when comparing models. The first step is a screening test that fails to perform
at a minimal operational level. The second test applies to those models that pass the screening
test that uses bootstrapping to generate a probability distribution of feasible outcomes (U.S. EPA,
1992). This section will discuss the methodology using the evaluation cases as examples.
The first step is to perform a screening test based on fractional bias:
\OB-PR\
FB = 2
OB+PR
(1)
Where FB is the fractional bias, OB is the average of the highest 25 observed concentrations and
PR is the average of the highest 25 predicted averages. The fractional bias is also calculated for
the standard deviation where OB and PR refer to the standard deviation of the highest 25
observed and predicted concentrations respectively. This is done across all monitors and
modeled receptors, unpaired in time and space for the 3-hour and 24-hour averaging periods. The
fractional bias of the means is plotted against the fractional bias of the standard deviation. Biases
that exceed a factor-of-two under-prediction or over-prediction are considered grounds for
excluding a model for further evaluation (U.S. EPA, 1992).
Models that pass the screening test are subjected to a more comprehensive statistical comparison
that involves both an operational and scientific component. The operational component is to
measure the model's ability to estimate concentration statistics most directly used for regulatory
purposes and the scientific component evaluates the model's ability to perform accurately
throughout the range of meteorological conditions and the geographic area of concern (U.S.
EPA, 1992). The test statistic used for the comparison is the robust highest concentration
(RHC) statistic and is given by:
RHC = X(JV) + [X - X(JV)] X In f^i] (2)
Where X(N) is the Nth largest value, X is the average of N-l values, and N is the number of
values exceeding the threshold value, usually 26.
The operational component of the evaluation compares performance in terms of the largest
network-wide RHC test statistic. The RHC is calculated separately for each monitor within the
network for observations and modeled values. The highest observed RHC is then compared to
the highest modeled RHC using equation 1, where RHC now replaces the means of the top 25
values of observed or modeled concentrations. Absolute fractional bias (the absolute value of
fractional bias), AFB is calculated for 3 and 24-hour averages.
26
-------
The scientific component of the evaluation is also based on absolute fractional bias but the bias is
calculated using the RHC for each meteorological condition and monitor. The meteorological
conditions are a function of atmospheric stability and wind speed. For the purposes of these
studies, six unique conditions were defined based on two wind speed categories (below and
above 2.0 m/s) and three stability categories: unstable, neutral, and stable. 2 In this evaluation,
only 1-hour concentrations are used and the AFB is based on RHC values paired in space and
stability/wind speed combination.
A composite performance measure (CPM) is calculated from the 1-hour, 3-hour, and 24-hour
AFB's:
CPM = -x (AFBt/) + -x
j j
Where AFBy is the absolute fractional bias for monitor i and meteorological condition],
is the average absolute fractional bias across all monitors and meteorological conditions, AFBs is
the absolute fractional bias for the 3-hour average, and AFB24 is the absolute fractional bias for
the 24-hour average. Once CPM values have been calculated for each model, a model
comparison measure is calculated to compare the models:
MCMA:B = CPMA - CPMB (4)
Where CPMA is the CPM for model A and CPMe is the CPM for model B. When more than two
models are being compared simultaneously, the number of MCM values is equal to the total of
the number of unique combinations of two models. In the case of these evaluations, it is not the
number of models but the number of meteorological scenarios. For Gibson and Martins Creek,
there are four meteorological scenarios each, so there were six MCM comparisons for each
location. For Herculaneum, CPM values could not be calculated because there were only 24-
hour average concentrations available from the two monitors.
In order to determine if the difference between models was statistically significant, the standard
error was calculated. A bootstrapping technique was used to create 1000 sample years based on
methodology outlined in U.S. EPA (1992). The original data is divided into 3-day blocks.
Within each season, the 3-day blocks are sampled with replacement until a total season is
created. The process is repeated until a 1000 boot-strap years are created3. The standard error is
calculated as the standard deviation of the bootstrap generated outcomes for the MCM.
The magnitude and sign of the MCM are indicative of relative performance of each pair of
models. The smaller the CPM the better the overall performance of the model. This means that
2 In U.S. EPA (1992), the three stability categories are related to the Pasquill-Gifford categories, unstable being A,
B, and C, neutral being D, and stable being E and F. Since AERMOD does not use the stability categories, the
stability class was determined using Monin-Obukhov length and surface roughness using methodology from
AERMOD subroutine LTOPG.
3 The bootstrapping was completed using the SASe SURVEYSELECT procedure with resampling for 1000
replicates.
27
-------
for two models, A and B, a negative difference between the CPM for A and CPM for B implies
that model A is performing better (Model A has a smaller CPM) while a positive difference
indicates that Model B is performing better.
Since more than two scenarios are being evaluated in these studies, simultaneous confidence
intervals of 90 and 95 percent were calculated. These were calculated by finding the 90th and
95th percentiles of the distribution across all MCM values from the bootstrapping procedure for
all model comparisons. The confidence intervals were then found by:
ฑ CXSA:B (5)
Where C!X,A,B is the confidence interval for X percent (90 or 95th) for models A and B, MCMA,B
is as defined in Equation 4, ex is the X percentile of the MCM values from the bootstrap results
and SA,B is the standard deviation of the bootstrap MCM results for models A and B. Note that in
Equation 5, MCMA,B is the MCM value from the original data, not the bootstrap results.
For each pair of model comparisons, the significance of the model comparison measure
depended on whether the confidence interval overlapped zero. If the confidence interval
overlapped zero, then the two models were not performing at a level which was considered
statistically different. Otherwise, if they did not overlap zero, then there was a statistically
significant difference between the two models.
3. Results
This section provides results for each of the three case study areas. Meteorological data
comparisons among the different scenarios for each site are shown as well as the associated
AERMOD results for each meteorological scenario.
3.1 Gibson
3.1.1 Meteorological data comparisons
Figure 9 compares the 2010 wind roses for site specific tower (GIB OBS), 12 km WRF cell of
the site specific tower (GIB MMIF), Evansville NWS observations (EVV OBS) and the 12 km
WRF cell of EVV (EVV MMIF). The wind roses for GIB OBS and GIB MMIF have similarities
but GIB OBS exhibits a stronger component of winds from the southwest while GIB MMIF
exhibits a stronger components of winds from the northwest. For Evansville, the observed winds
(EVV OBS) exhibit dominant directions of southwest, northwest and northeast. The MMIF
wind rose (EVV MMIF) exhibits a strong component of winds from the northwest with a
secondary component from the southwest.
28
-------
GIBOBS
V" *.
EVV OBS
GIBMMIF
EVVMMIF
Figure 9. Gibson 2010 wind roses for a) GIB OBS, b) GIB MMIF, c) EVV OBS, and d)
EVV MMIF.
Wind displacement (Figure 10), distribution for GIB MMIF - EVV MMIF shows a tighter
distribution than the other three indicating wind directions, on an hourly basis, are in more
agreement between the two model grid cells. The other three bias distributions are similar, with
larger displacement values between the two observed datasets.
29
-------
60-
o>
|40-
o
jo
CL
(0
b
20-
0-
GIB GIB OBS EW OBS
Scenario
GIB OBS-EW OBS
Figure 10. Wind displacement (km) among the Gibson meteorological scenarios.
Figures 11 through 13 show monthly values of surface roughness, z0 (m) by 10ฐ sectors for the
four meteorological scenarios. For most of the winter, the EVV MMIF surface roughness is the
highest, over 0.3 m. The plots show that GIB OBS can vary significantly depending on the wind
direction and month. EVV OBS also vary depending on month and sector.
30
-------
o =
Figure 11. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a)
January, b) February, c) March, and d) April.
31
-------
Figure 12. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a)
May, b) June, c) July, and d) August.
32
-------
s* # tf ^
Direction
Figure 13. Gibson study monthly surface roughness lengths (m) by 10 degree sectors for a)
September, b) October, c) November, and d) December.
Tables 4 through 6 show statistics for several primary variables including wind speed,
temperature, pressure, and relative humidity. The tables also show statistics for heat flux,
surface friction velocity (u*), convective velocity scale (w*), Monin-Obukhov length (L),
convective mixing height (Zic), mechanical mixing height (Zim), potential temperature gradient
(d0/dz), and cloud cover. In each table, the GIB OBS - EVV OBS are highlighted as they can
be considered a "control" since in the absence of prognostic data, these would be the only
available data sources. Box and whisker plots of the variable distributions as well as bias
distributions can be found in Appendix A. Findings include:
Differences between MMIF wind speeds and observed wind speeds appear to be lower
than the differences between the two observed datasets. The GIB MMIF scenario under-
predicts when compared to the GIB OBS and GIB MMIF over-predicts when compared
to EVV MMIF.
MMIF scenarios tend to under-predict temperatures and pressure when compared to their
respective observed counterparts. The GIB MMIF scenario under-predicts (over-
33
-------
predicts) when compared to EVV MMIF for temperature (pressure), but GIB OBS over-
predicts when compared to EVV OBS for both variables.
MMIF scenarios over-predict relative humidity compared to their observed counterparts.
It should be noted that the GIBS-EVV OBS are zero because EW observed RH is used
for the Gibson site-specific tower in AERMET.
For heat flux, u*, and w*, the MMIF scenarios over-predict compared to their observed
counterparts. GIB MMIF under-predicts compared EVV MMIF as does the GIB OBS
when compared to EVV OBS.
For the potential temperature gradient, the MMIF scenarios under-predict when
compared to their observed counterparts and the GIB MMIF scenario under-predicts
compared against EVV MMIF and GIB OBS under-predicts compared to EVV OBS.
For mixing heights (both convective and mechanical), the MMIF scenarios over-predict
compared to the observed counterparts with low R2 and IOA when compared to the GIB
OBS - EVV OBS differences.
Monin-Obukhov length differences show relatively low agreement between the MMIF
and observed scenarios.
Cloud cover differences also show relatively low agreement but this may be due to the
calculation methodology in AERMET when cloud cover is missing.
Overall, while there are differences, the MMIF scenarios appear to show relatively good
agreement with the observed data. Differences between the MMIF scenarios and their respective
counterparts is usually in line with the differences between the site-specific data and NWS data
(i.e., GIB OBS - EVV OBS).
34
-------
Table 4. Mean bias, fractional bias, root mean square error, and R2 for primary
meteorological variables.
Variable
Wind speed
Ambient
temperature
Pressure
Relative
humidity
Scenario
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
Mean
bias
0.2395
-0.6808
0.4345
1.3562
-0.3321
-0.6568
-0.1067
0.2133
0.4412
-3.9806
-1.0903
3.3315
1.1127
6.5496
5.4416
0.0000
Fractional
bias
0.0130
-0.0309
0.0564
0.0943
-0.0003
-0.0006
-0.0001
0.0002
0.0001
-0.0010
-0.0003
0.0008
0.0045
0.0216
0.0172
0.0000
RMSE
0.6873
1.7568
1.1826
2.1125
0.8500
3.0398
1.9698
3.0920
0.7131
4.1719
1.5625
3.6102
6.2611
14.3825
14.1551
0.0000
R2
0.8899
0.5385
0.6568
0.5348
0.9957
0.9376
0.9736
0.9351
0.9920
0.9658
0.9728
0.9557
0.9227
0.6555
0.6511
1.0000
Table 5. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables.
Variable
Heat flux
u*
w*
dฉ/dz
Scenario
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
Mean
bias
-8.1307
16.2727
11.4507
-13.8544
-0.0315
0.1660
0.1858
-0.0159
-0.1095
0.3639
0.1874
-0.2887
0.0001
-0.0016
-0.0018
-0.0001
Fractional
bias
-0.0409
0.1259
0.1025
-0.0673
-0.0209
0.1440
0.1643
0.0013
-0.0208
0.1077
0.0389
-0.0941
0.0018
-0.0375
-0.0434
-0.0040
RMSE
29.7447
54.6334
53.7118
36.5569
0.0757
0.2150
0.2271
0.0839
0.2595
0.5728
0.5022
0.4124
0.0013
0.0043
0.0045
0.0027
R2
0.9256
0.6997
0.7413
0.8467
0.9050
0.6087
0.6816
0.6753
0.8677
0.4937
0.5190
0.7618
0.7727
0.2171
0.2009
0.6755
35
-------
Table 6. Mean bias, fractional bias, root mean square error, correlation, and index of
agreement for calculated meteorological variables.
Variable
Zic
Zim
L
Cloud
cover
Scenario
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
GIB MMIF-EW MMIF
GIB MMIF-GIB OBS
EW MMIF-EW OBS
GIBOBS-EWOBS
Mean
bias
-48.3405
250.3559
66.4046
-236.2854
-52.0755
249.6961
274.1437
-35.5563
24.9063
572.5552
336.0061
-221.9609
0.1830
1.2902
2.2222
1.1128
Fractional
bias
-0.0122
0.1324
0.0431
-0.1140
-0.0172
0.1424
0.1652
-0.0001
-0.0197
0.1899
0.2854
0.0651
0.0152
0.0981
0.1789
0.1092
RMSE
244.6457
523.6197
512.7182
349.0821
159.9024
410.5375
439.7079
152.9793
1592.5484
2509.5879
2320.8618
1389.7583
2.2867
5.7003
5.4528
3.6298
R2
0.8405
0.4137
0.4251
0.8315
0.9166
0.5749
0.6514
0.6606
0.5897
0.0475
0.0404
0.0325
0.7180
0.0307
0.1123
0.4976
3.1.2 AERMOD results
Figure 14 shows the hourly QQ-plot among all four monitors for the four meteorological
scenarios. The GIB OBS concentrations tend to be the highest predicting scenario. The two
MMIF driven scenarios and the EW OBS scenario are visually are in close agreement with each
other and tend to under-predict at higher concentrations.
36
-------
900-
|eoo
"D
O
300-
200
Observed
400
Figure 14. Gibson hourly QQ plots. Concentrations are in |o,g/m3.
The modeled scenarios were evaluated using the Cox-Tikvart methodology as discussed in
Section 2.2. Figures 29 and 30 show the initial 3-hr and 24-hour screening results. For the 3-
hour results, all four scenarios are outside the ฑ0.67 box that identifies good performance. The
3-hour results correspond with the QQ-plots in Figure 14 in that GIB OBS over-predicts (bias
less than 0) and the other scenarios tend to under-predict (bias greater than 0). However, all four
scenarios are in close proximity to one another around a bias of zero. For the 24-hour average,
all four scenarios fall within the ฑ0.67 box. For 24-hour averages, all scenarios except for the
EVV OBS scenario have slight over-prediction. All four scenarios have a bias close to zero.
37
-------
2-
_g
.Is
">
Q
2 0
co
o
co
S
CO
-2-
-2
-1
0
of
Figure 15. Gibson 3-hour screening results.
38
-------
2-
_g
.Is
">
Q
2 0
co
o
co
S
CO
-2-
-2
-1
0
of
Figure 16. Gibson 24-hour screening results.
Figure 17 compares the 1-hour, 3-hour, and 24-hour fractional biases and a composite of the
fraction biases (Equation 3 using fractional biases instead of absolute fractional biases). Shown
are the fractional biases from the original data along with the 5th and 95th percentiles of the
fractional biases from the bootstrap results. Figure 17.d shows the modified CPM from the
original data with the 5th and 95th percentiles of the modified CPM from the bootstrap results. In
the plots, positive (negative) fractional biases indicate model under-prediction (over-prediction).
Based on the distributions, for 1-hour and 3-hour, all four scenarios tend to under-predict
compared to observations. For the 24-hour periods, the models tend to show more over-
prediction. The composite shows mostly under-prediction, with the four scenarios showing
comparable distributions.
39
-------
Figure 17. Gibson fractional biases for a) 1-hour, b) 3-hour, c) 24-hour, and d) CPM based
on fractional biases.
Table 7 shows the AFB values for the 1-hour, 3-hour, and 24-hour periods as well as the CPM
statistics for each meteorological scenario. Figure 18 shows the CPM values with the 5th and
95th percentiles of the CPM's from the bootstrap results. Based on the results shown in Table 7
and Figure 18 the EVV MMTF results were somewhat different compared to the other three
scenarios which had similar CPM values.
40
-------
Table 7.1-hour, 3-hour, 24-hour absolute fractional biases and composite performance
measures for Gibson meteorological scenarios.
Scenario
EVVMMIF
EVV OBS
GIBMMIF
GIB OBS
AFB
1-hour
0.77
0.75
0.77
0.90
3-hour
0.26
0.14
0.17
0.09
24-hour
0.19
0.04
0.09
0.05
CPM
0.41
0.31
0.34
0.35
0.4-
Q_
o
0.2-
0.0
EW MMIF
EWOBS
GIB MMIF
GIB OBS
Figure 18. Gibson composite performance metric values with 5th and 95th percentiles of
the CPM values from the bootstrap results.
41
-------
Table 8 lists the MCM values for the six model comparisons and also lists the best performing
model for each pair. As discussed in Section 2.2, positive (negative) values of an MCM indicate
that the second scenario of the difference (scenario to the right of the "-" sign) is performing
better (worse) than the first scenario (scenario to the left of the "-" sign). Figure 19 show the
MCM differences among the four scenarios at the 90th and 95th confidence intervals respectively.
. The results indicate that the EVV MMIF meteorology performs slightly worse when compared
to the other three scenarios. The results also indicate that the GIB MMIF scenario performs
worse against EW OBS but is almost equal to the GIB OBS scenario and that the GIB OBS
scenario performs worse than the EVV OBS scenario. While there is differences in performance,
the confidence intervals for each difference overlap zero, indicating that differences in
performance are not statistically significant. Of the two MMIF scenarios, the GIB MMIF
appears to be the better scenario, which is not surprising given that the cell contains the facility.
Table 8. Model comparison measures (MCM) for the four Gibson meteorological
scenarios.
MCM Scenario
EVV MMIF - EVV OBS
EVV MMIF - GIB OBS
GIB MMIF - EVV MMIF
GIB MMIF -EVV OBS
GIB MMIF - GIB OBS
GIB OBS -EVV OBS
MCM
0.1
0.06
-0.06
0.03
-0.004
0.04
Best performing scenario
EVV OBS
GIB OBS
GIB MMIF
EVV OBS
GIB MMIF
EVV OBS
42
-------
0
-0.1-
fl f>-
s
o
5
u.o
-0 1-
n 9-
EW MMIF
^^^
EW MMIF
i
EWOBS EWUMIF
I
^ ^
EW OBS EW MMIF
i
i
GIB OBS GIB MMIF
i
GI806S JlBlilJlh
I
EWMM1F GIB MMIF
scenario
EWMMJF GIB MMIF
scenario
i
EWOBS GIB MMIF
EWOBS UBI.W.T
1
GIB OBS GIB OBS
1
GIB OB? G:B:JBs
t,-. OB:
b
t'. . DBS
Figure 19. Gibson MCM differences with a) 90th percentile and b) 95th confidence
intervals.
43
-------
>.2 Martins Creek
3.2.1 Meteorological data comparisons
The wind roses for the four meteorological scenarios are presented in Figure 20. The Martins
Creek site-specific tower exhibits a strong southwest to northeast pattern. The two MMIF wind
roses do exhibit a southwest to northeast pattern, they also have more hours from other
directions. The ABE rose shows a strong westerly component to its winds with a secondary
maximum from the northeast.
Martins Creek
MMIF 1km
ABE
MMIF4km
Figure 20. May 1992 - May 1993 Martins Creek wind roses for a) Martins Creek, b) ABE,
c) MMIF 1 km, and d) MMIF 4 km.
Figure 21 shows the wind displacement distributions among the scenarios. For the most part,
displacements between the scenarios is less than 10 km. Figure 22 through Figure 24 compare
the surface roughness values by month and 10ฐ sector. ABE has the lower roughness values
throughout the year with Martins Creek spiking between 180ฐ and 260ฐ due to the presence of a
stand of trees from that direction.
44
-------
40-
30-
"c
E
aj
o
03
.220-
Q
10-
o-
I
f
I
*
*
*
t
t
1
*
t
t
*
*
1 Creek fv'fvllF 4 km-Martins Creek f.
j
IvIIF 4 km-MMIF 1 km
,
ABE-Martms Creek
Scenario
Figure 21. Wind displacement (km) among the Martins Creek meteorological scenarios.
45
-------
a 03
C 05
*> #
Esir i
jsrsr1
-IWM
Figure 22. Martins Creek study monthly surface roughness lengths (m) by 10 degree
sectors for a) January, b) February, c) March, and d) April.
46
-------
C 08
\_
Figure 23. Martins Creek study monthly surface roughness lengths (m) by 10 degree
sectors for a) May, b) June, c) July, and d) August.
-------
f
_J~
\_
*#*<ฃ.,*
Figure 24. Martins Creek study monthly surface roughness lengths (m) by 10 degree
sectors for a) September, b) October, c) November, and d) December.
Tables 9 through 11 present statistics for meteorological variables for the four scenarios. The
control difference, ABE - Martins Creek is highlighted in yellow. Box and whisker plots of the
variable distributions as well as bias distributions can be found in Appendix A.
Differences between MMIF wind speeds and observed wind speeds are lower than the
differences between the two observed datasets. Both MMIF scenarios over-predict when
compared to the Martins Creek winds. The MMIF 4 km simulation over-predicts
compared to the 1 km simulation.
The MMIF scenarios and ABE over-predict temperatures with the MMIF scenarios
having less over-prediction.
For pressures, the MMIF 1 km scenario over-predicts while the MMIF 4 km over-
predicts. ABE and Martins Creek do not differ, most likely due to ABE RH values being
used at Martins Creek.
48
-------
MMIF scenarios over-predict relative humidity and the MMIF 4 km scenario slightly
under-predicts compared to MMIF 1 km. ABE and Martins Creek do not differ, most
likely due to ABE RH values being used at Martins Creek.
For heat flux the MMIF 1 km scenario under-predicts while the MMIF 4 km scenario
over-predicts as does ABE.
For u*, both MMIF scenarios and ABE over-predict and are comparable to each other.
For w*, the MMIF 1 km scenario under-predicts while the MMIF 4 km and ABE over-
predict.
For the potential temperature gradient, the MMIF scenarios under-predict while ABE
over-predicts but values are relatively close.
For mixing heights (both convective and mechanical), the MMIF scenarios and ABE
over-predict.
For Monin-Obukhov length, the MMIF 1 km scenario under-predicts while the MMIF 4
km scenario and ABE over-predict. The MMIF 4 km biases appear to be an outlier when
compared to the MMIF 1 km biases and ABE biases.
Cloud cover differences also show relatively low agreement but this may be due to the
calculation methodology in AERMET when cloud cover is missing. ABE and Martins
Creek do not differ, most likely due to ABE cloud cover being used for Martins Creek.
Overall, while there are differences, the MMIF scenarios appear to show relatively good
agreement with the observed data. Differences between the MMIF scenarios and Martins Creek
counterparts is usually in line with the differences in the site-specific data and NWS data (i.e.,
ABE - Martins Creek).
49
-------
Table 9. Mean bias, fractional bias, root mean square error, and R2 for primary
meteorological variables.
Variable
Wind speed
Ambient
temperature
Pressure
Relative
humidity
Scenario
MMIF 1 km-Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
Mean
bias
0.58962
1.06006
0.47061
1.44933
0.23995
0.31796
0.07750
0.77481
3.24523
-1.83670
-5.06872
0
7.73760
7.61139
-0.12598
0
Fractional
bias
0.05979
0.09979
0.04196
0.09314
0.00021
0.00028
0.00007
0.00068
0.00081
-0.00046
-0.00126
0
0.02848
0.02804
-0.00049
0
RMSE
1.36357
1.66366
0.88542
2.17893
2.59116
2.42705
0.62537
1.89823
3.41835
2.11053
5.09355
0
16.10607
15.63945
3.84615
0
R2
0.53139
0.54214
0.83952
0.52709
0.92888
0.93764
0.99582
0.96915
0.98217
0.98365
1.0
1.0
0.55156
0.57766
0.96119
1.0
Table 10. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables.
Variable
Heat flux
u*
w*
dฉ/dz
Scenario
MMIF 1 km-Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
Mean
bias
-10.60956
15.76726
26.36619
19.72457
0.07382
0.09468
0.02140
0.09746
-0.03219
0.31158
0.33947
0.37888
-0.00149
-0.00151
-0.00003
0.00009
Fractional
bias
0.02594
0.09672
0.07375
0.19548
0.06145
0.09864
0.03829
0.11097
-0.02176
0.05245
0.07410
0.08166
-0.03543
-0.03592
-0.00084
0.00311
RMSE
39.27118
48.34657
62.95421
52.67964
0.15689
0.15717
0.09022
0.16210
0.41303
0.52603
0.49497
0.46271
0.00430
0.00430
0.00087
0.00329
R2
0.48069
0.75046
0.56546
0.89900
0.62727
0.62565
0.85083
0.57773
0.38025
0.56513
0.70120
0.86686
0.21148
0.21398
0.89759
0.51953
50
-------
Table 11. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables.
Variable
Zic
Zim
L
Cloud
cover
Scenario
MMIF 1 km-Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
MMIF 1 km -Martins Creek
MMIF 4 km -Martins Creek
MMIF 4 km -MMIF 1 km
ABE-Martins Creek
Mean
bias
205.20000
226.59382
18.69463
267.46126
158.29334
167.37733
10.09504
190.78705
-8.03602
360.81687
377.92163
42.92637
0.17313
0.56836
0.39765
0
Fractional
bias
0.05694
0.06771
0.01236
0.08113
0.10105
0.13701
0.03481
0.15120
0.10690
0.12134
-0.00433
0.09176
-0.01996
0.00469
0.03361
0
RMSE
567.66350
526.93158
216.79283
393.69289
371.44030
353.60026
145.49705
356.34586
2087.77469
2447.01436
1819.59481
667.80236
5.05242
4.97054
3.05554
0
R2
0.27664
0.36883
0.87216
0.81594
0.48209
0.48169
0.90391
0.55482
0.07088
0.04617
0.51254
0.25921
0.08225
0.09158
0.54030
1.0
3.2.2 AERMOD results
QQ plots for various averaging times are shown in Figure 25. For the 1-hour averages (Figure
25. a), ABE tends to over-predict outside of the factor of 2 line except for the highest
concentrations. The other scenarios tend to slightly over predict below 1,000 ng/m3, with the
MMIF 4 km showing better agreement with the observed concentrations. For the highest
concentrations, the MMIF 4 km and Martins Creek under-predict while the other two scenarios
over-predict. All are within a factor of two of the observations.
For the 3-hour averages (Figure 25.b), all scenarios over-predict below 600 |j,g/m3. Again, ABE
tends to be the higher of the concentrations. As with the 1-hour concentrations, at the higher
concentrations, the MMIF 4 km and Martins Creek under-predict. For the 24-hour averages
(Figure 25.c), the model values under-predict between 25 and 50 |j,g/m3 with over-prediction at
concentrations higher than 50 |j,g/m3. For the annual averages (Figure 25.d), all scenarios under-
predict. Figure 26 and Figure 27 show the 3-hr and 24-hour screening results. Both show that
the four scenarios over-predict at 3-hour and 24-hour averages but are within the factor of two
limit. Figure 28 shows the fractional biases for 1-hour, 3-hour, and 24-hour as well as a CPM
based on those values. ABE tends show over-predictions at all averages, agreeing with the QQ-
plots. Fractional biases for Martins Creek and the two MMIF scenarios over-predict but for 1-
51
-------
hour they do cross over zero, indicating under-prediction. All scenarios over-predict at the 24-
hour average and the CPM plot shows that.
1-hr
Figure 25. Martins Creek QQ plots for a) 1-hour, b) 3-hour, c) 24-hour, and d) annual
averages. Concentrations are in (j,g/m3.
52
-------
2-
_g
.Is
">
Q
2 0
co
o
co
S
CO
-2-
-2
-1 0 1
of
Figure 26. Martins Creek 3-hour screening results.
53
-------
2-
_g
.Is
">
Q
2 0
co
o
co
S
CO
-2-
-2
-1 0 1
of
Figure 27. Martins Creek 24-hour screening results.
54
-------
C ooo
4375
Figure 28. Martins Creek fractional biases for a) 1-hour, b) 3-hour, c) 24-hour, and d)
CPM based on fractional biases.
Table 12 shows the absolute fractional biases and CPM for each scenario for Martins Creek and
Figure 29 shows the CPM values with the 5th and 95th percentiles of the bootstrap results. The
CPM values show that Martins Creek site-specific data performed better with the other three
having comparable values. Table 13 shows the MCM values for the different model pair
differences and Figure 30 shows the MCM values with the 90th and 95th confidence intervals.
Based on Table 13, Martins Creek tends to be the better performing scenario. The plots in
Figure 30 show that the differences between the MMIF simulations and Martins Creek are
statistically significant.
55
-------
Table 12.1-hour, 3-hour, 24-hour absolute fractional biases and composite performance
measures for Martins Creek meteorological scenarios.
Scenario
ABE
Martins
Creek
MMIF 1 km
MMIF 4 km
AFB
1-hour
0.78
0.40
0.56
0.50
3-hour
0.51
0.08
0.64
0.31
24-hour
0.56
0.56
0.77
0.76
CPM
0.61
0.34
0.66
0.52
0.8
0.6-
0.2
0.0
ABE Martins Creek MMIF 1 km
scenario
MMIF 4 km
56
-------
Figure 29. Martins Creek composite performance metric values with 5th and 95th
percentiles of the CPM values from the bootstrap results.
Table 13. Model comparison measures (MCM) for the four Gibson meteorological
scenarios.
MCM Scenario
ABE - Martins Creek
MMIF 1 km - ABE
MMIF 1km - Martins Creek
MMIF 4 km - ABE
MMIF 4 km - Martins Creek
MMIF 4 km - MMIF 1km
MCM
0.27
0.05
0.31
-0.09
0.18
-0.14
Best performing scenario
Martins Creek
ABE
Martins Creek
MMIF 4 km
Martins Creek
MMIF 1 km
57
-------
s
o
5
-U.<ฃ
0.4-
o
5
0.0
-O.Z
(
A6E4IM
i
ABE Vป-i
raOM* MMIF1
1
.01 Cป*** UUIF 1
I
m-ABE MMtFlbn-
1
m-AfiE UMFIbn.
1
MvttoQMk MWIF4
scenano
i
IJIITIJ &Mซ MMlF 4
scenario
i
m-ABE MMiF*im.
i
i
i
WMimCraw MKUF4M
1
Martin Cm* MMIF 4 km
a
MMIF 1 km
b
MMIF 1 ton
Figure 30. Martins Creek MCM differences with a) 90th percentile and b) 95th confidence
intervals.
58
-------
3.3 Herculaneum
3.3.1 Meteorological data comparisons
Figure 31 and Figure 32 show the wind roses for the Herculaneum site-specific tower, CPS, and
three MMIF grid cells. Herculaneum and the three MMIF roses appear to show dominant
directions from the northeast, southwest, west, and northwest. CPS is dominated by a
southeasterly flow.
NORTH
Herculaneum
SOUTH
CPS
Figure 31. 2009 wind roses for a) Herculaneum and b) CPS.
59
-------
MMIF4km
MMIFlZkm
MMIF 36 km
Figure 32. 2009 wind roses for a) MMIF 4 km, b) MMIF 12 km, and c) MMIF 36 km.
Figure 33 compares the wind displacement distributions among several difference among the
Herculaneum scenarios. For the most part, displacements are less than 20 km. Figure 34
through Figure 36 compare the surface roughness values by month and wind direction. Surface
roughness values for Herculaneum and CPS can vary dramatically by direction and month.
60
-------
50-
40 -
o>
E
oป
o
_g
Q.
to
10-
Scenario
Figure 33. Wind displacement (km) among the Herculaneum meteorological scenarios.
61
-------
a ฐ4
,#> 4?
Direction
Figure 34. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors
for a) January, b) February, c) March, and d) April.
62
-------
Figure 35. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors
for a) May, b) June, c) July, and d) August.
63
-------
Figure 36. Herculaneum study monthly surface roughness lengths (m) by 10 degree sectors
for a) September, b) October, c) November, and d) December.
Table 14 through Table 16 show statistics for the meteorological variables for several scenario
differences. Box and whisker plots of the variable distributions as well as bias distributions can
be found in Appendix A.
Differences between MMIF wind speeds and observed wind speeds are lower than the
differences between the two observed datasets but are comparable.
The MMIF 4 and 12 km scenarios over-predict temperatures while the MMIF 36 km and
CPS over-predict temperatures with the MMIF scenarios having less over-prediction.
For pressures, the MMIF scenario under-predict. CPS and Herculaneum do not differ,
most likely due to CPS values being used at Herculaneum in AERMET processing.
MMIF scenarios and CPS over-predict relative humidity and heat flux.
For u*, both the MMIF 4 km under-predicts while the MMIF 12 and 36 km scenarios
over-predict.
MMIF scenarios and CPS over-predict w*.
64
-------
For the potential temperature gradient, the MMIF scenarios over-predict while the CPS
over-prediction is essentially zero.
For mixing heights (both convective and mechanical) and Monin-Obukhov length, the
MMIF scenarios and CPS over-predict.
Cloud cover differences also show relatively low agreement but this may be due to the
calculation methodology in AERMET when cloud cover is missing. ABE and Martins
Creek do not differ, most likely due to ABE cloud cover being used for Martins Creek.
Overall, while there are differences, the MMIF scenarios appear to show relatively good
agreement with the observed data. Differences between the MMIF scenarios and Herculaneum
are usually in line with the CPS - Herculaneum biases.
65
-------
Table 14. Mean bias, fractional bias, root mean square error, and R2 for primary
meteorological variables.
Variable
Wind speed
Ambient
temperature
Pressure
Relative
humidity
Scenario
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
Mean
bias
1.11391
1.10712
1.15359
-0.00679
0.03969
0.04647
1.37366
-0.03556
0.00787
-0.05384
0.04342
-0.01828
-0.06170
-0.38928
-0.72089
-2.80479
-6.45605
-2.08390
-5.73516
-3.65126
0
2.81507
1.88505
1.06450
-0.93002
-1.75057
-0.82055
0.81473
Fractional
bias
0.10602
0.10918
0.11262
0.00354
0.00759
0.00411
0.10400
-0.00002
0.00002
-0.00004
0.00004
-0.00002
-0.00006
-0.00035
-0.00018
-0.00070
-0.00162
-0.00052
-0.00144
-0.00092
0
0.00556
0.00103
-0.00176
-0.00463
-0.00746
-0.00284
0.00436
RMSE
1.60036
1.56114
1.60752
0.30025
0.46448
0.30716
1.93392
2.65892
2.66439
2.55690
0.34129
0.63005
0.47229
2.09388
1.42029
3.05647
6.56495
2.10606
5.75746
3.68374
0
14.68699
14.86928
14.50951
2.81453
4.48268
3.12158
9.40803
R2
0.58907
0.57557
0.56262
0.97318
0.93287
0.96801
0.59851
0.93389
0.93362
0.93889
0.99886
0.99602
0.99780
0.96207
0.97096
0.97200
0.97386
0.99801
0.99454
0.99470
1.0
0.58847
0.59163
0.59984
0.98725
0.96733
0.98291
0.75540
66
-------
Table 15. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables.
Variable
Heat flux
u*
w*
dฉ/dz
Scenario
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
Mean
bias
18.72392
30.29101
31.98523
11.55608
13.21507
1.65898
6.65078
-0.01028
0.05683
0.10084
0.06712
0.11108
0.04397
0.01861
0.44585
0.70574
0.72036
0.34209
0.33219
-0.00180
0.17976
0.00314
0.00298
0.00280
0.00004
-0.00028
-0.00035
5.6xlO-7
Fractional
bias
0.03237
0.00573
0.01935
0.08710
0.06863
0.01543
0.04541
-0.02039
0.03896
0.07518
0.06483
0.10116
0.03955
0.02198
0.11391
0.15819
0.16160
0.06955
0.06015
-0.00490
0.08154
0.07345
0.06886
0.06495
0.00012
-0.00648
-0.00710
0.00078
RMSE
56.91295
79.39036
82.15212
35.43597
51.23716
38.98357
23.37517
0.14290
0.16563
0.18834
0.08534
0.13904
0.07255
0.12423
0.65988
0.88205
0.88035
0.42287
0.50998
0.30334
0.33074
0.00647
0.00649
0.00620
0.00151
0.00214
0.00201
0.00304
R2
0.58847
0.59163
0.59984
0.98725
0.96733
0.98291
0.75540
0.58847
0.59163
0.59984
0.98725
0.96733
0.98291
0.75540
0.58847
0.59163
0.59984
0.98725
0.96733
0.98291
0.75540
0.58847
0.59163
0.59984
0.98725
0.96733
0.98291
0.75540
67
-------
Table 16. Mean bias, fractional bias, root mean square error, and R2 for calculated
meteorological variables.
Variable
Zic
Zlm
L
Cloud
cover
Scenario
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
MMIF 4 km-Herculaneum
MMIF 12 km-Herculaneum
MMIF 36 km-Herculaneum
MMIF 12 km-MMIF 4 km
MMIF 36 km-MMIF 4 km
MMIF 36 km-MMIF 12 km
CPS-Herculaneum
Mean
bias
566.12886
570.47601
555.89452
25.53641
10.05379
-17.08090
178.88253
89.93761
111.37459
107.75352
21.22066
17.59007
-3.63059
28.58368
249.61225
231.00914
288.82372
-16.39179
39.35898
55.75078
141.90371
3.71020
3.67508
3.55515
-0.03505
-0.15502
-0.11998
0
Fractional
bias
0.21689
0.21825
0.21464
0.00685
0.00148
-0.00619
0.11968
0.03008
0.04574
0.04503
0.01780
0.01751
-0.00001
0.02725
-0.10265
-0.06609
-0.45512
0.06853
0.09618
0.07233
0.01109
0.23923
0.23811
0.23551
-0.00294
-0.00932
-0.00655
0
RMSE
689.85788
699.26281
682.41628
82.12590
134.17900
113.15424
288.74963
523.68956
534.07238
544.49645
105.80767
151.40997
122.67669
276.80805
1681.53419
1699.98627
1823.38370
1356.42715
1458.10173
1570.10520
1276.36930
5.81028
5.78924
5.72211
0.45694
0.94022
0.88577
0
R2
0.55479
0.55115
0.57861
0.98265
0.94961
0.96540
0.79372
0.20612
0.20443
0.18224
0.96336
0.92336
0.94935
0.62259
0.00120
0.00065
0.00419
0.00001
0.01241
0.00005
0.20768
0.13757
0.13824
0.14100
0.98193
0.92781
0.93537
1.0
68
-------
3.3.2 AERMOD results
Figure 37 shows the 24-hour QQ plots for Herculaneum. Since the pollutant is lead, daily
averages are the only averaging period available at the monitors. Figure 37 shows that all five
scenarios are under-predicting concentrations. Figure 38 shows the 24-hour screening results
and the results concur with the QQ-plots. With the exception of CPS, the scenarios are outside
the factor of two limit but the MMIF scenarios are comparable to the Herculaneum site specific
scenario.
25-
20-
15-
T3
J3
0)
T3
O
10-
5-
0-
4 8
Observed
12
Figure 37. Herculaneum 24-hour QQ plots. Concentrations are in (o,g/m3
69
-------
2-
_g
.Is
">
Q
2 0
co
o
co
S
CO
-2-
-2
-1 0
of
Figure 38. Herculaneum 24-hour screening results.
4. Summary and Conclusions
MMIF output was evaluated for three areas involving differing complexities of terrain from
relatively flat to complex, differing resolutions of prognostic model output, and two different
pollutants, 862 and lead. Evaluation of the meteorological data for several variables indicated
that while there are differences between the prognostic model output and observations, the
prognostic model output was not unreasonable. However, it should be noted that small
differences in meteorological variables, such as temperature, wind speed or direction, can lead to
vastly different air quality results.
70
-------
Statistical evaluations of the resulting AERMOD results from observed meteorological data and
prognostic data exhibited differing behaviors.
For Gibson, all meteorological scenarios, observed or prognostic, tended toward under-
prediction of SO2 concentrations at higher concentrations but the screening analyses indicated
biases close to zero. At the 3-hour averaging time, all scenarios exceeded the factor-of-two limit
in the screening analysis but were within a factor-of-two for the 24-hour averages. Evaluations
of the MCM values indicated the best performing scenarios tended to be the observed
meteorological data but that the prognostic outputs were not statistically different. Of the two
prognostic outputs, the MMIF output of the grid cell of Gibson performed better than the grid
cell of the Evansville NWS station.
For the Martins Creek case study, the meteorological scenarios tended to over-predict and the
CPM and MCM analyses revealed that the Martins Creek site-specific observed data performed
better. From the MCM analysis, the prognostic meteorological scenarios were statistically
different from the Martins Creek site-specific scenario. Compared to the ABE NWS data, both
prognostic scenarios were not statistically different, which was a goal of this evaluation;
prognostic meteorological data performs no worse than a representative NWS station.
Finally, for Herculaneum, detailed statistical analyses could not be performed due to the lack of
hourly lead observations, but a screening analysis of the outputs indicated that while the
prognostic scenarios were not within the factor-of-two agreement for 24-hour averages, their
performance was actually comparable to the Herculaneum site-specific tower.
While there is a need for more evaluations in more challenging environments, i.e. complex
terrain or meteorological conditions, these results indicate promise for the use of prognostic
meteorological data in AERMOD applications.
71
-------
5. References
Cox, W.M. and J.A. Tikvart, 1990: A statistical Procedure for Determining the Best Performing
Air Quality Simulation Model. Atmos. Environ., 24A (9): 2387-2395.
Environ, 2014: The Mesoscale Model Interface Program (MMIF) Version 3.1 User's Manual.
Frost, Kali, 2014: AERMOD Performance Evaluation for Three Coal-fired Electrical Generating
Units in Southwest Indiana. Journal of the Air & Waste Management Association., 64:3
280-290.
Perry Steven G., Alan J. Cimorelli, Robert J. Paine, Roger W. Erode, Jeffrey C. Weil, Akula
Venkatram, Robert B. Wilson, , Russell F. Lee., and Warren D. Peters, 2005:
AERMOD: A Dispersion Model for Industrial Source Applications. Part II: Model
Performance Against 17 Field Study Databases. Journal of Applied Meteorology, 44,
694-708.
U.S. EPA, 1992: Protocol for Determining the Best Performing Model, EPA-454/R-92-025.
U.S. Environmental Protection Agency, Research Triangle Park, NC.
U.S. EPA, 2003: AERMOD: Latest Features and Evaluation Results, EPA-454/R-03-003. U.S.
Environmental Protection Agency, Research Triangle Park, NC.
U.S. EPA, 2013: AERSURFACE User's Guide. EPA-454/B-08-001. U.S. Environmental
Protection Agency, Research Triangle Park, North Carolina 27711.
U.S. EPA, 2014: Meteorological Model Performance for Annual 2011 WRF v3.4 Simulation.
http://www.epa.gov/ttn/scram/reports/MET_TSD_201 l_fmal_ll-26-14.pdf
U.S. EPA, 2015a: Guideline on Air Quality Models. 40 CFR Part 51 Appendix W.
U.S. EPA, 2015b: Guidance on the Use of the Mesoscale Model Interface Program (MMIF) for
AERMOD Applications. EPA-454/R-15-004. U.S. Environmental Protection Agency,
Research Triangle Park, NC 27711.
72
-------
Appendix A. Meteorological data comparisons
A.I Gibson
Figures A-l through A-6 compare the distributions of several meteorological variables among
the four meteorological datasets (panel a of the figures). These variables are the input variables
into AERMET: wind speed, temperature, pressure, relative humidity, albedo, and Bowen ratio.
Comparisons of key meteorological variables calculated by AERMET including: heat flux, u*,
w*, Monin-Obukhov length, and mixing heights can be found in Figures A-7 through A-14. In
panel b of each figure are distributions of the biases of each variable among the different
scenarios. The first bias distribution is between the two MMTF outputs, GIB MMTF and EVV
MMTF. This bias shows how the meteorology can vary in a distance of a few grid cells. The
next two bias distributions show the differences between MMTF and observations for the Gibson
and Evansville grid cells respectively. The final distribution shows the bias between the two
observed datasets and acts as a control since, in the absence of prognostic data, the two observed
datasets would be the only ones available for consideration. If the differences between the
MMTF outputs and the two observed datasets are similar, then from qualitative standpoint, the
MMIF output is reasonable for use.
Based on the figures the following can be seen:
For wind speed (Figure A-l), EVV OBS tends to show lower wind speeds (Figure A-l .a)
among the four datasets. Each locations' MMIF - OBS bias tends to be more positive,
correlating with the distributions in Figure A-l.a. The GIB MMIF - EVV MMIF bias
trends positive as does the GIB OBS- EVV OBS, indicating that the MMIF output
appears reasonable when compared to the observed datasets.
Temperature (K) distributions (Figure A-2) indicate very similar patterns for all scenarios
with the bias distributions tending to show fairly unbiased results among the scenario
(median bias near zero degrees).
The GIB OBS pressure distribution (Figure A-3) shows higher pressures than over the
other scenarios which also bears out in the bias distributions. The two MMIF scenarios
are similar. The differences between the two observed datasets indicate higher pressures
at Gibson over Evansville.
Relative humidity (Figure A-4) is identical for GIB OBS and EVV OBS. This is due to
EVV being the source of RH for GIB in AERMET. The two MMIF scenarios are very
similar. Their bias distributions when compared to their respective OBS dataset are very
similar.
Daytime albedo (Figure A-5) somewhat similar distributions among the scenarios with
GIB MMIF - EVV MMIF, GIB MMIF - GIB OBS exhibiting positive bias and the other
two being negative.
A-l
-------
Bowen ratios (Figure A-6) vary widely among the scenarios. This could be due to the
subjectivity of the average, dry, and wet selections in AERSUKFACE.
Heat flux (Figure A-7) shows GIB OBS with lower values compared to the other
scenarios. The differences between the two MMIF sites and the differences between the
observed heat fluxes were small compared to the differences between the respective
MMIF and observed datasets.
Surface friction velocity, u* (Figure A-8) shows that the MMIF scenarios tended to have
higher values than the observed scenarios. The GIB MMIF - EVV MMIF bias
distribution is similar to the GIB OBS - EVV OBS bias distribution.
Convective velocity scale, w* (Figure A-9) show a similar trend as u*, the MMIF
scenarios exhibited higher values than the observed scenarios.
Monin-Obukhov length, L (Figure A-10) distributions appear very similar across the
scenarios with bias distributions being similar as well.
Convective mixing height, Zic (Figure A-l 1) distributions and bias distributions indicate
that the MMIF mixing heights were higher than their observed counterparts. Both the
MMIF and observed bias plots indicate a tendency for Gibson mixing heights to be lower
than the Evansville mixing heights.
Mechanical mixing height, Zim (Figure A-12) distributions show similar behavior as the
convective mixing heights.
Potential temperature gradient (dฉ/dz), (Figure A-13), distributions indicate smaller lapse
rates for the MMIF data compared to the observed data.
Cloud cover (Figure A-14) distributions show no differences between EVV and GIB for
the observed datasets since EW cloud cover is used at Gibson. Cloud cover estimates
for MMIF are calculated in AERMET. Differences between the MMIF and OBS for
Gibson and Evansville show a large swing in biases.
For the most part, the meteorological data processed through MMIF for Gibson and Evansville
appear reasonable when compared to the Gibson and Evansville observations.
A-2
-------
15-
10-
o
0>
Q.
lf>
5-
0
10
5-
T3
8 o-
%
-5-
-10
a
GIB DBS GIBMMIF EW OBS EW MMIF
Scenario
4
1
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EVV OBS GIB OBS-EW OBS
Scenario
Figure A-l. Gibson wind speed (m/s): a) annual distributions and b) bias distributions.
A-3
-------
300
2!
3
~m
1
1280
260
15-
10-
5
0)
3
1
Q.
1 o-
ฃ
-5
-10-
GIB OBS
a
h
GIB MMIF EWOBS
Scenario
EWMMIF
,
<
4
<
b
GIB MMIF-EW MMIF GIB MMIF-GIB DBS EW MMIF-EW DBS GIB OBS-EW OBS
Scenario
Figure A-2. Gibson ambient temperature (K): a) annual distributions and b) bias
distributions.
A-4
-------
1020-
1010-
B
ฃinnn-
1000
990
980
20
10
GIBOBS GIBMMIF EW OBS EW MMIF
Scenario
-10
E
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIB OBS-EW OBS
Scenario
Figure A-3. Gibson station pressure (mb): a) annual distributions and b) bias
distributions.
A-5
-------
100i
75-
50-
25
60
30
-30
-60-
GIBOBS GIB MMIF EW OBS
Scenario
EW MMIF
GIB MMIF-EW MMIF GIB MMIF-GIB DBS EW MMIF-EW DBS GIB OBS-EW OBS
Scenario
Figure A-4. Gibson relative humidity (percent): a) annual distributions and b) bias
distributions.
A-6
-------
08
0.6-
0.4
0.2-
0.10
0.05
0.00
-0.05-
GIBOBS
GIB MMIF EW OBS
Scenario
EW MMIF
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIB OBS-EW OBS
Scenario
Figure A-5. Gibson daytime albedo (fraction): a) annual distributions and b) bias
distributions.
A-7
-------
4
2
GIBOBS GIBMMIF EVV OBS
Scenario
EWMMIF
03 -
o: 2
-2
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIB OBS-EW OBS
Scenario
Figure A-6. Gibson Bowen ratio: a) annual distributions and b) bias distributions.
A-8
-------
400
300-
5200-
100-
o-
200
-200-
GIB DBS
to 0
<0
GIBMMIF EWOBS
Scenario
EW MMIF
GIB MMIF-EW MMIF GIB MMIF-GIB DBS EVV MMIF-EW DBS GIB OBS-EW OBS
Scenario
Figure A-7. Gibson heat flux (W/m2): a) annual distributions and b) bias distributions.
A-9
-------
1.25
1.00
0.75
*
0.50
0.25
0.00
1.0-
0.5-
*
ID
u.u
GIB DBS GIBMMIF EW OBS EW MMIF
Scenario
.
1
i
*
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EVV OBS GIB OBS-EW OBS
Scenario
Figure A-8. Gibson surface friction velocity, u* (m/s): a) annual distributions and b) bias
distributions.
A-10
-------
3-
2
k
1
0
2
1
^o
-1
-2
GIB OBS
GIB MMIF EWOBS
Scenario
'
a
EW MMIF
i
I
!
'
h
u
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIB OBS-EW OBS
Scenario
Figure A-9. Gibson convective velocity scale, w* (m/s): a) annual distributions and b) bias
distributions.
A-ll
-------
5000-
- 0-
-5000-
10000
- 0
-10000
I
j
a
^^
GIB OBS GIB MMIF EWOBS EWMMIF
Scenario
4
b
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIBOBS-EWOBS
Scenario
Figure A-10. Gibson Monin-Obukhov length (m): a) annual distributions and b) bias
distributions.
A-12
-------
3000
2000-
1000
2000
1000-
-1000
-2000
GIBOBS
GIBMMIF EWOBS
Scenario
EW MMIF
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW DBS GIB OBS-EW OBS
Scenario
Figure A-ll. Gibson convective mixing height (m): a) annual distributions and b) bias
distributions.
A-13
-------
3000-
2000-
E
Kj
1000-
0
2000
1000
N
0
-1000
-2000
1
I
GIB OBS
I
.
.
GIB MMIF EWOBS
Scenario
I
I
j
I
:
.
a
EW MMIF
i
i
I
b
=
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW DBS GIB OBS-EW OBS
Scenario
Figure A-12. Gibson mechanical mixing height (m): a) annual distributions and b) bias
distributions.
A-14
-------
0.03
N0.02
0.01-
GIBOBS GIB MMIF EW OBS EW MMIF
Scenario
0.02
0.01
0.00
Q
-0.01
-0.02
GIB MMIF-EW MMIF GIB MMIF-GIB OBS EW MMIF-EW OBS GIB OBS-EW OBS
Scenario
Figure A-13. Gibson potential temperature gradient (K/m) above Zic: a) annual
distributions and b) bias distributions.
A-15
-------
10,0
7.5
-------
A. 2 Martins Creek
Figures A-15 through A-20 show the distributions of the AERMET input variables for the
Martins Creek study and Figures A-21-27 show the calculated variables from AERMET. As
with the Gibson plots, panel b of each figure shows key biases. The first two bias distributions
show the biases of the two MMIF scenarios against the Martins Creek site-specific data. The
third fourth distribution compares the two MMIF scenarios against each other, and the fourth
distribution compares the two observed datasets and acts as a control. The following can be seen
in the plots:
For wind speed (Figure A-15), Martins creek tends to show lower wind speeds (Figure A-
15.a) among the four datasets. The bias distributions appear very similar between the
MMIF scenarios and Martins Creek and ABE and Martins creek.
Temperature (K) distributions (Figure A-16) indicate very similar patterns for all
scenarios with the bias distributions tending to show fairly unbiased results among the
scenario (median bias near zero degrees).
The pressure distribution (figure A-17) appear to be similar with the MMIF 1 km
scenario exhibiting slightly higher values.
Relative humidity (Figure A-18) is identical for Martins Creek and ABE are identical
being due to ABE being the source of RH for Martins Creek in AERMET. The two
MMIF scenarios are very similar. Their bias distributions are very similar.
Daytime albedo (Figure A-19) show comparable values for ABE and the MMIF
scenarios. Martins Creek has a wider distribution of values. Bias distributions for the two
MMIF scenarios are tighter than for the ABE - Martins Creek bias distribution.
Bowen ratios (Figure A-20) vary widely among the scenarios. This could be due to the
subjectivity of the average, dry, and wet selections in AERSURFACE.
Heat flux (Figure A-21) distributions show comparable distributions between Martins
Creek and the MMIF 1 km scenario while ABE and the MMIF 4 km scenario appear
similar. The bias distributions appear comparable to each other.
Surface friction velocity (Figure A-22) distributions show comparable distributions
among the scenarios. The bias distributions appear comparable to each other.
Convective velocity scale (Figure A-23) distributions show comparable distributions
between Martins Creek and the MMIF 1 km scenario while ABE and the MMIF 4 km
scenario appear similar. The bias distribution for the MMIF 1km and Martins Creek is
noticeably lower than the others which appear comparable to each other.
Monin-Obukhov length, L (Figure A-24) distributions appear very similar across the
scenarios with bias distributions being similar as well. It is difficult to detect differences,
which could be due to large absolute values for near neutral conditions.
Convective mixing height, Zic (Figure A-25) distributions and bias distributions indicate
that the MMIF mixing heights were comparable to ABE and Martins Creek
Mechanical mixing height, Zim (Figure A-26) distributions show similar behavior as the
convective mixing heights.
A-17
-------
Potential temperature gradient (d0/dz), (Figure A-27), distributions indicate smaller lapse
rates for the MMIF data compared to the observed data.
Cloud cover (Figure A-28) distributions show no differences between ABE and Martins
Creek for the observed datasets since ABE cloud cover is used at Martins Creek. Cloud
cover estimates for MMIF are calculated in AERMET. Differences between the MMIF
and Martins Creek show a large swing in biases.
For the most part, the meteorological data processed through MMIF for Martins Creek appear
reasonable when compared to Martins Creek observations.
A-18
-------
15
10-
S
Q.
CO
5-
o-
10-
5-
T3
Q.
CO
0
-5-
I
I
4
a
Martins Creek ABE MMIF 1 km MMIF 4 km
Scenario
1
"
1
D
MMIF 1 km-Martins Creek MMIF 4 km-Mamns Creek MMIF 4 km-MMIF 1 km ABE-Marbns Creek
Scenario
Figure A-15. Martins Creek wind speed (m/s): a) annual distributions and b) bias
distributions.
A-19
-------
300
290
2
7c
|280
1
270
260
Temperature
-* W 0
o o c
Martins Creek ABE MMIF 1 km
Scenario
i
j
H
i
^^^H
a
MMIF 4 km
;
^^^2
MMIF 1 km-Martins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1km
Scenario
b
ABE -Martins Creek
Figure A-16. Martins Creek ambient temperature (K): a) annual distributions and b) bias
distributions.
A-20
-------
1020-
1000-
980-
960
Martins Creek
ABE
Scenario
MMIF 1 km
MMIF 4 km
10
L
MMIF 1 km-Martms Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km
Scenario
ABE-Martins Creek
Figure A-17. Martins Creek station pressure (mb): a) annual distributions and b) bias
distributions.
A-21
-------
100
75
X
s
50
25
50-
25-
o:
0-
-25-
-50
Martins Creek
*
ABE MMIF 1 km
Scenario
a
MMIF 4 km
^^^m
b
MMIF 1 km-MartinsCreek MMF 4 tan-Martins Creek MMIF 4 krn-MMIF 1 km
Scenario
ABE Martins Creek
Figure A-18. Martins Creek relative humidity (percent): a) annual distributions and b)
bias distributions.
A-22
-------
0.75
0
o
0.25-
i
i
i
<
i
i
Martins Creek ABE MMIF 1 km
Scenario
0.00
-0.05
-ง-0.10
u>
-0.15
-0.20
-0.25
MMIF 4 km
MMIF 1 km-Mamns Creek MMIF 4 km Martins Creek MMIF 4 km-MMIF 1 km
Scenario
a
b
ABE-Martms Creek
Figure A-19. Martins Creek daytime albedo (fraction): a) annual distributions and b) bias
distributions.
A-23
-------
2.0i
1.5
1.0
0.5-
0.0
1
0
-1
Martins Creek
ABE
Scenario
MMIF 1 km
MMIF4km
MMIF 1 km-Martins Creek MMIF 4 km-Martms Creek MMIF 4 km-MMIF 1 km ABE-Martins Creek
Scenario
Figure A-20. Martins Creek Bowen ratio: a) annual distributions and b) bias distributions.
A-24
-------
300
X
3
q=
"<3
100
o-
200
100
1 0
-100
-200
ซ
a
Martins Creek ABE MMIF 1 km MMIF 4 km
Scenario
i
:
b
= ' I
MMIF 1 km Martins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km ABE Martins Creek
Scenario
Figure A-21. Martins Creek heat flux (W/m2): a) annual distributions and b) bias
distributions.
A-25
-------
1.5-
1.0-
*
0.5-
o.o-
1.0
0.5
*
z>
0.0
-0.5
.
Martins Creek
.
.
ABE MMIF 1 km
Scenario
*
ป
a
MMIF 4 km
I
b
MMIF 1 km-Martins Creek MMIF 4 km-Marlins Creek MMIF 4 km-MMIF 1 km
Scenario
ABE-Martins Creek
Figure A-22. Martins Creek surface friction velocity, u* (m/s): a) annual distributions and
b) bias distributions.
A-26
-------
Martins Creek ABE MMIF1 km MMIF 4 km
Scenario
MMIF 1 knvMartins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km ABE-Martins Creek
Scenario
Figure A-23. Martins Creek convective velocity scale, w* (m/s): a) annual distributions
and b) bias distributions
A-27
-------
5000-
0-
-5000-
Martins Creek
ABE MMIF 1 km
Scenario
I
I
MMIF 4 km
10000
-10000
MMIF 1 km-Martins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km ABE-Martins Creek
Scenario
Figure A-24. Martins Creek Monin-Obukhov length (m): a) annual distributions and b)
bias distributions.
A-28
-------
3000-
2000-
.H
M
o-
2000
1000
ฃ3 0
-1000
-2000
.
Martins Creek
.
ABE MMIF 1 km
Scenario
1
1
MMIF 4 km
I
.
i
^
MUf 1 km-Martins Cieek MMIF 4 km-Martins Creek MMIF 4 km-MIWF 1 km
Scenario
a
b
ABE-Martins Creek
Figure A-25. Martins Creek convective mixing height (m): a) annual distributions and b)
bias distributions.
A-29
-------
4000-
3000-
|2000-
1000-
0
3000
2000
1000
rsi
0
-1000
-2000
i
I
.
a
>
Martins Creek ABE MMIF 1 km MMIF 4 km
Scenario
>
i
I
:
b
1 i
'
',
MMIF 1 tan-Martins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km ABE-Martins Creek
Scenario
Figure A-26. Martins Creek mechanical mixing height (m): a) annual distributions and b)
bias distributions.
A-30
-------
0.04-
n r\i-
2
P
Q
0.02-
0.01-
0.02
0.01
a
go.oo
-0.01
-0.02
*
^^
Martins
I 1
a
Creek ABE MMIF 1 km MMIF 4 km
Scenario
|
ป
t
:
>
ป
:
|
,
^
b
MMIF 1 km-Martins Creek MMIF 4 km-Martins Creek MMIF 4 km-MMIF 1 km ABE-Martins Creek
Scenario
Figure A-27. Martins Creek potential temperature gradient (K/m) above Zic: a) annual
distributions and b) bias distributions.
A-31
-------
Cloud Cover
O M CJl ->l p
O Ul O Ul O
-
*
a
Martins Creek ABE MMIF 1 km MMIF 4 km
Scenario
10
5
g
o n
o
O
-5
-10-
.
b
MMIF 1 km-Martins Creek MMIF 4 km. Martins Creek MMIF 4 km-MMIF 1 km ABE-Maitns Creek
Scenario
Figure A-28. Martins Creek cloud cover (tenths*10): a) annual distributions and b) bias
distributions.
A-32
-------
A.3 Herculaneum
Figures A-29 through A-34 show the distributions of the AERMET input variables for the
Herculaneum study and Figures A-35 through A-41 show the calculated variables from
AERMET. As with the other evaluation plots, panel b of each figure shows key biases. The first
three bias distributions show the biases of the three MMIF scenarios against the Herculaneum
site-specific data. The fourth through sixth distribution compares the three MMIF scenarios
against each other, and the final distribution compares the two observed datasets and acts as a
control. The following can be seen in the plots:
For wind speed (Figure A-29), Herculaneum tends to show lower wind speeds (Figure A-
29.a) among the datasets. The bias distributions appear very similar between the MMIF
scenarios and Herculaneum and CPS and Herculaneum. Biases between the three MMIF
scenarios are relatively low and comparable to each other.
Temperature (K) distributions (Figure A-30) indicate very similar patterns for all
scenarios with the bias distributions tending to show fairly unbiased results among the
scenario (median bias near zero degrees).
The pressure distribution (Figure A-31) appear to be similar among the two observed
datasets and the MMIF 4 km scenario. Pressures tend to be lower with the 12 and 36 km
MMIF scenarios.
Relative humidity (Figure A-32) are comparable for Herculaneum and CPS. The three
MMIF scenarios are very similar. Their bias distributions are also very similar and
comparable to the CPS - Herculaneum bias distribution.
Daytime albedo (Figure A-33) show comparable values among the scenarios. The bias
distributions for the MMIF 4 km scenario shows a more positive bas than the other two
MMIF scenarios.
Bowen ratios (Figure A-34) are comparable for the two observed datasets and
comparable among the three MMIF scenarios. MMIF bias distributions show more
differences than the CPS -Herculaneum biases but are comparable.
Heat flux (Figure A-35) distributions show comparable among the scenarios. The bias
distributions appear comparable to each other.
Surface friction velocity (Figure A-36) distributions show comparable distributions
among the scenarios. For the most part, the bias distributions appear comparable to each
other with differences in the ranges of the biases.
Convective velocity scale (Figure A-37) distributions show comparable distributions
between Herculaneum and CPS as well as the 12 and 36 km MMIF scenarios. The bias
distributions for the MMIF scenarios (compared to Herculaneum) are slightly higher than
the CPS - Herculaneum bias distribution.
Monin-Obukhov length, L (Figure A-38) distributions appear very similar across the
scenarios with bias distributions being similar as well. It is difficult to detect differences,
which could be due to large absolute values for near neutral conditions.
Convective mixing height, Zic (Figure A-39) distributions and bias distributions indicate
that the MMIF mixing heights were higher than the two observed datasets. The MMIF
A-33
-------
bias distributions (compared to Herculaneum) are noticeably higher than the CPS -
Herculaneum bias distribution.
Mechanical mixing height, Zim (Figure A-40) distributions show comparable
distributions and bias distributions among the scenarios.
Potential temperature gradient (d0/dz), (Figure A-41), indicate lower lapse rates for the
two observed datasets when compared to the MMIF scenarios. The bias distributions for
the MMIF scenarios (compared to Herculaneum) show a more positive bias when
compared to the CPS - Herculaneum bias distribution.
Cloud cover (Figure A-42) distributions show no differences between CPS and
Herculaneum since CPS cloud cover is used at Herculaneum. Cloud cover estimates for
MMIF are calculated in AERMET. Differences between the MMIF and Herculaneum
show a large swing in biases with a tendency to positive bias.
For the most part, the meteorological data processed through MMIF for Herculaneum appear
reasonable when compared to Herculaneum and CPS observations.
A-34
-------
10
0)
Q.
CO
R
q
Herculaneum CPS MMIF4km MMIF12km MMIF 36 km
Scenario
co"
/ ,
.# *
Scenario
Figure 29. Herculaneum wind speed (m/s): a) annual distributions and b) bias
distributions.
A-35
-------
310-
Herculaneum CPS
10
Q.
I
-10-
MMIF 4 km
Scenario
MMIF12km MMIF 36 km
Scenario
Figure A-30. Herculaneum ambient temperature (K): a) annual distributions and b) bias
distributions.
A-36
-------
1030-
1020-
1010-
o>
"1000-
990
980-
970-
15
10
-5
-10
Herculaneum CPS MMIF4km MMIF12km MMIF 36 km
Scenario
Scenario
Figure A-31. Herculaneum station pressure (mb): a) annual distributions and b) bias
distributions.
A-37
-------
100
75-
50-
25
30
-30-
-60-
Herculaneum CPS
MMIF 4 km
Scenario
MMIF 12 km MMIF 36 km
ป* /
s
Scenario
Figure A-32. Herculaneum relative humidity (percent): a) annual distributions and b) bias
distributions.
A-38
-------
08
0.6-
0.4
0.2-
Herculaneum CPS
0.4
0.2
0.0
-0.2
MMIF 4 km
Scenario
MMIF 12 km MMIF 36 km
ill
^^"^^^^"^^ ^^^^"^^^^n
Scenario
Figure A-33. Herculaneum daytime albedo (fraction): a) annual distributions and b) bias
distributions.
A-39
-------
10.0
7.5
5-0
2,5
o.o-
10
5
Cฃ
ฃ 0
i
-5-
-10-
Herculaneum CPS
MMIF 4 km MMIF 12 km MMIF 36 km
Scenario
b
Scenario
Figure A-34. Herculaneum Bowen ratio: a) annual distributions and b) bias distributions.
A-40
-------
400-
x 200
I
o-
-200-
500
250-
x 0
"ro
-250-
1
Herculaneum
-ป
CPS
H
1
-500-
/
/\x*
^
^ E
MMIF 4 km
Scenario
3
V
u
a
J
MMIF12km MMIF 36 km
= ^
-^ -<ง
Scenario
=a
_ =
>
*
.
*&<ฃ*
& -3r
j^ jv
j^ j?
ป -&r
^ o
b
^B
*
Figure A-35. Herculaneum heat flux (W/m2): a) annual distributions and b) bias
distributions.
A-41
-------
1.5-
1.0-
0.5
1.0-
0.5-
0.0-
-0.5-
Herculaneum CPS
MMIF4km MMIF12km MMIF 36 km
Scenario
/ / / / /
' "
Scenario
Figure A-36. Herculaneum surface friction velocity, u* (m/s): a) annual distributions and
b) bias distributions.
A-42
-------
Herculaneum CPS MMIF4km MMIF12km MMIF 36 km
Scenario
2-
-1
-2
/ X X >'
X *
^ /
-:'
Scenario
Figure A-37. Herculaneum convective velocity scale, w* (m/s): a) annual distributions and
b) bias distributions.
A-43
-------
5000-
o-
-5000-
Herculaneum CPS
10000
-10000
MMIF4km
Scenario
MMIF 12 km MMIF 36 km
Scenario
Figure A-38. Herculaneum Monin-Obukhov length (m): a) annual distributions and b)
bias distributions.
A-44
-------
3000-
2000-
1000
Herculaneum CPS
MMIF4km MMIF12km MMIF 36 km
Scenario
2000-
1000-
-1000-
y
/ X
Scenario
Figure A-39. Herculaneum convective mixing height (m): a) annual distributions and b)
bias distributions.
A-45
-------
4000
3000-
1000
2000-
-2000-
Herculaneum CPS MMIF4km MMIF12km MMIF 36 km
Scenario
-* 4f
Scenario
Figure A-40. Herculaneum mechanical mixing height (m): a) annual distributions and b)
bias distributions.
A-46
-------
0.04-
0.03-
0.02-
0.01-
Herculaneum CPS
MMIF 4 km
Scenario
MMIF 12 km MMIF 36 km
0.02-
0.00
-0.02
/ / /
Scenario
Figure A-41. Herculaneum potential temperature gradient (K/m) above Zic: a) annual
distributions and b) bias distributions.
A-47
-------
10.0-
7.5-
o
5.0-
2.5-
o.o-
Herculaneum CPS MMIF 4 km MMIF 12 km MMIF 36 km
Scenario
Scenario
Figure A-42. Herculaneum cloud cover (tenths*10): a) annual distributions and b) bias
distributions.
A-48
-------
Appendix B. NWS/AERMET and WRF/MMIF analyses for Region 8 sites.
B.I Introduction
The AERMOD Meteorological Data workgroup selected a number of sites within EPA Region 8
to examine and better understand the issues of utilizing various types of meteorological data and
post-processors for air quality dispersion modeling. The workgroup focused on meteorological
data reported through the National Weather Service (NWS) and processed through AERMET, as
well as meteorological data generated by a the Weather Research and Forecasting (WRF)
prognostic model and processed through the Mesoscale Model Interface Program (MMIF). This
section summarizes the results of an analysis that evaluated the differences among
NWS/AERMET and WRF/MMIF output for various surface meteorological parameters at sites
within EPA Region 8. This work does not comment on the accuracy of the results because of the
lack of observational data (i.e., monitored air quality concentrations) available for model
validation at the selected sites.
The basic purpose of AERMET is to use meteorological measurements to compute certain
boundary layer parameters used to estimate profiles of wind, turbulence and temperature. The
depth of this layer and the dispersion of pollutants within it are influenced on a local scale by
surface characteristics, such as surface roughness, reflectivity (albedo), and the availability of
surface moisture. Surface characteristics in the form of albedo, surface roughness and Bowen
ratio, plus standard meteorological observations (wind speed, wind direction, temperature, and
cloud cover), are input to AERMET. AERMET then calculates the boundary layer parameters,
including the Monin-Obukhov Length (L), surface friction velocity (u*), surface roughness
length (z0), surface heat flux (H), and the convective scaling velocity (w*). AERMET also
provides estimates of the convective and mechanical mixed layer heights, z;c and z;m,
respectively. These parameters are then passed to AERMOD, where similar expressions (in
conjunction with measurements) are used to calculate vertical profiles of wind speed, lateral and
vertical turbulent fluctuations, potential temperature gradient, and potential temperature.
Although AERMOD is capable of estimating meteorological profiles with data from as little as
one measurement height, it will use as much data as the user can provide for defining the vertical
structure of the boundary layer. In addition to the boundary layer parameters, AERMET passes
all measurements of wind, temperature, and turbulence in a form AERMOD needs.
B.2 Methodology
Based on the data available during the time of this study, this work evaluated a total of five
model cases covering year 2010 and 2011 and eight meteorological parameters at six sites with
EPA Region 8. This work selected sites based on proximity to flat terrain, valleys, and mountains
to understand the impacts various types of terrain may have on the meteorological data. Table B-
1 and Figure B-l presents information about the sites selected for this work.
B-l
-------
Table B-l. Meteorological sites analyzed for study.
Site
Lamar, CO
Miles City, MT
Minot, ND
Rapid City, SD
Vernal, UT
Riverton, WY
Latitude
38.07 N
46.43 N
48.26 N
44.05 N
40.44 N
43.06 N
Longitude
102.69 W
105.89 W
101.28 W
103.05 W
109.51 W
108.46 W
NLCD
Codes
090900
050600
083000
101100
081500
081700
Surface Site
Codes
KLAA (03013)
KMLS (24037)
KMOT (24013)
KRAP (24090)
KVEL (94030)
KRIW (24061)
Upper Air Site Codes
(2010/2011)
KDDC (3 1484/17452)
KGGW (3 1986/24880)
KBIS (18650/15494)
KRAP (19280/17035)
KGJT
(1005/29527)
KRIW (1410/29950)
Figure B-l. Map of sites analyzed for study.
The work also focused on meteorological information generated from two meteorological
platforms. The first platform included EPA's preferred meteorological model programs,
comprising of AERSURFACE, AERMINUTE, and AERMET. This platform, referenced as
NWS/AERMET, was used to model calendar 2010 and 2011 at the selected sites. The second
B-2
-------
platform included an alternative meteorological model and post-processor, comprising of the
WRF model and the MMIF post-processor. This platform, referenced as WRF/MMIF, was used
to model calendar 2010 and 2011 using 4 kilometer (km) and 12 km grid resolutions. The details
of the model descriptions and model assumptions for each model case are outlined in Table B-2
for NWS/AERMET and Table B-3 for the WRF/MMIF cases.
Table B-2. Description of NWS/AERMET model cases.
Case Name
2010
NWS/AERMET
2011
NWS/AERMET
Year
2010
2011
Model
Platforms
AERSURFACE
v!3016
AERMINUTE
V14237
AERMET
V14134
AERSURFACE
V13016
AERMINUTE
v!4237
AERMET
v!4134
Model Assumptions
Datum: NAD83
Study radius for surface roughness: 1
Vary By Sector: Yes
Number of Sectors: 12
Temporal Resolution: Monthly
Snow Cover: Yes
Months to Seasons: No
Airport: Yes
Arid:N
Surface Moisture: Average
Default
Default
Datum: NAD83
Study radius for surface roughness: 1
Vary By Sector: Yes
Number of Sectors: 12
Temporal Resolution: Monthly
Snow Cover: Yes
Months to Seasons: No
Airport: Yes
Arid:N
Surface Moisture: Average
Default
Default
Input Data
USGS NLCD92 Data
ASOS 1 -minute Data
[6405 Datasets]
NWS Hourly Surface
Data
[ISHD Format]
NWS Hourly Upper
Air Data
[FSL Format]
USGS NLCD92 Data
ASOS 1 -minute Data
[6405 Datasets]
NWS Hourly Surface
Data
[ISHD Format]
NWS Hourly Upper
Air Data
[FSL Format]
B-3
-------
Table B-3. Description of WRF/MMIF model cases.
Case Name
20104-km
WRF/MMIF
2010 12-km
WRF/MMIF
2011 12-km
WRF/MMIF
Year
2010
2010
2011
Model
Platforms
WRF
vS.5.1
MMIF
v3.0
WRF
vS.5.1
MMIF
v3.0
WRF
vS.3.1
MMIF
v3.0
Model Assumptions
Resolution: 4km
Configuration: Details included in
WestJumpAQMS WRF Evaluation
Report4
Resolution: 12km
Configuration: Details included in
WestJumpAQMS WRF Evaluation
Report6
Resolution: 12km
Configuration: Details included in
EPA's Evaluation Report8
Input Data
Details included in
WestJumpAQMS
WRF Evaluation
Report5
Details included in
WestJumpAQMS
WRF Evaluation
Report7
Details included in
WestJumpAQMS
WRF Evaluation
Report9
Using these two platforms, this study analyzed eight meteorological surface parameters,
including temperature, relative humidity, wind speed, wind displacement, mechanical mixing
height, convective mixing height, surface friction velocity, and convective velocity. A number of
statistical metrics and graphical displays were also generated and reviewed to evaluate the
differences among the platforms at the selected sites for the various meteorological parameters
and model years. The statistical metrics included hourly, monthly or annual averages, bias,
fractional bias, error, fractional error, and coefficient of determination. Time series, scatter and
box plots were also generated as part of the graphical displays. This work only includes a
summary of the statistical results. However, all of the graphical displays and statistical analyses
can be provided upon request.
4 Western Regional Air Partnership (WRAP) West-wide Jump Start Air Quality Modeling Study (WestJumpAQMS)
WRF Application/Evaluation, UNC, ENVIRON, Alpine Geophysics, February 29, 2012,
http://www.wrapair2.org/pdf/WestJumpAQMS_2008_Annual_WRF_Final_Report_February29_2012.pdf
5 Ibid.
6 Ibid.
7 Ibid.
8 XXX
9 XXX
B-4
-------
Given that the NWS/AERMET platform may be more representative of the meteorological
conditions at the selected sites, the statistical analysis were based on comparing the model cases
that used the WRF/MMIF platform relative to the model cases that used the NWS/AERMET
platform. For instance, the statistics were based on comparing the 2010 NWS/AERMET case to
the 2010 4-km WRF/MMIF case or the 2010 NWS/AERMET case to the 2010 12-km
WRF/MMIF case. Some of the analyses or graphical displays also compared the 2010 4-km
WRF/MMIF case to the 2010 12-km WRF/MMIF case. Although the WRF model utilizes
observational datasets, the observations used for this study were prescribed to the associated grid
box with a resolution of 4 km or 12 km, which may not be representative of the actual conditions
that cover the entire grid box or at the selected sites. Therefore, the meteorological information
generated by the NWS/AERMET platform may be more representative at the selected sites than
the WRF/MMIF platform.
B.3 Results/Summary
A summary of the results by meteorological parameter and the significant differences among the
model cases are outlined below. In general, careful evaluation and consideration is recommended
when selecting a meteorological model platform because of the range and variability observed in
the results of this analysis. Although additional evaluation is needed for model validation, the
results from the WRF/MMIF platform for temperature, relative humidity, and wind speed were
found to be similar to the NWS/AERMET platform. However, mechanical and convective
mixing heights and surface friction and convective velocities varied significantly among the
platforms, with notable differences in magnitudes, temporal variability, and low correlations.
Generally, the WRF/MMIF platform predicted significantly higher mechanical and convective
mixing heights and convective velocities and lower surface friction velocities relative to the
NWS/AERMET platform. The results of the wind displacement analysis also suggest that the
WRF/MMIF 4 km model case could potentially displace the plume significantly, relative to the
NWS/AERMET platform. These results raise concerns on the adequacy of the WRF/MMIF
platform for generating meteorological information for the dispersion models and whether the
information would generate conservative results.
This work also found that the 4 km and 12 km WRF/MMIF model cases compared well to one
another, with correlations greater than 0.70 for all meteorological, except wind speed at the
Lamar, CO, Vernal, UT, and Riverton, WY sites and surface friction velocity at the Vernal, UT
and Riverton, WY sites. These results suggests systematic difference among the WRF/MMIF
and NWS/AERMET platforms. Without additional evaluation, this work did not find that the
WRF/MMIF 4 km case out-performed the 12 km case relative to the NWS/AERMET platform.
B.3.1 Temperature
Table B-4 shows statistics for monthly average temperatures. The findings for temperature are:
The WRF/MMIF results for both model years and all sites are similar to the
NWS/AERMET results. In general, the WRF/MMIF platforms slightly over-estimated
the values relative to the NWS/AERMET platform at all sites, except at the Minot, ND
and Riverton, WY sites. Although minor, this work found that the most significant
B-5
-------
differences among the platforms were observed at the Lamar, CO, Vernal, UT and
Riverton, WY sites during January and February.
Relative to NWS/AERMET, the WRF/MMIF results show similar temporal variability
for both model years and at all sites.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.98. This work did not find that the WRF/MMIF 4 km case out-performed
the 12 km case relative to the NWS/AERMET platform.
Table B-4. Monthly averaged temperature (K) across all modeled cases at each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
267.45
301.41
285.93
263.54
297.35
280.54
258.97
295.48
278.48
266.64
297.69
281.40
261.15
297.30
281.17
262.49
296.40
279.94
Bias
-0.61
3.40
1.43
-0.73
2.24
0.65
-2.71
0.99
-0.73
-0.35
2.39
0.88
-1.17
5.70
0.98
-3.65
1.29
-0.07
Fractional
Bias
-0.20
1.27
0.52
-0.26
0.78
0.23
-1.05
0.35
-0.27
-0.12
0.81
0.31
-0.41
2.17
0.37
-1.39
0.45
-0.03
Error
1.67
4.40
2.48
1.16
3.88
1.93
1.01
3.33
1.82
1.24
3.22
1.96
1.33
5.87
2.30
1.27
4.07
1.97
Fractional
Error
0.56
1.63
0.88
0.41
1.49
0.70
0.35
1.29
0.66
0.44
1.22
0.70
0.45
2.22
0.83
0.43
1.54
0.71
R2
0.76
0.94
0.88
0.72
0.95
0.88
0.51
0.95
0.86
0.24
0.97
0.82
0.24
0.94
0.80
0.61
0.93
0.83
B. 3.2 Relative humidity
Table B-5 shows statistics for monthly average relative humidity. Findings are:
The WRF/MMIF results for both model years and all sites are similar to the
NWS/AERMET results. In general, the WRF/MMIF platforms slightly over-estimated
the values relative to the NWS/AERMET platform at the Miles City, MT, Minot, ND,
and Riverton, WY sites, and slightly under-estimated the values at the Lamar, CO, Rapid
City, SD, and Vernal, UT sites. Although minor, this work found that the most significant
differences between the platforms were observed at the Miles City, Montana site during
the summer and the Riverton, Wyoming site during January and February.
B-6
-------
Relative to NWS/AERMET, the WRF/MMIF results show similar temporal variability
for both model years and at all sites.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.86. This work found that the WRF/MMIF 4 km case compared better to
NWS/AERMET platform than the WRF/MMIF 12 km case at all sites.
Table B-5. Monthly averaged relative humidity across all modeled cases at each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
39.41
81.49
53.25
39.92
92.10
65.58
55.54
92.44
71.98
41.94
89.09
65.51
26.57
85.06
54.70
33.20
93.77
55.96
Bias
-12.13
7.37
-1.43
-16.08
14.35
0.88
-5.88
15.53
5.72
-14.78
13.54
-0.44
-16.09
8.25
-1.54
-3.85
20.10
5.40
Fractional
Bias
-23.03
19.72
-0.36
-27.27
17.19
-0.54
-10.53
20.93
7.28
-26.91
17.18
-1.22
-29.53
22.66
-1.41
-8.23
25.03
9.03
Error
6.02
13.23
9.50
6.90
16.48
10.61
6.28
17.84
10.02
7.15
15.40
10.35
7.35
17.05
10.20
6.42
20.35
11.29
Fractional
Error
11.96
32.68
20.61
8.94
28.04
17.60
9.89
25.37
14.62
11.42
28.27
17.40
10.60
32.55
20.93
15.88
27.53
21.31
R2
0.51
0.88
0.74
0.10
0.84
0.60
0.26
0.84
0.62
0.03
0.88
0.55
0.03
0.79
0.53
0.08
0.80
0.56
B.3.3 Wind speed
Table B-6 shows statistics for monthly average wind speed. Findings are:
The WRF/MMIF results for both model years and all sites are similar to the
NWS/AERMET results. In general, the WRF/MMIF platforms slightly under-estimated
the values relative to the NWS/AERMET platform at all sites, except at the Vernal, UT
site. Although minor, this work found that the most significant differences between the
platforms were observed at the Vernal, UT site.
Relative to NWS/AERMET, the WRF/MMIF results show similar temporal variability
for both model years and at all sites.
The WRF/MMIF 4 km and 12 km are comparable to one another at the Miles City, MT,
Minot, ND, and Rapid City, SD sites, with correlations greater than 0.70. Generally, the
B-7
-------
WRF/MMIF 12 km case compared better to NWS/AERMET platform than the
WRF/MMIF 4 km case at all sites except at the Vernal, UT site.
Table B-6. Monthly averaged wind speed (m/s) across all modeled cases at each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
2.75
6.07
4.09
2.61
5.88
4.03
3.52
6.17
4.58
3.21
5.77
4.39
1.35
4.04
2.75
1.59
5.39
3.25
Bias
-1.44
-0.12
-0.75
-1.29
-0.17
-0.66
-1.36
-0.14
-0.45
-1.04
0.29
-0.47
-0.57
1.38
0.46
-1.59
0.30
-0.99
Fractional
Bias
-32.47
3.83
-15.23
-30.36
-5.49
-16.70
-26.36
0.77
-6.98
-22.30
14.60
-3.24
-18.96
55.55
15.58
-40.45
11.52
-27.16
Error
0.94
2.06
1.44
0.98
1.69
1.28
0.87
1.62
1.07
1.21
1.92
1.49
0.80
1.72
1.30
0.96
2.20
1.59
Fractional
Error
33.80
49.37
39.25
23.99
48.03
36.80
21.93
39.41
27.89
33.42
50.93
39.58
41.04
69.55
51.28
39.88
57.21
49.53
R2
0.24
0.75
0.55
0.30
0.83
0.55
0.55
0.90
0.74
0.02
0.90
0.34
0.02
0.71
0.26
0.04
0.59
0.35
B.3.4 Wind displacement
Wind displacement, as defined in Section 2.2, is shown in Table B-7 for each site. Findings are:
The calculation of wind displacement is important for determining how far a plume may
be displaced relative to its actual location. Given that this work is assuming that the
NWS/AERMET platform is more representative that the WRF/MMIF platform, this
means that the wind displacement results will determine how far the WRF/MMIF
platform will potentially displace the plume relative to the NWS/AERMET platform. The
wind displacement results can also determine whether the resolution of the prognostic
model could be sufficient for capturing the plume. For instance, when the wind
displacement is within 4 km, these results could suggest that the WRF/MMIF 4 km
model case could potentially capture the plume within that selected grid box. However,
careful consideration of the adequacy of the WRF/MMIF 4 km model platform should be
considered when the wind displacement is greater than 4 km.
B-8
-------
The results of the wind displacement analysis suggests that the WRF/MMIF 12 km model
case could potentially capture the plume given the resolution and distance of less than 12
km. However, careful consideration is needed when determining the adequacy of the
WRF/MMIF 4 km model case because the distances are more than 4 km. Further, the
most notable differences were found at the Vernal, UT site.
Table B-7. Monthly averaged wind displacement (km) across all modeled cases at each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Minimum
6.32
5.58
6.32
7.13
5.47
6.60
Maximum
13.13
8.64
11.03
11.33
14.42
10.84
Average
9.23
7.08
9.04
8.11
10.65
9.33
B. 3.5 Mechanical mixing heights
Table B-8 shows statistics for monthly average mechanical mixing heights. The findings for
mechanical heights are:
The WRF/MMIF results for both model years and all sites are not similar to the
NWS/AERMET results, including notable differences in magnitudes, temporal
variability, and low correlations. In general, the WRF/MMIF platforms significantly
over-estimated the values relative to the NWS/AERMET platform at all sites, except at
the Minot, ND site.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.80. This work did not find that the WRF/MMIF 4 km case out-performed
the 12 km case relative to the NWS/AERMET platform.
B-9
-------
Table B-8. Monthly averaged mechanical mixing heights (m) across all modeled cases at
each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
166.76
1125.91
637.47
114.02
974.30
520.02
112.38
1469.21
679.38
201.56
858.03
490.40
45.52
988.12
394.62
31.61
1036.19
539.10
Bias
-234.50
360.06
128.03
-368.02
443.90
24.49
-1168.20
-120.02
-506.86
-261.57
442.28
23.49
12.35
661.08
275.89
-299.73
379.77
7.36
Fractional
Bias
-46.96
60.41
5.69
-90.02
19.53
-24.62
-110.96
-24.69
-66.89
-35.49
31.27
-2.58
-19.08
92.88
24.32
-113.72
-4.74
-45.85
Error
130.18
844.73
482.44
102.72
706.95
384.27
358.86
1200.46
696.03
150.71
646.23
354.88
34.11
761.05
372.65
97.79
836.78
489.74
Fractional
Error
69.77
92.30
78.08
58.59
99.80
77.71
72.09
117.07
95.24
63.89
93.02
74.73
68.10
102.31
83.12
79.51
118.22
98.40
R2
0.03
0.47
0.22
0.06
0.61
0.27
0.06
0.37
0.20
0.03
0.61
0.25
0.16
0.47
0.31
0.05
0.44
0.18
B.3.6 Convective mixing heights
Table B-9 shows statistics for monthly average convective mixing heights. The findings for
convective heights are:
The WRF/MMIF results for both model years and all sites are not similar to the
NWS/AERMET results, including notable differences in magnitudes, temporal
variability, low correlations. In general, the WRF/MMIF platforms significantly over-
estimated the values relative to the NWS/AERMET platform at all sites.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.75. Generally, the WRF/MMIF 12 km case compared slightly better to
NWS/AERMET than the WRF/MMIF 4 km case at all sites except at the Vernal, UT site.
B-10
-------
Table B-9. Monthly averaged mechanical mixing heights (m) across all modeled cases at
each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
121.70
1932.54
947.26
68.00
1576.02
767.75
94.26
1227.90
640.71
132.75
1548.62
781.64
96.43
1981.48
996.73
68.18
1919.11
971.69
Bias
108.28
993.78
510.25
-117.41
619.99
249.92
-22.78
420.21
149.98
-144.11
479.86
139.84
-200.12
493.66
86.06
-294.78
311.17
31.04
Fractional
Bias
15.73
114.41
60.82
-11.38
147.31
43.43
-30.46
118.19
24.62
-7.36
95.05
24.46
-18.94
98.98
21.06
-49.76
75.16
-1.52
Error
228.66
1035.89
605.79
74.57
672.85
363.57
62.00
491.13
271.34
106.70
628.09
341.08
81.84
711.62
425.01
50.88
698.34
401.16
Fractional
Error
39.38
114.41
71.23
34.06
151.82
62.45
30.96
123.38
52.45
31.95
95.68
50.63
30.55
103.24
53.47
29.24
86.45
51.31
R2
0.06
0.71
0.39
0.00
0.71
0.43
0.03
0.74
0.46
0.00
0.76
0.38
0.00
0.65
0.35
0.02
0.76
0.41
B.3.7 Surface friction velocity
Table B-10 shows statistics for monthly average surface friction velocity (u*). The findings for
u* are:
The WRF/MMIF results for both model years and all sites not similar to the
NWS/AERMET results, including notable differences in magnitudes, temporal
variability, and low correlations. In general, the WRF/MMIF platforms significantly
under-estimated the values relative to the NWS/AERMET platform at all sites, except at
the Rapid City, SD and Vernal, UT sites.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.75, at all site except at the Vernal, UT and Riverton, WY sites. This work
did not find that the WRF/MMIF 4 km case out-performed the 12 km case relative to the
NWS/AERMET platform.
B-ll
-------
Table B-10. Monthly averaged surface friction velocity (m/s) across all modeled cases at
each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
0.15
0.49
0.33
0.16
0.48
0.31
0.16
0.67
0.40
0.20
0.43
0.32
0.06
0.36
0.21
0.05
0.47
0.26
Bias
-0.11
0.12
-0.01
-0.09
0.07
-0.01
-0.31
-0.07
-0.16
-0.02
0.09
0.03
-0.02
0.14
0.05
-0.19
-0.02
-0.10
Fractional
Bias
-29.97
45.34
-2.21
-47.52
23.88
-15.17
-73.39
-10.17
-36.58
-17.23
44.04
9.28
-29.02
74.01
19.17
-80.06
-5.00
-41.32
Error
0.07
0.18
0.13
0.07
0.15
0.12
0.14
0.34
0.20
0.06
0.14
0.11
0.04
0.18
0.11
0.07
0.20
0.15
Fractional
Error
39.07
67.36
48.56
35.23
69.38
49.08
35.89
76.68
53.49
40.28
64.88
48.19
47.41
89.71
62.22
48.41
89.44
64.77
R2
0.20
0.75
0.51
0.34
0.78
0.55
0.26
0.73
0.51
0.01
0.84
0.38
0.12
0.80
0.39
0.01
0.46
0.30
B.3.8 Convective velocity scale
Table B-10 shows statistics for monthly average convective velocity scale (w*). The findings
for w* are:
The WRF/MMIF results for both model years and all sites are not similar to the
NWS/AERMET results, including notable differences in magnitudes, temporal
variability, and low correlations. In general, the WRF/MMIF platforms significantly
over-estimated the values relative to the NWS/AERMET platform at all sites.
The WRF/MMIF 4 km and 12 km are comparable to one another, with correlations
greater than 0.85. This work did not find that the WRF/MMIF 4 km case out-performed
the 12 km case relative to the NWS/AERMET platform.
B-12
-------
Table B-ll. Monthly averaged convective velocity scale (m/s) across all modeled cases at
each site.
Site
LAA
MLS
MOT
RAP
VEL
RIW
Temperature
(K)
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Minimum
Maximum
Average
Mean
0.32
2.83
1.50
0.18
2.68
1.27
0.19
2.23
1.09
0.29
2.49
1.29
0.25
2.90
1.46
0.25
2.92
1.41
Bias
-0.02
1.91
0.88
-0.46
1.58
0.67
-0.13
1.25
0.47
-0.45
1.22
0.57
-0.32
1.11
0.50
-0.25
1.16
0.37
Fractional
Bias
-5.37
111.72
57.65
-35.61
114.61
53.13
-13.22
99.65
40.50
-34.27
86.88
43.62
-25.71
99.42
35.60
-36.53
64.27
19.37
Error
0.25
1.92
0.95
0.33
1.58
0.78
0.25
1.29
0.58
0.38
1.24
0.73
0.27
1.17
0.66
0.16
1.20
0.58
Fractional
Error
21.82
111.72
64.59
24.65
117.97
65.53
26.07
100.92
56.00
29.55
98.56
60.05
25.65
108.27
50.93
23.05
84.81
44.81
R2
0.07
0.70
0.42
0.00
0.61
0.31
0.00
0.64
0.34
0.00
0.74
0.41
0.00
0.67
0.39
0.00
0.74
0.43
B-13
-------
-------
United States Office of Air Quality Planning and Standards Publication No. EPA-454/R-15-004
Environmental Protection Air Quality Assessment Division July, 2015
Agency Research Triangle Park, NC
------- |