f/EPA
I*
United States
Environmental Protection
Agency
Environment
Canada
Environnement
Canada
Office of Research and
Development
Research Triangle Park NC 27711
Atmospheric Environment
Service
Downsview, Ontario M3H 5T4
EPA 600 3-87 008
May 1987
Service de I'environnement
atmosphenque
Downsview, Ontario M3H 5T4
International
Sulfur
Deposition
Model
Evaluation
-------
EPA Publication No. EPA/600/3-87/008
AES Publication No. ARD-87-1
INTERNATIONAL SULFUR
DEPOSITION MODEL EVALUATION
FINAL REPORT
May 1987
and Robin L. Dennis
Terry L. Clark
Office of Research and Development
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina
Eva C. Voldner and Marvin P. Olson
Atmospheric Environment Service
Environment Canada
Downsview, Ontario
Steve K. Seilkop
Analytical Sciences, Inc.
Research Triangle Park, North Carolina
Mayer Alvo
Department of Mathematics
University of Ottawa
Ottawa, Ontario
* Also affiliated with the U.S. Department of Commerce, National Oceanic and
Atmospheric Administration.
** Also affiliated with the Systems and Informatics Directorate, Environment Canada.
... - ,..,., i-,__j Pmt option Agency
L'.-urbo a St-ed
, IL -Uoi1 *
70
-------
Notice
The research described in this document has been funded in part by the United
States Environmental Protection Agency. It has been subject to the Agency's peer and
administration review and has been approved for publication as an EPA document.
Cover Illustration
The kriged analysis of 1980 annual amounts of sulfur wet deposition (kg S/ha/yr).
11
-------
ABSTRACT
The International Sulfur Deposition Model Evaluation (ISDME) project, jointly
conducted by the U.S. Environmental Protection Agency and Atmospheric
Environment Service of Environment Canada, assessed the performance of eleven
linear chemistry atmospheric models in predicting amounts ot sulfur wet deposition.
Standardized model input data sets were distributed to the participating modelers, who
later submitted seasonal and annual 1980 model predictions of dry/wet deposition and
air concentrations of sulfur dioxide and sulfate at up to 65 sites across eastern North
America.
The models were evaluated in an operational mode using new, more rigorous
approaches, as well as the more conventional distribution statistics recommended by
the American Meteorological Society. The new approaches focused on the ability of
the models to replicate features of the spatial patterns of sulfur wet deposition, as
determined by an interpolation technique known as kriging. This technique quantified
the uncertainties in the observations which were used in the evaluation process to
identify areas where interpolated predictions were statistically significantly different
from the interpolated observations. To supplement the evaluation, predictions of dry
deposition amounts and air concentrations of each model were intercompared to
identify apparent peculiarities.
Finally, a scoring system based on criteria for six model performance measures was
devised to compare seasonal, annual and overall performances of the models. Three
clusters of models, each with similar overall scores, were identified.
in
-------
CONTENTS
Page No.
ABSTRACT
FIGURES
TABLES
ACKNOWLEDGMENTS
EXECUTIVE SUMMARY
1. INTRODUCTION
1.1 Background
1.2 ISDME Goals
1.3 ISDME Participants
1.4 Structure of the Report
2. MODEL DESCRIPTIONS
3. ISDME DATA SETS
3.1 Standardized Model Input Data
3.1.1 General Considerations
3.1.2 Transport Winds
3.1.3 Precipitation Amounts
3.1.4 Climatological Mixing Heights
3.1.5 Anthropogenic Sulfur Dioxide Emissions Data
3.1.5.1 Canadian Data
3.1.5.2 United States Data
3.1.5.3 Aggregation Methods
3.2 Sulfur Wet Deposition Data
3.2.1 Data Screening and Calculation Procedures
3.2.2 Results of Data Screening and Calculations
3.2.3 Differences in Amounts at Nearby Sites
3.3 Air Concentration Data
in
ix
xv
xviii
ixx
1
1
2
4
6
21
21
21
22
22
25
25
26
27
28
32
32
35
40
44
-------
Page No.
4. MODEL EVALUATION APPROACH 47
4.1 Introduction 47
4.2 Elements of the Model Evaluation 48
4.3 Model Performance Measures 50
4.3.1 Basis for Selection 50
4.3.2 Description of Measures 53
4.3.2.1 Spatial Patterns 53
4.3.2.2 Maxima 58
4.3.2.3 Temporal Patterns 60
4.4 Model Clustering 60
5. SPATIAL INTERPOLATION 65
5.1 Introduction 65
5.2 Selection of an Interpolation Method 66
5.3 The Use of Kriging in the Model Evaluation 70
6. MODEL EVALUATION RESULTS 75
6.1 Introduction 75
6.2 Seasonal Temporal Patterns 76
6.3 Seasonal Model Evaluation Results 78
6.3.1 Summer 1980 Results 78
6.3.1.1 Summary of Distributional Statistics 78
6.3.1.2 Comparison of Predictions and APN Measurements 82
6.3.1.3 Spatial Analyses 82
6.3.1.4 Model Performance Measures 92
VI
-------
Page No.
6.3.2 Spring 1980 Results 98
6.3.2.1 Summary of Distributional Statistics 98
6.3.2.2 Comparison of Predictions and APN Measurements 98
6.3.2.3 Spatial Analyses 102
6.3.2.4 Model Performance Measures 102
6.3.3 Autumn 1980 Results 114
6.3.3.1 Summary of Distributional Statistics 114
6.3.3.2 Comparison of Predictions and APN Measurements 117
6.3.3.3 Spatial Analyses 117
6.3.3.4 Model Performance Measures 128
6.3.4 Winter 1980 Results 133
6.3.4.1 Summary of Distributional Statistics 133
6.3.4.2 Comparison of Predictions and APN Measurements 133
6.3.4.3 Spatial Analyses 137
6.3.4.4 Model Performance Measures 145
6.4 Consistencies of Seasonal and Annual Results 151
7. MODEL CLUSTERING 153
7.1 Introduction 153
7.2 Overall Model Performance 154
7.2.1 Group A Models 155
7.2.2 Group B Models 158
7.2.2 Group C Models 159
7.3 Annual Model Performance 161
8. SUMMARY AND CONCLUSIONS 163
8.1 The ISDME Approach 163
8.2 Comparison of Predictions and Observations 164
VII
-------
Page No.
8.3 Model Performance Scores 166
8.4 Concluding Remarks 169
9. RECOMMENDATIONS FOR FUTURE RESEARCH 171
REFERENCES
APPENDICES
A SUMMARY OF PREVIOUS EVALUATIONS OF SULFUR
DEPOSITION MODELS A-l
B STANDARDIZED MODEL INPUT DATA (FIGURES AND
TABLES) B-l
C SCREENING AND CALCULATION PROCEDURES FOR
THE MODEL EVALUATION DATA SET C-1
D COMPARISON OF PREDICTIONS AND OBSERVATIONS
OF SULFUR WET DEPOSITION AT THE ISDME SITES D-l
E ANNUAL 1980 MODEL PERFORMANCES E-l
vm
-------
LIST OF FIGURES
Figure No. Title Page No.
3.1 The three layers into which sulfur dioxide emissions from
point and area sources are injected. 29
3.2 The 1980 annual ISDME grids of sulfur dioxide emissions
(tonnes/year) for the UMACID configuration from area sources
(a) and point sources in the lowest (b), middle (c), and
highest (d) layers. 30
3.3 Locations and levels of regional representativeness of the
precipitation chemistry sites operating in 1980 and passing at
least the Level 3 criteria. 35
3.4 Winter 1980 amounts of sulfur wet deposition (kg S/ha) at the
ISDME evaluation sites. 37
3.5 Spring 1980 amounts of sulfur wet deposition (kg S/ha) at the
ISDME evaluation sites. 37
3.6 Summer 1980 amounts of sulfur wet deposition (kg S/ha) at the
ISDME evaluation sites. 38
3.7 Autumn 1980 amounts of sulfur wet deposition (kg S/ha) at the
ISDME evaluation sites. 38
3.8 Annual 1980 amounts of sulfur wet deposition (kg S/ha) at the
ISDME evaluation sites. 39
3.9 The location and identification codes of the ISDME sites. 40
4.1 The four subregions where significant differences between
interpolated predictions and observations are identified. 53
4.2 The ISDME sites in the high deposition region as denoted by
the circles. 55
5.1 Comparison of the interpolated patterns of annual 1980
amounts of sulfur wet deposition (kg S/ha/yr) using a kriging
technique and a standard least squares regression technique. 67
5.2 The uncertainty estimates (%) of the kriged values of annual
1980 observations of sulfur wet deposition. 71
IX
-------
LIST OF FIGURES
Figure No. Title Page No.
5.3 The histograms of kriged residuals for the summer and annual
1980 amounts of sulfur wet deposition (kg S/ha). 73
6.1 The mean percentage of annual 1980 sulfur wet deposition
observed and predicted for each season at 30ISDME sites. 77
6.2 Summer 1980 box plots of the predictions and observations of
sulfur wet deposition (kg S/ha). 79
6.3 Summer 1980 box plots of the predictions of sulfate air
concentrations (u,g/m3). 79
6.4 Summer 1980 box plots of the predictions of sulfur dioxide air
concentrations (u,g/m3). 80
6.5 Summer 1980 box plots of the predictions of sulfur dry
deposition (kg S/ha). 80
6.6 Comparison of the summer 1980 observations and predictions of
the air concentrations (ug/m3) of sulfur dioxide and sulfate at
the APN sites. 83
6.7 Summer 1980 kriged analyses of the predictions and
observations of sulfur wet deposition (kg S/ha). 84
6.8 Areas of significant under/oyerpredictions of summer 1980
amounts of sulfur wet deposition. 89
6.9 Summer 1980 locations of the centroids. 95
6.10 Summer 1980 locations and magnitudes (kg S/ha) of the
maximum amount of sulfur wet deposition at the ISDME sites. 97
6.11 Spring 1980 box plots of the predictions and observations of
sulfur wet deposition (kg S/hu). 9()
6.12 Spring 1980 box plots of the predictions of sulfate air
concentrations (ug/m3). 99
6.13 Spring 1980 box plots of the predictions of sulfur dioxide
air concentrations (pg/m3). 100
-------
LIST OF FIGURES
Figure No. Title Page No.
6.14
6.15
6.16
6.17
6.18
6.19
6.20
6.21
6.22
6.23
6.24
6.25
6.26
6.27
Spring 1980 box plots of the predictions of sulfur dry
deposition (kg S/ha).
Comparison of the spring 1980 observations and predictions
of the air concentrations (u.g/m3) of sulfur dioxide and sulfate
at the APN sites.
Spring 1980 kriged analyses of the predictions and
observations of sulfur wet deposition (kg S/ha).
Areas of significant under/overpredictions of spring 1980
amounts of sulfur wet deposition.
Spring 1980 locations of the centroids.
Spring 1980 locations and magnitudes (kg S/ha) of the
maximum amount of sulfur wet deposition at the ISDMIi sites.
Autumn 1980 box plots of the predictions and observations of
sulfur wet deposition (kg S/ha).
Autumn 1980 box plots of the predictions of sulfate air
concentrations (u,g/m3).
Autumn 1980 box plots of the predictions of sulfur dioxide
air concentrations (u,g/m3).
Autumn 1980 box plots of the predictions of sulfur dry
deposition (kg S/ha).
Comparison of the autumn 1980 observations and predictions
of the air concentrations (u,g/m3) of sulfur dioxide and sulfate
at the APN sites.
Autumn 1980 kriged analyses of the predictions and
observations of sulfur wet deposition (kg S/ha).
Areas of significant under/overpredictions of autumn 1980
amounts or sulfur wet deposition.
Autumn 1980 locations of the centroids.
100
101
103
107
112
113
115
115
116
116
118
119
125
131
XI
-------
LIST OF FIGURES
Figure No. Title Page No.
6.28 Autumn 1980 locations and magnitudes (kg S/ha) of the
maximum amount of sulfur wet deposition at the ISDME sites. 132
6.29 Winter 1980 box plots of the predictions and observations of
sulfur wet deposition (kg S/ha). 134
6.30 Winter 1980 box plots of the predictions of sulfate air
concentrations (ng/m3). 134
6.31 Winter 1980 box plots of the predictions of sulfur dioxide
air concentrations (ng/m3). 135
6.32 Winter 1980 box plots of the predictions of sulfur dry
deposition (kg S/ha). 135
6.33 Comparison of the winter 1980 observations and predictions
of the air concentrations (u,g/m3) of sulfur dioxide and sulfate
at the APN sites. 136
6.34 Winter 1980 kriged analyses of the predictions and
observations of sulfur wet deposition (kg S/ha). 138
6.35 Areas of significant under/oyerpredictions of the winter 1980
amounts of sulfur wet deposition. 142
6.36 Winter 1980 locations of the centroids. 148
6.37 Seasonal variation of the orientation of the major axes of
the annual 1980 predictions and observations of sulfur wet
deposition. 149
6.38 Winter 1980 locations and magnitudes (kg S/ha) of the
maximum amount of sulfur wet deposition at the ISDME sites. 150
7.1 The distribution of the seasonal, overall and annual 1980
model performance scores. 157
B.I The four model grid configurations for which model input
data have been processed. B-2
B.2 Annual 1980 patterns of precipitation amounts (cm) using
a) CMC gridded data and b) data from precipitation chemistry
monitoring sites. B-3
xii
-------
LIST OF FIGURES
Figure No. Title Page No.
B.3 Winter 1980 patterns of precipitation amounts (cm) using
a) CMC gridded data and b) data from precipitation chemistry
monitoring sites. B-4
B.4 Spring 1980 patterns of precipitation amounts (cm) using
a) CMC gridded data and b) data from precipitation chemistry
monitoring sites. B-5
B.5 Summer 1980 patterns of precipitation amounts (cm) using
a) CMC gridded data and b) data from precipitation chemistry
monitoring sites. B-6
B.6 Autumn 1980 patterns of precipitation amounts (cm) using
a) CMC gridded data and b) data from precipitation chemistry
monitoring sites. B-7
B.7 Histograms of 1980 annual precipitation amounts (cm) a) for
127-km, CMC grid cells containing precipitation chemistry
monitoring sites and b) at the 72 precipitation chemistry
monitoring sites across eastern North America. B-8
B.8 A comparison of the weekly time series of precipitation amounts
(mm) measured at 18 precipitation chemistry monitoring sites
(validation data) and calculated for respective 127-km grid cells
by the CMC model. B-0
E.I Annual 1980 box plots of the predictions and observations of
sulfur wet deposition (kg S/ha). E-2
E.2 Annual 1980 box plots of the predictions of sulfate air
concentrations (u,g/m3). E-2
E.3 Annual 1980 box plots of the predictions of sulfur dioxide air
concentrations (u,g/m3). E-3
E.4 Annual 1980 box plots of the predictions of sulfur dry
deposition (kg S/ha). E-3
E.5 Annual 1980 kriged analyses of the predictions and
observations of sulfur wet deposition (kg S/ha). E-4
E.6 Areas of significant under/oyerpredictions of annual 1980
amounts of sulfur wet deposition. E-8
xiii
-------
LIST OF FIGURES
Figure No. Title Page No.
E.7 Annual 1980 locations of centroids. E-l 1
E.8 Annual 1980 locations and magnitudes (kg S/ha) of the
maximum amount of sulfur wet deposition at the TSDMR sites. F.-11
E.9 Comparison of the annual 1980 observations and predictions
of the air concentrations (u.g/m3) of sulfur dioxide and sulfate
at the APN sites. E-12
XIV
-------
LIST OF TABLES
Table No. Title Page No.
1.1 The models and participants in the ISDME. 5
2.1 Attributes of the statistical models as applied in the ISDME. 9
2.2 Attributes of the deterministic models as applied in the
ISDME. 13
3.1 The effects on the 1980 AES predictions of sulfur wet
deposition as a consequence of using precipitation data from
the precipitation chemistry sites as opposed to the grids. 25
3.2 The UDDBC and ISDME overall data quality levels. 34
3.3 The distribution of sites according to site representativeness
level. 36
3.4 The number of sites in the 1980 ISDME evaluation data set for
each season and the year. 39
3.5 The identification codes and coordinates of the ISDME sites. 41
3.6 Mean air concentrations of sulfur dioxide and sulfate at the
four APN sites during 1980. 45
4.1 Elements of the evaluation of the ISOME models. 49
4.2 The ISDME model performance measures. 52
4.3 The criteria values of the primary model performance measures
used to cluster the models. 63
5.1 Comparison of kriged variances for 1980 annual amounts of
observed and predicted sulfur wet deposition. 74
6.1 Summary of the predictions provided by each model. 75
6.2 Types of patterns of significant differences for summer 1980. 92
6.3 Summary of model performance measures for summer 1980. 93
6.4 Types of patterns of significant differences for spring 1980. 110
6.5 Summary of model performance measures for spring 1980. 111
xv
-------
LIST OF TABLES
Table No. Title Page No.
6.6 Comparison of the geographical regions where each model
predicts the highest amounts of sulfur wet deposition for
summer/spring versus autumn 1980. 124
6.7 Types of patterns of significant differences for autumn 1980. 128
6.8 Summary of model performance measures for autumn 1980. 129
6.9 Types of patterns of significant differences for winter 1980. 145
6.10 Summary of model performance measures for winter 1980. 146
7.1 The seasonal, overall and annual model performance scores. 156
A.1 Summary of previous evaluations of regional deposition models. A- 2
B.I The annual 1980 emission rates of sulfur dioxide (ktonnes/yr)
and precision estimates for Canada. B-l 1
B.2 The annual 1980 emission rates of sulfur dioxide (ktonnes/yr)
for the United States. B-l2
B.3 The distribution of sulfur dioxide emissions from United States
and Canadian point sources by physical stack height. B-13
C.I Criteria for unrepresentative monitoring sites. C-5
C.2 Data completeness rating criteria for seasonal sulfate values. C- 8
C.3 Data completeness rating criteria for annual sulfate values. C- 3
C.4 Differences in the seasonal and annual 1980 amounts of sulfur
wet deposition (expressed as percent) at colocated or closely
located samplers. C-12
C.5 Differences in the annual amounts of sulfur wet deposition
(expressed as percent) at colocated or closely located
samplers for 1981-1982. C-13
D.I Comparison of the observations and predictions of annual 1980
sulfur wet deposition at the 1SDMH evaluation sites. D- 2
D.2 Comparison of the observations and predictions of winter 1980
sulfur wet deposition at the ISDME evaluation sites. D- 4
xvi
-------
LIST OF TABLES
Table No. Title Page No.
D.3 Comparison of the observations and predictions of spring 1980
sulfur wet deposition (kg S/ha) at the ISDME evaluation sites. D- 6
D.4 Comparison of the observations and predictions of summer 1980
sulfur wet deposition (kg S/ha) at the ISDME evaluation sites. D- 8
D.5 Comparison of the observations and predictions of autumn 1980
sulfur wet deposition (kg S/ha) at the ISDME evaluation sites. D-10
D.6 Standardized residuals of annual 1980 sulfur wet deposition
at the ISDME evaluation sites. D-12
D.7 Standardized residuals of winter 1980 sulfur wet deposition
at the ISDME evaluation sites. D-14
D.8 Standardized residuals of spring 1980 sulfur wet deposition
at the ISDME evaluation sites. D-16
D.9 Standardized residuals of summer 1980 sulfur wet deposition
at the ISDME evaluation sites. D-18
D.10 Standardized residuals of autumn 1980 sulfur wet deposition
at the ISDME evaluation sites. D-20
D.I 1 Statistical comparison of the observations and predictions of
annual 1980 sulfur wet deposition. D-22
D.12 Statistical comparison of the observations and predictions of
winter 1980 sulfur wet deposition. D-23
D.13 Statistical comparison of the observations and predictions of
spring 1980 sulfur wet deposition. D-24
D. 14 Statistical comparison of the observations and predictions of
summer 1980 sulfur wet deposition. D-25
D.15 Statistical comparison of the observations and predictions of
autumn 1980 sulfur wet deposition. D-26
E.I Summary of model performance measures for the annual 1980
period. H-13
xvn
-------
ACKNOWLEDGMENTS
The authors wish to acknowledge the efforts of all of those who contributed to the
International Sulfur Deposition Model Evaluation project, especially the modelers and
their colleagues who have provided model results. Without their assistance, this report
would not nave been possible. The following people have also contributed to this
study:
Ms. Carmen Benkovitz, Ms. Veronica Evans and Mr. Marty Leach, Brookhaven
National Laboratory
Dr. Perry Samson and Mr. Jerry Keeler, University of Michigan
Dr. Maris Lusis, Ontario Ministry of the Environment
Dr. Anthony Olsen, Battelle, Pacific Northwest Laboratory
Mr. Dale Coventry, Mr. Brian Eder, and Dr. Peter Finkelstein, U.S. Environmental
Protection Agency
Ms. Debbie Palka and Ms. Debbie Bieler, Program Resources Incorporated
Dr. Bernard Fisher, Central Electricity Research Laboratories,
United Kingdom
Mr. Dan McNaughton, TRC Environmental Consultants
xvm
-------
EXECUTIVE SUMMARY
Computer models have been developed to estimate mean air concentrations of
sulfur dioxide and sulfate and sulfur dry and wet deposition amounts at receptor points
or across grid cells approximately 100 km on a side. Many of these models have been
applied to estimate source-receptor relationships, particularly for sulfur wet deposition,
and total sulfur deposition across geopolitical entities. In the past, these models could
not be rigorously evaluated for their predictions of sulfur wet deposition, because of
the paucity of sampling data. The increased number of sites in the North American
precipitation chemistry networks after 1979 affords the opportunity to more rigorously
evaluate these models for sulfur wet deposition.
Recognizing the need to evaluate existing North American models, the Office of
Research and Development of the U. S. Environmental Protection Agency and
Atmospheric Environment Service of Environment Canada jointly conducted llic
International Sulfur Deposition Model Evaluation (ISDME) Project. Using unique
methods, this project evaluated eleven linear-chemistry atmospheric models of sulfur
deposition for each season and the year of 1980. These models were either statistical,
Lagrangian or hybrids. The selection of 1980 as the evaluation year was based, in part,
on the availability of the most recent emissions inventory.
The approach of the study was to i) compile and distribute standardized model
input data sets to the modelers, ii) apply the eleven models using these data sets and
iii) evaluate the models in a "blind" test mode, meaning the computer code of these
models was not modified specifically to improve the results for 1980. The evaluation
data consisted of sulfur wet deposition amounts calculated from screened precipitation
chemistry data from the five major North American networks and daily air
concentrations of sulfur dioxide and sulfate at four Canadian sites. Dry deposition
data were not available during this study.
The ISDME screening and calculation procedures were very similar to those
recommended by the Unified Deposition Data Base Committee established by the
Canadian Research and Monitoring Coordinating Committee and the U.S. National
Acid Precipitation Assessment Program. The number of sites with data passing the
ISDME criteria ranged from 32 for the annual to 46 for the spring periods.
Unlike the evaluations of the past, the ISDME focused on the ability of the models
to replicate the spatial patterns of seasonal, as well as annual, amounts of sulfur wet
deposition within the uncertainties of the data. Seasonal and annual evaluations, rather
than only annual evaluations, were conducted to provide a more stringent test of the
models. Patterns were generated by interpolating the site observations and predictions
xix
-------
to half-degree grid cells via a technique known as kriging. This technique minimizes
the interpolation errors and quantifies the uncertainties arising from both the
interpolation and the errors of measurement.
The emphasis on pattern replication is based on the fact that point measurements
are not necessarily representative of the spatial resolution of the models. That is, large
spatial variability of meteorological parameters and air concentrations can cause
"local" effects on the measurements at a site. As a result, these measurements could be
representative of only a point and not the region. Therefore, it is necessary to compare
predictions and observations on commensurate spatial scales. Moreover, pattern
comparisons provide at least a limited evaluation of the models in data sparse regions.
The second unique aspect of the evaluation approach was the consideration of the
uncertainties of the observed amounts of sulfur wet deposition. Rather than
comparing predictions to the best estimates of the observations, this evaluation
determined whether and by how much the predictions were outside the uncertainty
limits of the interpolated observations. Based on these uncertainty limits, the
statistical significance of the differences was determined. These uncertainties arise
from sampling and analytical errors, local or nonregional effects, network protocol
differences, and the interpolations of both the predictions and observations. The
uncertainties of the annual pattern of sulfur wet deposition ranged from approximately
_+. 50% in the region of highest sulfur wet deposition to +_ 100% across data-sparse
regions of eastern North America.
The models were evaluated using six model performance measures, which
quantified the differences between the predicted and observed patterns of sulfur wet
deposition. The measure receiving the most weight was the percentage of the half-
degree cells where the predictions significantly differed from the measurements. The
remaining five measures were assigned equal weights. Two of these measures
quantified differences in pattern position, while two additional measures quantified the
differences in the magnitudes and locations of the maxima. The final measure
quantified differences in the seasonal distribution of the annual amounts.
The comparison of predicted and observed patterns of seasonal sulfur wet
deposition revealed that the predicted patterns tended to resemble concentric ellipses
and to show less detail than the observed patterns. The predicted seasonal and annual
patterns also consistently exhibited maximum amounts within the high emissions
region of Ohio, western Pennsylvania and West Virginia. This behavior was not
evident in the patterns of the observations, which indicated that the location of the
maximum was not anchored, but migrated from northern West Virginia to southern
Ontario.
Depending on the model, distances separating the predicted and observed
locations of the maxima for all seasons were less than 170 to 350 km. Most models
were within the uncertainty limits of the magnitude of the seasonal and annual
maxima.
xx
-------
The determination of the areas of statistically significant differences between the
predictions and observations of sulfur wet deposition revealed that across 80% of the
evaluation region:
* The seasonal predictions of five of the eleven models were within the
uncertainty limits of the observed seasonal patterns
* The seasonal predictions of all but three models were within the same
uncertainty limits for at least three seasons
* The annual predictions of all but three models were within the uncertainty limits
of the observed annual pattern.
A contributing factor to this high percentage was the degree of uncertainty of the
observed pattern of sulfur wet deposition, which was as high as +_ 50% across the area
of greatest site density. Therefore, the models in this study were afforded a rather
large statistical tolerance for differences.
Regarding the seasonal contributions to the 1980 amount of sulfur wet deposition,
nearly 40% and 30% of the annual observed amounts occurred during summer (July,
August and September) and spring (April, May and June), respectively.
Approximately 15% occurred during both winter (January, February and March) and
autumn (October, November and December). Most of the models are within 10% of
the summer and spring contributions and within 25% of the autumn and winter
contributions. Three models did not mimic the observed seasonal variability as well as
the other models.
Two features of model performance were common to the statistical models but not
to the deterministic models. First, the patterns of the sulfur wet deposition, as
determined by the statistical models, tended to be oriented more toward the east-west
axis, as opposed to the southwest-northeast axis of the observed patterns. On the other
hand, the orientation of the patterns produced by the deterministic models tended to
be closer to that of the observed patterns. This distinction could be due to the fact that
the statistical models base their transport simulation on seasonal mean winds, which in
North America tend to be westerly, rather than on wind measurements available twice
a day. The second feature common to statistical models was the tendency to predict
sulfate air concentrations higher than those of the deterministic models at both the
ISDME sites and the four Canadian monitoring sites.
Finally, the seasonal, annual and overall performances of the models were
summarized by scoring and clustering procedures based on subjectively determined
criteria values for each of the six performance measures. A model received points if
the values of the performance measures did not exceed the criteria values. Overall
performance scores were based on weighted seasonal performance scores. The
weights, the sum of which equalled one, were determined by the mean relative
contributions of the observed seasonal amounts of sulfur wet deposition.
Consequently, the model performance for the summer, the season of greatest sulfur
wet deposition, received a weight of approximately 0.4. The distribution of overall
xxi
-------
performance scores illustrated three distinct clusters or groups of models. One model
having one of the highest overall performance scores was deemed to have performed
relatively well, but used an empirical "correction" (or background) term accounting for
40 to 60% of the predicted sulfur wet deposition.
In conclusion, this study demonstrated the use of a unique approach to model
evaluation, which accounts for the uncertainties of the evaluation data. This approach
is appropriate for future evaluations of the recently developed Eulerian models of wet
deposition. By applying this unique approach, this study assessed the performance of
linear-chemistry atmospheric models of sulfur deposition to identify the strengths and
weaknesses of each model.
It is recommended that the reasons explaining the behavior of these models be
investigated as a means of identifying and implementing scientifically defensible
improvements to the models. It is further recommended that the improved models be
evaluated for subsequent years when emissions data are available and more
precipitation chemistry data exist. Additional sites would reduce the interpolation
uncertainties and thus provide for a more rigorous model evaluation.
xxn
-------
1. INTRODUCTION
1.1 Background
Within the last ten years, regional long-term sulfur deposition models have been
developed to estimate monthly, seasonal, and annual air concentrations of sulfur
dioxide and sulfate and wet/dry or total sulfur deposition amounts across Europe and
North America with spatial resolutions ranging from 80 to 150 km. The results of
some of these models, which do not usually concur, have been used to estimate source
region-receptor region relationships (hereafter referred to as source-receptor
relationships) on regional and international scales. These relationships have been
used to investigate emission control strategies to reduce sulfur deposition amounts in
sensitive regions of Europe and North America and to improve visibility by reducing
sulfate air concentrations in the United States.
Since at this time it is impossible to validate these source-receptor relationships,
the models should at least be evaluated for consistency with long-term regional
patterns using available air concentration and wet deposition measurements of sulfur
compounds and a rigorous statistical protocol. One could reason that if a model
cannot replicate long-term regional patterns, its defined source-receptor relationships
would be less credible. This is not to say, however, that a model which is capable of
replicating the observed patterns is also capable of accurately defining source-receptor
relationships. One would first need to substantiate that the model is producing the
right answer for the right reasons. However, because of severe data limitations, this is
difficult to do.
Not only should these models be evaluated via a rigorous statistical protocol, but
the performance of each model should be intercompared to illustrate the similarities
and differences and to attempt to link performance with model approach. In past
evaluation studies, intercomparisons were infeasible for several reasons. First, with
few exceptions, common meteorological, emissions, and evaluation data and
evaluation protocol were not used in the independent studies (see Appendix A).
Secondly, in North America, model evaluations were limited to the eastern portion of
the continent and to periods ranging from a few days to a few months. The short
evaluation periods and the restricted geographic region reflected the limitations of
available evaluation data bases and, in some cases, the difficulty in generating long-
term model input data bases and model results.
The first North American model evaluations focused on air concentrations, since:
1) An extensive data base for daily sulfur dioxide and sulfate air concentrations
was compiled during the 1977 - 1978 SUlfur Regional Experiment (SURE)
1
-------
intensive sampling periods in the northeastern United States (Perhac, 1978).
2) No dry deposition measurements were available for the pertinent temporal
and spatial scales.
3) Sulfur wet deposition measurements were sparse.
The few operational sites of the Multi-State Atmospheric Power Production
Pollution Study (MAP3S) and the CANadian Sampling of Air Pollution (CANSAP)
networks provided sulfur wet deposition data across parts of the northeastern United
States and eastern Canada during the same period. However, the small number of
sites precluded a statistically meaningful evaluation for sulfur wet deposition amounts
(Schiermeier and Misra, 1983).
By 1980, however, the number of precipitation chemistry sites in North America
significantly increased as a result or the expansion of existing networks and the
establishment of additional networks. The new networks were the United States
National Atmospheric Deposition Program (NADP) network, the Electric Power
Research Institute (EPRI) network, the Canadian Air and Precipitation Monitoring
Network (APN) and the Acid Precipitation In Ontario Study (APIOS). The significant
increase in the number of wet deposition monitoring sites now made it feasible to
evaluate regional models for sulfur wet deposition amounts.
Recognizing the need to evaluate existing regional sulfur deposition models on a
common basis and to determine whether these models satisfactorily replicate sulfur
wet deposition patterns, the Office of Research and Development of the U.S.
Environmental Protection Agency and the Atmospheric Environment Service of
Environment Canada conducted the International Sulfur Deposition Model Evaluation
(ISDME). This study evaluated eleven regional models using seasonal sulfur wet
deposition amounts calculated from screened 1980 precipitation chemistry data.
1.2 ISDME Goals
The ISDME goals have been tailored to the needs of both the modeler and the
model user (e.g., policy analyst). These goals also reflect the experience gained from
past model evaluations and the perceived limitations of the model input and available
evaluation data.
For the modeler, the results of a rigorous model evaluation can identify strengths
and weaknesses of a particular version of a model. From this and other information,
the modeler can re-evaluate and refine the model, since model development is a
continuous, evolutionary process relying on iterations of both model modifications and
evaluations.
-------
For the model user, specifically, the policy analyst who focuses on the predicted
source-receptor relationships, a thorough model evaluation can establish the reliability
of a model. This brings us closer to the point of assessing the confidence of its source-
receptor relationships.
It is recognized that a thorough model evaluation should not only investigate
whether a model produces the right results, but it also should address the question, did
the model produce the right results for the right reasons'! This question needs to be
answered to establish a level of confidence one would have that a model which
produced the right results for one scenario would also produce the right results for
another scenario.
The unavailability of air concentration measurements across North America,
however, precluded a thorough model evaluation. Therefore, it is not always possible
to substantiate whether those models producing the right results did so for the right
reasons. Furthermore, the lack of a credible means of identifying source-receptor
relationships precluded an evaluation of the modeled relationships. On the other
hand, with the available wet deposition data, we can assess the performance of the
models in estimating sulfur wet deposition.
The specific goals of the ISDME are as follows:
1) Evaluate operational sulfur deposition models for each season for an entire
year.
2) Identify which models best replicate the observed spatial and temporal
patterns of sulfur wet deposition within data uncertainty limits.
3) Delineate for each model, areas and periods of good and poor agreement with
the observed sulfur wet deposition amounts.
4) Intercompare predictions of air concentrations and dry deposition amounts.
5) Investigate the effects on model performance of the uncertainties of processed
precipitation data grids and the observed sulfur wet deposition amounts.
6) Attempt to explain the model performances.
Originally, the ISDME goals included an intercomparison of model source-
receptor matrices. However, due to the time constraints of the modelers, this task was
deleted.
-------
To achieve these goals, special investigations were first conducted to explore
methods and techniques to:
1) Screen the precipitation chemistry measurements for completeness and the
sites for regional representativeness;
2) Calculate sulfur wet deposition amounts;
3) Spatially analyze predictions and observations of sulfur wet deposition with
minimal interpolation errors;
4) Quantify the uncertainties of the measurements and the interpolations; and
5) Rigorously evaluate and intercompare the performance of the models, given
the limitations of the evaluation data.
The results of these investigations, discussed in subsequent sections and
appendices of this report, are useful in future model evaluation studies. In addition, a
special investigation conducted by Clark et al. (1986) assessed the significance of the
bias observed in the precipitation grids on the predictions of sulfur wet deposition and
the performance of one of the ISDME models.
1.3 ISDME Participants
The criteria considered in selecting models for this evaluation were as follows:
1) The model must be capable of simulating seasonal sulfur dioxide and sulfate
air concentrations and sulfur wet and dry deposition amounts across eastern
North America with a resolution of at least 150 km and
2) By June of 1985, the modeler must be able to submit model results for all
seasons using the 1980 ISDME standardized model input data set.
Letters inviting modelers to participate were mailed to those modelers
participating in the U.S./Canadian Memorandum of Intent on Transboundary Air
Pollution and authors of published scientific articles concerning regional sulfur
deposition models. The nine participating modelers and the eleven ISDME models
-------
are listed in Table 1.1.
TABLE 1.1 THE MODELS AND PARTICIPANTS IN THE ISDME.
Model Name
Participant and Affiliation
Atmospheric Environment Service
Long-Range Transport Model (AES)
Advanced Statistical Trajectory
Regional Air Pollution (ASTRAP)
Model
Ontario Ministry of the Environment
(MOE) Long-Range Transport Model
Regional Impacts on Visibility
and Acid Deposition (RIVAD)
Model
Regional Lagrangian Model of Air
Pollution (RELMAP)
Regional Transport Model (RTM-II)
Statistical Estimates of Regional
Transport and Acidic Deposition
(SERTAD) Model
Statistical Model (STATMOD)*
Tennessee Valley Authority (TVA)
Model-T
University of Michigan Atmospheric
Contributions to Interregional
Deposition (UMACID) Model
Eva Voldner and Marvin Olson
Environment Canada
Jack Shannon
Argonne National Laboratory
Barbara Ley
Ministry of the Environment
Doug Latimer
Systems Applications Inc.
Terry Clark
U.S. Environmental Protection
Agency
Doug Latimer
Systems Applications Inc.
Ron Portelli
Concord Scientific Corp.
Akula Venkatram
Environmental Research &
Technology Inc.
William Norris
Tennessee Valley Authority
Perry Samson
University of Michigan
A second version of this model, SIMPMOD, was also applied.
-------
1.4 Structure of the Report
The remaining portion of this report is structured as follows:
* Chapter 2 highlights the attributes of each of the ISDME models.
* Chapter 3 discusses the ISDME standardized model input data set and the
sulfur wet deposition and air concentration data bases.
* Chapter 4 describes the ISDME model evaluation criteria and the measures to
quantify model performance.
* Chapter 5 describes the interpolation technique applied to spatially analyze
sulfur wet deposition amounts.
* Chapter 6 presents and discusses the values of the performance measures for
each model, intercompares model performances and intercompares model
results for those parameters for which appropriate model evaluation data are
unavailable.
* Chapter 7 compares the model performances for each season and discusses the
overall performance of each model.
* Chapter 8 presents the conclusions of the study.
* Finally, Chapter 9 offers recommendations for future research.
Numerous appendices discuss the results of special investigations or provide
details not included in one of the above chapters. The text in the main body of the
report sometimes references figures and tables found in one of the appendices. These
are distinguished from the figures and tables appearing in the main body by the use of
the letter of the Appendix where they are located. For example, Figure B.I is the first
figure of Appendix B.
6
-------
2. MODEL DESCRIPTIONS
The regional models evaluated in this study are designed to predict average air
concentrations and total wet/dry deposition amounts of sulfur dioxide and sulfate for
periods of one month to a year. Since their primary objective is to assess control
strategies on groups of sources or emission regions, the spatial scale of the models is
such that point and area sources within 80 to 150 km grid cells are aggregated into
single virtual sources.
Emissions of sulfur dioxide from these virtual sources and the sulfate resulting
from chemical transformation are subjected to vertical mixing and horizontal diffusion
at rates prescribed by the model as the pollutants are transported across North
America. During transport, percentages of the pollutant mass are dry deposited and
scavenged by precipitation according to rates prescribed by the model and the
precipitation data used by the model. A portion of the total mass is transported
beyond the continental shores, off the model grid, and is no longer considered.
Some models consider a background level of air concentrations attributed to
biogenic and distant anthropogenic sources. As a result, these models enhance their
predictions of dry and wet deposition.
Sulfur deposition models can be classified as being either purely statistical,
Lagrangian, Eulerian (ordered here on the basis of increasing levels of sophistication)
or hybrids of the three. The fundamental differences in these three model types,
elaborated by Pasquill and Smith (1983), relate to the approaches of simulating the
physical and chemical processes.
In simple terms, the four ISDME statistical models (Table 2.1) generally simulate
transport and diffusion via one or more long-term average, single-level wind roses,
which represent speed and directional frequencies for 22.5 degree sectors. Wet
deposition of pollutant mass is typically dependent on the average frequency and
duration of wet and dry periods rather than the actual occurrences.
On the other hand, the seven ISDME Lagrangian or deterministic models (Table
2.2) simulate the transport of pollutant puffs via objectively analyzed winds at one or
more levels. Physical and chemical processes are simulated for each puff. Horizontal
diffusion is simulated in a variety of approaches based on puff travel time and/or
diffusion coefficients and/or ensemble puff trajectory fluctuations. Typically, the wet
deposition of pollutant mass is dependent on the rate and time of occurrence of
precipitation and pollutant air concentrations. Ensemble concentrations and
deposition amounts are determined by adding the contributions of each puff.
-------
Finally, Eulerian models simulate transport and diffusion, as well as other
processes, across a geographically fixed area, ratner than computing ensemble averages
and totals. None ofthe ISDME models were purely Eulerian.
Basic attributes of the ISDME models are only highlighted here. More detailed
information can be obtained from the modelers whose names appear in Table 1.1.
These models can be compared to European and other North American models in
Pasquill and Smith (1983) and the U.S. Environmental Protection Agency (1987).
-------
w
Q
to
i i
5
t
5
a
5
GO
S13QOJ
^
U
E
P
H
00
M
O
p*y
r^
H^
PQ
HH
M*4
5
i i
0
1
PMOO/ST
3E
in
o
VI **
*3 8"
a o
CO *
vi
c
O
o
vt <->
i S
*» 41
co ae
tn
+j
c
S *
5 s
'5 8
*J 01
co o:
§
r
V *-»
>> *_ «-
4_> O)
flj (U <*
"G u c
o
1
c
n>
(O 0)
o o
c.
K c
£ «
+-» **n
i- T3
* "
13
c
*a vi
c
s z
t- O
1. O
!i
e -o
Z to
c
T
o
o
$
o
Z
u
CM
O 1
in a +»
3 01 '>i
i £ * v,
? S -a §
« H , S «
CO i-
VI CO
CM
S 4 *,
U 01
|
C C w VI
IO Q C
i O -a o
10 II * « 4J
co -
o en vi vi
VI _ CO
10 T3 o O.
OJ C Jf
CO IO +J ^
o i o 1?
^ JO 0.-S
^- 4-> "O C4
*o c o
C U 0)
C C X "O t/>
7 8 ^ S g
c OJ c -tl
!S -a § w °
(^ (O *-*
C
I
o
c
$
c
"o
a.
CM
C
VI
0)
in
4-*
3
a. vi
c c
*"* O
*"" VI
O 10 E
ac o LU
OJ
c
0
s
t_
82
!o"i.
fej
C«M
C
0
VI
at
u
c.
0 TJ
bl E
IO Ji
COM
C
o
01
u
3 -4
£! E
orl
c cvj
ia f~t
c
o
c
*^o
f fe,
o
in E
i
1 M
I
§i
1
C «M
§C
0
»-l VI -*
73 -a
vi
a
c-«
* 'r
ll
o r~
-H
1 C
VO 0
c
c
o> o> a)
1 t- 1_
3 3
C L. 01
3 Q.Q.
C
O
VI ^O
C l_
3 Ol
|E
1
inS
c
0
VI "O
Ifc
ii
T oj
c
o
(XI <^
c -
| OS
"? rL
CVJ CM
r-l ^-1
|
*J
^L
-------
Q
W
D
L^
bw^
8
rH
(S
W
0 1 1
O T- r^
JE ££J
< VI VI
1 - I/I
CO XI v» »
^ O 41
a >, t. o
O ^" O
£ E 10 -a
3E O XI "u
« i H- 01 O>
= S j!
.
e
1 1
t. CM
< XI VI
* ^-b^
E 10 41
O 'O
i*- 4) XI
ilfe,
*- r*.
O I/I VI
( 5 VI
ae o 4)
UJ >t C t->
00 *e !e xi
t_ -
O XI 1.
«- 41 Ol
C 3 E
r> .a -*
o
4-1 VI
C C
4j O
4-1 VI
IO VI
41 -
t- e
1- UJ
«-
c
IO H- Of C\J CH
3 *J *J 41
C O 3 L- t.
e 41 a. o
10 «_ I? 14-
xi xi o xi 3
§ 10 vi $i"cn
^ 41 10 o
IO * O 4) O
c .a i_ > w
10 o e x> t-
oo U § o
R
t- a»csj
1 -O XrH
o a> L. "O "O
«-
S 49
t. * c
VI O -r-
X) VI i- X Tl
C 0 C
* U S O CTl
/O ^ IO
1. 14- E
o -o c. u .*
4-> 41 4-> 3 1
U 4-> C. VI r^
Sf^ES
i S .ge
rH U Ul «- O
C (.
0 0
a.
4J VI
10 C
10
3 (_
O > T3
< c
003
Ol
4 e
°Z
V) C\J
(U «
ia £
t. 0
«3
C V
SL. i
0
H- 4)
* U
S'T,
C 4J »-
O 41
^_ 3 *^
^y c o. t.
1
O» O. Of
*£°* *
-« ^ u .* c.
** 1 O
c i»- vi r*. 10
§O 0 CM 41
C - 1-
10 O t- J= O
r- 01 O *-
10 10 at
1*-
o
4->
C 1
8-S.c
1o o5
IV 01 «J
1- A. 10
H- Ou. 4^
£
VI
-1
c g*
c <
IO O
out.
C 'f- 01
-55
o o i
c ** r**
t> f-
fli *» c
i c i- £ 4J c
X VI VI IO IO
4J CM CM VO f-» tO
10 O i* f** PO O
a.f-. CM CM ao o
01 (.at
c 01 o e*
f- U 1*- I-H
X .,- 1
E c o> r-
1 VO 41
IO C_ ** *-
e o r^ "O .a
o »-« c «
VI <0 4->
IO I/I VI
41 4-» C 41 C
§f- S 10 t,
41 O 4-> O
t J= "0 VI H-
.*^.
o t.
41 41
X >»
z 2
-fjj
0 XI
c c:
1 « C
u x o
41 >, .-
C-
Ol*^> C IO U
C Xt 41 41 41
10 a 41 >, xi o
4) t. ^ 4-*
C. X< i»- CO.
Q. 41 XI -i- 41
I/I «j 41 u
S « £ XI 01
0 Ol C 3 4) fc.
O - o 01
kO 41 f t- O.»-
cn x 4J xi v> o
XI XI
i! "^ J
c'xi g S 4i
xi >. o -i c.
0 3 41 >>V) O
41 C. -O 4J
a. 0)** xi £ &
S. IO +» X» 41
O 91 e 3 41 i.
O *- o " 41
CO X *> "Xt vi O
c c
41 >»
1- O C «->
cn-r- c 10 u
C XI fl) 01 01
*- 3 E C.
x> >, cr i- c.
10 A 41 >t XI o
4) C. ^ 4->
i- xi i*. c ex
Q> Bf XI *r u
VI 4-> 41 U
JC lO 4-> XI 41
0 01 c 3 41 t-
O -^- o 41
IO 11 -r- -r- Q."-
ro x 4-» xi vi o
^HB ^
10 O
C VI
o t.
N 41
r- Q.
U VI
55
^
CO
L.
ai
.c
o
VI
41
«
t/f
c
o
«
^~
J2
iO <->
*-> 41 XI
vi « 41
41 VI
UJ I/I 3
S 41 4J
i2(S*
^ rJ m
10
-------
Q
W
D
P
Z
O
u
T"H
oi
PJ
O 4-1
O 3
«C »
1 3
t/} O
§ £2,
E 8.2
x <-
a o-o
w g
51
I
3
O
» £ Ol
1- 4-> >.
10
s
O T3
-5
5s
£
^'oo*'
SO ^i
41 3J L.
i jc * o
Of + Ol **
LU
i/1 +» 4-> >»
C C i '
O> <0 f-
ra c in
c
o
(O VI
O l_
_ u
*-> Q.
1. I/I
01 »-
> 0
^
10
c
VI U W C. C
n o< c oi E ra
01 *> E 33
vi c C. B *J c
^ Q. 3 3 C
L! i ro in r^ o
5 o' ~ ^ o' -
(O
C -~ ' .
o u cn u c r
vi 01 c 0) E c
VI i* Q. 3 3 C
01
U CJ 00 OS CO (/>
> o o oo o
cn
C "*""*'
"g ^ ^
3 ouo»c^-t- COVI4IOO O
vtc. ^ v£) o mm . * *
CO <0«. . COO O
»-»*- ^O«-< «-*t/>-r-CXJOO «^
c
o
f^
£^
VI
C 01
n *J
1- Q!
^
^ ^"
10 »
%^., .^.^ §"£ '^'c'Z?
in i_ 01 i_ c inoi 0( S ra
ieoicoiS'0 «>-> §33
HI 4J -i- B 3 3 OIC S « C
in c l- B *J c in 33C
>-a.33C X tfi«o eaoo
in
m o
o c" ^
c
c_
in
0
O
^ -o
O t_ O» e- C. u
vi
VI «_-*_" ^ o
<&) U
-r- ro O O
t_ ro ir> *o 4-»
H~*
O "u t/i
*"5s o" o*
(5» ~- oo v>
E
11
-------
a
o
I. C
vi 01 c 01 B »- 6 3 3 « «»- 3 3
OICC-S*->C 41 C i- B « C
VI *- Q. 3 3 C VI a. 3 3 C
Xviviiaia x v> vt a iq
ot ot
»- CM co co o en >-«fooanm 01
m f*^ co o 91 CA 10 f) ^o to en *4 n
> I-II-H ^cjrOfOCMro z
Ot-Wec Ot-O>l-C"~
MOICOJEK) vtoicOjB -^ E 3 3
OP c i- e *j c aict-Bfc
VI-^-Q.33C VI-^Q.33C
XVIVItOt) XVIVIIOitl
Ol V
^o«roi/ioi ^-o^oinai o>
tacocMiomo tooocxjvomo o
§
01 i-
«j «
C f
4-> O
a. o *j O. L. U L.
A9 Ol Ol O *
4-> XJ J= u «-
x a.
_>.
«'
§*» «>
^- c
4-><#- > vie t/> oiQcoom***
ooc I -^-31. -^Sr-
CO>&. O> Q U O. i« >, > O 4 CM CM CM
BT
a c «
c O J=
i«. ,
{. - o
Ol VI C/l
o
(/)
O
01
s
12
-------
w
s
Q
C/5
V*^
£
g
Q
cd
f"H
t/5
2
w
Q
O
5
[INISTIC 1
2
ft!
B
W
Q
W
p- '
fc
O
CO
L_i
C
h^
CQ
t^
fc
H
"*
tN
rj
Q- 3
fa.
c
LU 10
o; <-
Ol
c
IO
l_
Ol
IO
14-
a.
LU
O c
Z IO
c
a
£*
IO
1
0
4J >»
10 L.
0
t'S
CX 3 '->
c a. a
<9X L.
1 C 4J
oo 10
Ol IO
C 0
10 f
(. <->
Ol VI
10
X
00
*"" 2-
c
fa.
J3
Ol
Q,
t~
r-
S
s.
IO
ox.
^H Z
O
x oo o
0
0 ..«»
rH VI
* IO
0 01
CO 0 6
X TJ
f O
£ h."*
^ C7*f t
VI
"t> z'
U
o
o o
f- to
Ol +J
oo
co 9
X
r-. CM
m rH
...
VI
"aj z
u
o
o o
h*
U. *
i- e
X "*
"H CM
m i
t- VI
O *"""
VI CU Z
4-> U
c o
i TJO
O "^" to
Q. L
o ao
4JU-1 E
8" x "*
CU CM CM
o£ m « »
c
o
4->
t_ L-
O 2.
Q.'"-
01 14-
U C
« vi
c *-
VI I/I
c o
o a.
S*
to
^
-»
£
T3 O
IT 4->
VI VI
C 0
F* ^
4J«
,
O OO
f-
e vi
fe T3 Z>
C *-
T3 IO VI
SO
VI CX
c sr
vi o "O
c >-
0 4J 4J
^ -.- Ol
4J VI X
4^
Ol
Iv,
c
o o
<0 4J
VI t/l
c o
3
a.
c
»"«
_
*3
^cS
.
Ol
c
0
a.
CM
O
oo
cz
o
a.
o
00
a
c
0
VI
IO
(X
c
o
CL
o
OO
10
1
IO
c
o
a.
CM
o
_ -
10
c
o
VI
10
&
VI
c
o
£
I/I
LU
C.
8.2
C_ 1
ir
IO
01 X
L.
iO 0 -.
o -
ecu
10 o 01
VI
Ol -"
(-) TJ
L. *f-
3 L.
O Ol
VI
e
IO ^t
Ol 1
10 CM
« 1
a
c c
10 o
1
§.* L.
riS
VI CM VI
ai i e u
<-> JTN CU
L. ^- O OI4J
3 « O C CU
S e-* - S
VI 0 0 a. t>
01 C VI CL
SI*--
^ T3 IO O
a -o i- o 10
C 1 U t- 4J
IO CO Ol 4J VI
VI
Ol -4
0 T3
Ih
VI
e
IO .14
01 I
PO 10
§
c
4J O-
Ifl W
5sT
*°°f
n -Or-.
i e CM
0
r 1
(tl 13
^1 .,«
«f 0.
-?SS
S oo .*
C. "O r-l
1 CZ CD
to IO CO
o
c
ia
cu cu 01
« 4J VI
"O ^ VI
5 S.a:
c
o^
vi XJ
e o
s. x
t
CM O
1
l~- O
CM f-
c m %
o c
a >,
VI C VI
4-* ia
C 4^
3 IO
E *O V)
a f- w
t- c in
£ 01 3 ai
CM E B *<
>" AC IO I/I
VI TJ
4 ' -
IE
(O Jf
1 CM
o
3 t^
10 e
^ i
rr CM
CM 1
C
0
IO
«r
Q.
U
V
ariatlon
*
o
e
°E
JC
^
4^ 4rf
CU 01
VI VI
IO IO
4J 4^
3
1 1
T5 T>
1 3
c c
10 a
VI t/l
LU LU
S E
CO CO
-* (N
3
13
-------
Q
W
P
^^
^H
8
(S
(N
o
^I P_
4J 1 *# 1 IB
IB f- 1 C. C
C XI C =, I.
in f- Ol IB ~ 3
t- in +J *J
**- - IB VI w U
Q. 3 t- E O
LU *- t. VI C
oc oi m i. >»
o «-> in 41
oi c a >> <" >> _
VI -r- E IB 3 IB Ol
f- B
4-> "O
IB Ol
X C
in ,- i-
v- B
a. in in
UJ C IB O l_
O i- > 41 41
I L. C >,
O Ol IB IB
41
X
s ^
o
C X *
»-«- in
> e 3
I/I o o C.
UJ E 41 41
«t J= C >,
O IB IB
T3 >, IB "O
qj t ** 41
TJ ot v» X
T3 > C i-
«t oi f e
o
4-» in
c c
*J in
si
t- UJ
*-'
UJ
1 1
X <- c
O 3
L. VI
»- 1 IB O C
O -i-
in >jO oi x
o oo Q. c i-
C O -Q 41
'"" IT w j= i i?
Ol C OIO
S IB " ^j 00 Ol
1 in X
O 13 41 T3
§*j c >» o TJ
in T- .. IB -
-H a) » -* i- e
1
in c in in
Ol 4-» *" JS t3
U 1. IB I C
3 O *- fl -i-
in E X
VI IB O Ol
Ol *" C *-» U
L. > >i IB -
"o o 3 a.
IB U in
jC IB C O
1 C O O 01
ro IB C7i 4.* o>
1/1 in
T3 'OfO
c «
01 O
»_ X -^ <-> -O
° «*" o-fc.
Ol B O 3 Ol
C 1 1
f- O CM VI E
+j m t- .*
£ CO Ol 41 1
o> o x<
S"" C « CO
IB O
1.10 C
c^ o ca. o
^-,0 0^ J
i?
180
IT*
o oo
o.
t. «
Ol C
4>* IB
c
~- 1
o
t. 0
IB O in
01 -i tj
c c
Zj O X
c t-
0 O
^ a.
*j in
IB C
E Ol Ol
«l_ IB Ol
o a. c. o
o-c. a. o
m ex i 3 i*. x>
4-> « CM in
C "< « L.
3 >> C f Ol
8^'S0SB
IB ^3 ^ ^
41 ** 41 >> 1
j= c e m c_f<-
1 O! 1? »
U
O IB
e vi
o
C 1
41 Ol 4J
U I- IB
! a- 4->
=
_
o
?l
o c
U v O>
X
5-x
C IB
£.--
^-^,
>. 1
IB 3
in c. oi 41
IB 41 C E
Ol *J - E
in c L. 3
<- Q. VI
in x m *-'
> in ao r-i
1 <^->
01 3 X 3
>- «. a in
c_ t- -~.-~
« .- c. en e
> -0 « 41 C E
Ol 4-> f- 3
>> >> C t- *>
i T3 - -^- o. 3
X> t- x in IB
IB 01 0
C U
1. J= O.Q O 0
3V O O O
5 5 **-< i 2
jr i
4-1 # ^1
B '^ ^
X g
0 B
X -*:
J= CM
^ i! g
^E
0 L.
dJ
-------
Q
§
C
2
8
(S
(S
<£
-*
ro
Q. f*>
«c
54->
c
10
4-J
VI
c -«
"
«
1 U -.
3 Ol o ivi
.Q -r-> . . *
r- (O VI +J CM
(_ C. 4J XI
4J 4J C -* XI E
I/I f- Cl/l
UJ f- 4-» O + IO O
o x> 3 o. i-«
10 10 c o
*- oi o CM in
VI VI CO
VI C >» II
3 0 c. n n
5'*- O O XI
4J 4J O O X
C 1
O 3 O 1
.a 4J o i_
I/I i- 01 O
i_ t. vi -> ,c 14-
ai 4-> c
*t ^- XI 4-> L.
1 O> c O O O VI
E C 3 L. VI £
V £ "« 'i o! 3 §
O >*- C O - 1
_J .O 4-> 41 4-> > f-4
VI >>
VI 1 XI
>, O Ol ' t»-
1 c- c c VI g. C VI
C 3 VI ~- 3 O
O -O OJ "^ "* ^ 4-*
« -f O XI XI E 4-> 4-* *">
i -i- tl at a.*- c.
UJ ^ C7> 4-> 3 0
3 * c
o
3 ^"> Ol
o t_
c ^: c 3
g
CTJ at
3 >>
t. r
4-> X)
«u
t. £
O E
55
XI Ol
C C >) 1 C
IO - 4J 3 O t-
XI .i- c - O
>> t- > 4->
^ 8 ^i i i. &
IO U 3 « 1 Ol 4_>
C >J- -r- C VI 01
^ ^ XI -r- 0 >.
XJ i I/I t. (O
ai o xi T- o xi
»- vi ai <-*- x oi
L. VI 4J QL E 14- E
3
o
-C
Ol
3
0
1_ t_
£ Ol
4-> >,
re
e
c.
0 XI
^ X
5 1
o
(O VI
U L
^- ai
4-* a.
(_ VI
^5
J
XI --~
5 E!D~
-r- O *-* C
*-) O c E
x^",3
p t. ^ ;j
i -0 (010 «
rtl C ^ fO *«^^^«
C IO O Ol ^
c. vio c ai
^3 ^>1 - g
x> ^ loa: Q. 3
fO VI I/I
VI C **°
01 O >t^>
* VI «! Ch »
&. 13 »-l Ch
« HI ID 4->
» VI C tl r-l r-l
^> i
IO 3
c _«
I/I (^ Ol (_
IO Ol C Ol
Ol 4 ' 1~ g
vi c u E
*- Q. 3
V. X VI VI
u in o o
(O .
= O t-l CM
x>
c
t- Ol c. C
_>> o» c g S
IO t- Q. 3 3
C X V> I/I (O
3 >>
r- CM O O O
XI r 1 CM O O
VI C ^* CO *» CNJ
Ol O 1 1 1 1
*- VI \O CM O ^
.
C IO-^ 10
§3 >i 3
c c
10 3 C
VI "O O > C 11 <- Ol l_ c
T- Q. Ol C O) E
C 4J -r- Q. 3 3
O - Ol X VI I/I IO
(O 3
oi JQ i in in in
VI - t: in in in
L. 4J XI O O O
IO -r- C
> X 10 OO O
.."Si
>> c
IO Q.
C VI
O ^s*'-^'.
VI «- C t-
Ol 4J 3 g
VI C 4J E
'»" 3 3
VI X IO VI
t- LO OO
IO
> o o
^-x*-^^-^*-^
t- on. c
^> oi c v E
C H 5 4-1
IO »- Q. 3 3
C X VI VI IO
3 >>
r- o o o m
T3 f r^ OO CTI IO
IO
VI c O O O O
Ol O 1 II 1
- VI O O O O
10 Ol . .
> VI O O O O
c
ro
J= >. ^_
§3 >> 3
c c
IO 3 C
I/I ^O ^^ IO
iSSS
c:
o
^
o >»^~^
O. |Ji 1 (/>
Q >xii ^*
E*
a
15
-------
Q
W
D
2:
f4
^^
Jj£
O
u
(S
(S
X
,.*-^^,^
>> C OI t- en e L. >, O)
> - O. 01 C S 0) t.
>i«J - 3 S «J-O 0»CftO
«o >i «> c t- 4-1 £ io *-* r-* ijzfnr^
C »> O. 3 3 c C I
O - CM o .a m u ~o
4J C J U
UJ in V> «O O O O in_ V V i/) V >/
ee 01 i i i ti° v ..
f- J= m o O ,- ,oj yj oi
t_*J^Ortl t_Q-O--l l_ ^O O C
IQ^-C** to a*-*^f_*o
=. S roOO 0 >«00 g aid C} g
en
C I
>>- i
<- i-
Q- >.
« in
C *^» " ^^ ^~* r^
0 (. C 1. 1
in oi E u a
S^5I £J
in - 3 3 U) ai
UJ X fO in r-4 JT
0 in^^^ * a. ^ en X
I a> a. .* i.
*-u» oo co C
1. O O GO O dtf
IQ O O >
» o o ^-i _ c\j oi
U> IA
0 0
a ia 10
c OI
IO ^~ ** **-K.^ 4J Q. Z CU
t- O> L. C IO
>>OICOJE L. r^U>
cjTtnvtio >> H-
Q. j_ ..^-» ._ Q^~ O
< 3 >, e j= .._ , -
ac i- 1/> o in vo 4) j^t
1 -a o 4-> aj
ft nj . . E u *j E
«t mcOooo coi-aoca
01 o ' ' i i oi S c £ E
f-viininmm j^ CL - »-i^ oi
L.IOOOOO c*»c c
5Sooo'o £Z~ s 2= 1 5
c
>;- i
^ J1 - ^
in . 3 .
f o c c 0
S -0 3 C Z J-
-3-3 » -C S
in- 3
OI csl Ck.
i- oo o " a,
U O r-> O Q. <-, £
5o=»0 25
c
o *
4-* « ~t
j= c S
o x'^ a 3-2
Q.**FH S »« 0 W
oi f i 01 > £ Jr
Q o in a: oi ui
O 01 J< O
X E»* iJ+JrM » Oo.
L.OIUO mioo o aai
o»^^on ^ ac
j=
en
c
1<
'e
o
«j
in
fe
«*-
«
^
X
^^
1/1
L.
1
c
<_
49
Q-
1
(/I
0)
w
4
^
§
4J
*O
<-
a.
o
V
L.
Q.
Q
4J
in
t.
01
«-
«i
t.
c
oT
*
16
-------
^
«^_
a 3
o
-3
E
3
Q CL.
X
c T! a) >>
o s c IB c w t/i
ex « C CO 01 ^
O O 0) " C U
TI ll O 3 3 C
CT> 01 C T3 o
S 4J C II J C t-
E .c: u >i i/i vi
X.K *J ro i T3 CO
t. *»- C O O.
* oo z IB a *j o
M 1
C CM t- V
o c oo c
T3 O 0) *"~ ^ ^
* ** O IB C u I/I
C7I QJ C Q ^ O
5 " C II = ^
m a> i o *-
E ^ u >i oo vi m
O -r- Ol
3 C 3C
o o *- c
I/I * o o O
00 * U C 4->
__ 3 r: IB 4-> x> c c-
IB -f- O *^> o
C *J- ^ X X g E
UI » "O II II .C 1
IB -r- 1 jc 1**
» §bs3^ 2 J.2
t/l
U O 1
L. O O
3 - O S C
O «-H o
f- *j r» */»
OWL. CO JJ X*
{/I C7> .O C *
trt E -* 3 L.
r- g -0 Q 0»
IB J^ C O T) E
O O X CO t- a
UI 00 Ol f 1
41 C 1 C S M OJ
OO O ID IB -* -* 1
I/I
4)
U Q 1
i_ O O
3 - O 00 C
° T> *" °-«
CM *^> 4^ o It QJ
^3 r m ^j ^3 y^
- E «'8- IS. 2
O O » 00 H -X
I/I 00 Ol J= 1 "O
0) C 1 C E 4J
C 01 41 01 4J
O I t. l_ -r- tu
*- 3 3 Q. JC
E 41 L. t_
UJ 3C O.O- Q-
*
17
-------
Q
W
g
r^
O
u
(N
c4
u
o
vi e
-
O
OI
a-
O)
c.
I
VI
i , =
4-> £ C
O 4J OI
vi a vi x
S«-> c >-
vi ,- E t.
t- m 01
3 01 V> *> >>
o > - c «
vi i- vi o vi
4J 4J *- 3
C OI OI VI OI OI
f t- f- f c X
0 *- Ol E <0 i-
0. 01 J= OI 4J S
1 1
4-> CO) in m
n n -~ i/> c 4J
U (Q 10 > O Ol C
VI - VI flj
4-> 4-> 4J V9 >,
4JU£(QX: UC4J
caio>>oii_oi
-t-f- t.-»- 3 Q.'
o>«- oioi 01 oaif-
Q. aj i: 4J c vi-ea
£ c
a o 5
Ol C 4J 4J 4J
«-> y c «
3 .C Ol -i- Ol
t4J "-> O
"0 Q. Ol
0 X I- S
O 4-> "O
fl CtN
I/I 4-> «- Ol "O
^ JJ 0 ^^
C U 4-» ^4
X Ol C Ol Ol
,= ^0!^
i vi o ai vi
<"» 3 fO VI 3
o i
1 010
a. c O
3 0
B i vi
T3 £ -0
C 01 01 c
« 4J C i-
iu X
U w 4J
Ol JC «
S>«^f
C Ol O
a * *«
^ z oo
Si. ST.
£cUS
"O >vt
1 ai ,
Ol 4_» TD
M a* *-
o ^- t-
u -0 o. en
E -^ u IQ
3 O 0) C 4->
TJ O *J j 4>
C C »-*-» T3 "O
»- 0* C O
7 E J= -5 «3 c
4-»
C L.
O O
4-) VI
«3 C
^~ «
3it^
c
>b V- -r-
003-
Ol
01 -0
E 0 3
E 4J 4J
3 C
VI o Ol
VI -D §
w at
c -iS -~
3 13 01
§ '" 3
« Ol 4J VI
C "O 4->
1 C c u
o >o
in 01
4J e
c <- 01
3 O U
ft. <*.
3 'O
U T3 -r-
o o i~
£0-0,
CM o Ol S
" 4J O..X
o
01 x: I
£ 1 O
3 m oo
I/I
VI Ol £
K> C U
o «
I/I Ol f
4-> C
C <- C. Ol
§0 Q
<_ 0-
3 T3
m u ^3 f~
0 o t.
£ 0 01
CV 0 01 E
" 4J Q.-W
V-
o
V
C 1
i^c
So£
Ol Ol 4J
c. t_ «»
1 Ck 4J
£
_>,
£
*»
c
g
I/I
Ol
»»
l_
ia
^
01«
- 0 E
>y~- f-
*~~ o c
4J -^- ^
> S J*
vi ^ 6 i
OJ £T ^3
»-*>' 00
u c *
« a o c
=» 6 -r- 0
1
oi g
-OS
X
1 o c
4-> -^ -4
flj 2 E T3
c C ^--^
«- *- a v
3 S OS
5 015 s
-,^11"
Ol £ O
. »j . 00
i- c is
(0 0 0 C
-> S o
E
o u
x5^
r- (Q
Z -1
r^
4J
<«
w
o
Ol
rsi
5
L.
3
c
«o
4J
I/I
LU
S
en
-
18
-------
Q
W
D
2
(~*
O
U
o4
W
I
O CO 3
_i . o
CJ *-« C_ U
3 19 3 C £ Ol
»i Ol (3 E C3 *"" Q.IO .Q
> o E >- oj 4-> e
1 C <_ X C_ X> ci-X
z: 1| o a o o - e .c 3
1 * 1- E 1- S O4J
oe <=oj .Q c u «-«-
Ol X) t> >,3 10 X
Q- Ot O tn ^-.« .-VI
«"O l/l I/I ' C_ L. l_
F O * c VI Ol Ol Ol Ol
^C^-Evi <3 S
1 J.
^* 0 ^ 0
v »-< .a ««_
+J A 4J _^ CO
4-» 0*O
Q c t_ t_
»m ^: o *-
11 .. -* *J 3E L_
QC co e **- ^
"O 4-» (/t CVJ
VI O O C » >» O. >> t- <->
4-J OJ
C3OO O-r-O>U-
51 _g
C VI «O VI
O <- 0 <_
NO) ^- 0)
- Q. *-> Q.
i. 1/1 l_ VI
I O > Q
-4 J
TJ
C 'O
*o c
' 1- t- >l
§r
n
1- ^- 3 C
3 ^3.J2. *-
5 1 ' " ^
IO « CO T3
VI C
Ol O *""* ^^ l/l
r- VI I 1 01
U *O ^H CM *-
(O Ol L.
^ VI O C3 IO
1 1
Ol -i- 1
VI 13 O
3 <-
M ^.^_^^^, °
< to ai c 01 E ai
«o i w ^- g 3 E
c c c- E «J
U "O «f- o. 3 3 J=
3 c * in i/i jj *J
"O J
x, in in o 10
VI « . I/I
OI*"-..^-«^-ICMf-4 Ol
-«>, 1 1 1 1 -r-
t. c ' vn in in m c_
»vi
T3
10 U
1 1- Ol c t- ^
(O 4_J .^ 3 B VI
t_ .. - Q. 3 Z3 XT
3 >» S vi aj in 4-)
^ ^~~ X
«o
= in O o O »
§
io, ^
*-> o >,-^
O 1- E *--^
U t-
-- 3
3. vi
*~**^-^
i m CM
m oo
10 ..
coo
O 1 1
in o O
(O l 1 *-<
Soo
tA 4->
L. - JZ
V* O. C *
0) >, C
e 4-> u
<0 3 4J
L. Ot w (O
(O U
tl0.0.
IO 3 "-< O
u in i
.»- in
OIT3 CO >.
5 SoS
,
3
a
o to
in
. O
-« O
i
f O
t 1
0 £
(O
" 13 4->
S.O.J,
19
-------
Q
W
P
Z
P
O
u
fS
c4
o
c
!_ C-
**"" 4-* 5
0 ra 3
c _^»,^n_
0 u ^^^
S X) c\j «*
(O
in c O O
Ol O ' 1
t- in in co
U 10 O O
5 in O 0
l
O
c in
8k. »
Ol 01 9>
«-> «-> Q. c
11 OJ >>f-
1-1 E B »-> <-
>-4 Itl 3
1 f C. 0) 4
£ 4-> (O U
| - 0. « 0
0£ S *- O
L. .
in (O 3 *-H
0) U in I
t- -^ in
C_ O>TJ CO >,
> §0-0
Ol
«-
in in
o
5 5?
1-1 - Cvj
oe X
in
Oi .-
f ,^ E » ,
i_ OJ U O
o s* -^^ oo
6
OJ
CVJ
a: .
^ o
Q. Q. 0)
C
v^ O
IO CU Z
^
m
-1 CM
^. 0
C*^ f*>
a. a.
(Q A 0)
c
5
^
z
in
-* CNJ
z c
^- ° 3
a. a.
!c ^ §
§2. o **
41 a o
<->«-> N n j. o S.
a* *o o o n At
3tt <-o (>o ao Q
0
a
^^
S *~
H 4-*
> ' < in
"m o
O 3
is er
in
L. C
01 c
k, O
rc *j
* c
in c
ki Ot
o c
c
01
a> 4-»
ex
«*-
g o
VI 4-»
l_
*o
c
a. <
* -*
w
-------
3. ISDME DATA SETS
3.1 Standardized Model Input Data
Since models can be sensitive to the input data (i.e., the emissions and
meteorological data), sets of standardized data grids were compiled and distributed to
the modelers to minimize the differences in model results due solely to differences in
the data sources or data processing methods. Whenever more than one source of the
data existed, the most appropriate source was selected. As described later, data from
these selected sources were screened for potential errors.
As with any measurement or estimate, uncertainties are inevitable. Assessing the
uncertainties of the model predictions arising from the uncertainties in the input data
was not within the scope of this study. However, as discussed in Chapter 5,
uncertainties in the sulfur wet deposition data were considered when evaluating the
models.
This chapter discusses the processing methodologies of the input data provided to
the participating modelers and the screening and calculation procedures to compile the
evaluation data base.
3.1.1 General Considerations
In the process of selecting a period for regional deposition model evaluation one
must consider the availability of quality controlled wet deposition data and model
input data (i.e., wind, precipitation and emissions data). Meteorological data, which
have been collected via networks with rather constant resolutions over many years, do
not constrain the evaluation to certain years. On the other hand, appropriate
emissions and wet deposition data are not available for many years. Wet deposition
data were not available at a sufficient number of sites until 1980, while the most recent
year for which detailed North American sulfur dioxide emissions data are available is
1980. Consequently, 1980 was selected as the year for model evaluation. Essential
data for this year were acquired and processed to create a standardized set of model
input data.
Processing of model input data involved the creation of standardized data grids
that were compatible to the ISDME models (Clark et al., 1985). The universal
application of one set of data grids is complicated by the fact that models do not always
use the same grid configuration or spatial resolution. Since the grid configuration is an
integral part of the model, model input data grids were created for each of the four
grid configurations used by the ISDME models as illustrated in Figure B.I. The spatial
resolution of the grids, which varies with latitude and longitude, averaged from 80 to
21
-------
127km.
Most regional sulfur deposition models require wind velocity grids for at least one
layer of the boundary layer, high resolution precipitation grids and sulfur dioxide
emission grids. These are the major components of the ISDME standardized data set.
Grids of mean monthly maximum and minimum mixing heights are also included.
3.1.2 Transport Winds
Upper-air winds are measured only twice per day (early morning and early
evening) at sites spaced approximately 350 km apart, except across northern Canada
where distances are greater. Thus, errors are introduced when interpolating the data
to finer resolutions (e.g., 6-h grids with an 80-km resolution for several levels of the
boundary layer).
On the other hand, wind analyses from dynamic meteorological models offer a
better alternative for several reasons. First, since the dynamic meteorological models
include three-dimensional parameterizations of pertinent physical processes and a time
history of the data, the analyses between measurement sites and for intermediate
periods is likely to be more accurate than analyses derived solely from horizontal
interpolations. Secondly, these dynamic models screen the wind, temperature and
height data before using the values in the analyses by examining measured wind
profiles and using time series to identify measurements that are inconsistent with
physical laws. Finally, satellite data are included in the data assimilation process to
compensate for data paucity in some areas.
On the basis of these assets, it was decided that dynamic model winds would be
used for the ISDME. Since they were readily available from archived tapes, 6-h 381-
km wind grids produced for the 1000- and 850-mb levels by the Canadian
Meteorological Centre (CMC) primitive equation, spectral model were selected for
ISDME model applications. The wind analysis method is described by Rutherford
(1977). These 381-km wind grids were reduced to the finer resolution ISDME
configurations via a triangularization technique (Lawson, 1972).
Quality control measures consisted of comparing the finer resolution ISDME wind
velocity contour plots to the 381-km wind velocity plots for each 6-h interval of one day
in each season. Maximum and minimum speeds for each interyal were scanned for
each grid configuration to identify any questionable extremes. High wind speeds were
verified using National Climatic Data Center upper-air charts. No significant
inconsistencies were detected.
3.1.3 Precipitation Amounts
There are several archived precipitation data bases of varying temporal and spatial
resolutions for North America. Since the timing, duration and amount of precipitation
22
-------
could be important factors in simulating wet deposition amounts, one would prefer to
use the finest spatial and temporal resolution precipitation data base to derive model
input data grids. The data with the highest resolution is the hourly North American
precipitation data base, which consists of measurements from nearly 2000 sites.
Unfortunately, unresolved data inconsistencies and spatial voids prohibited their use.
The major problems with this data set were discovered after grids of hourly
precipitation amounts were generated. With no explicit indication, data were either
frequently missing and/or the reported amounts at many sites were much less than
actual amounts. Monthly grids generated by summing hourly amounts for all sites in
each grid cell detected numerous cells with unrealistically low amounts, differing by an
order of magnitude from the amounts at nearby 24-h network sites and at adjacent
cells. The cause of this problem was traced to the data themselves and not the data
processing procedure. Another problem was the sparse coverage in Canada,
in northern Canada during the snow season when many of the sites are inoperable.
As an alternative to this data base, the 24-h 1980 objectively analyzed, CMC
precipitation grids, with a resolution of 127 km at 60 degrees North latitude, were
selected. Despite the coarser temporal and spatial resolutions, this data base was more
attractive for several reasons. First, the data were already in a grid form. Secondly,
unrealistic monthly totals were not detected for any cell. Finally, amounts for data
sparse or data void areas, estimated from ship data or data from nearby sites, were
already assimilated into the data base. The precipitation analysis method is described
byDavies(1967).
One drawback of these data grids, however, is the consequential spatial smoothing
of the analysis method. As a result, the spatial variability of the precipitation amounts
decreases. Theoretically, this would tend to smooth the model results as well.
Since many models require 6-h precipitation amounts, the 24-h amounts in each of
the CMC grid cells were resolved to shorter periods. Since the 24-h periods in this
data base overlapped the preceding and succeeding periods by 12 hours, 12-h amounts
could be inferred. As the average precipitation event in the northeastern United
States ranged from 2.5 to 3.9 hours, depending on the season (Thorp and Scott, 1982),
the 12-h amounts were assumed to have occurred over a 3-h period beginning at the
midpoint of the 12-h period (i.e., 06-09 or 18-21 GMT).
Although they are the only existing data, McNaughton (1984) concluded that the
precipitation data from conventional hourly and daily networks do not adequately
represent those precipitation amounts required by the models. The relatively sparse
network forces modelers to apply spatial and temporal averaging techniques, which
contribute to the uncertainty of the predictions of sulfur wet deposition. He further
concluded that by reducing the precipitation grid dimensions from 80 to 40 km, a factor
of two uncertainty in deposition predictions could arise as a consequence of the
increased significance of the gaps in the coverage of precipitation monitoring sites.
23
-------
Since the dimensions of the ISDME precipitation grids ranged from 80 km to
approximately 130 km, the significance of the gaps is not expected to be as great.
Nonetheless, uncertainties do exist because of the averaging techniques employed by
the model preprocessors.
As a means of identifying any bias that might have been introduced in spatially
averaging precipitation data, the 1980 annual precipitation amounts at 72 eastern
North American precipitation chemistry sites, not used in creating the CMC analyses,
were compared with the annual amounts at the 127-km CMC grid cells containing each
site. Figures B.2-B.6 compare the patterns of 1980 seasonal and annual precipitation
amounts derived from the CMC grids and the measurements at the precipitation
chemistry sites.
The patterns are somewhat similar in shape and position, but the magnitudes of
the grid cell amounts appear to be less than the site amounts. Ruff et al. (1984)
reported a similar observation in another study. The histograms of Figure B.7 verify
the lower grid values: The annual medians for the grid and site amounts were 815 and
910 mm, respectively. For the grid amounts, the first and third quartiles were much
closer to the median indicating less variability than was found in the measurements.
This can be attributed in part to the fact that data averaging operations to derive
precipitation amounts for a given area smooth the measurements.
Differences in the grid cell and site precipitation data sets are further illustrated by
the 1980 weekly time series plots for 18 ISDME sites (Figure B.8). At these sites,
much of the annual bias in the grid cell precipitation amounts can be explained by the
bias for only a few weeks primarily in spring and summer. During these seasons, one
would expect that the larger biases reflect the increased spatial variability of both the
precipitation occurrences and rates due to the domimance of convective versus
stratified precipitation events. Since predicted deposition amounts are a function of
precipitation occurrences and rates, the uncertainties of the grid cell precipitation
would be expected to be the greatest in spring and summer.
The effects of these uncertainties can be assessed by substituting precipitation
amounts measured at the precipitation chemistry sites for the spatially averaged
precipitation amounts conventionally used by the models. For each season and the
year of 1980, Clark et al. (1986) have assessed these effects by comparing AES model
predictions to observations of sulfur wet deposition at 18 precipitation chemistry sites
in eastern North America (Figure B.8). The AES model was selected because of the
availability of its weekly predictions based on the CMC precipitation amounts.
The results of this comparison, presented in Table 3.1, indicate that for every
season, as well as for the year, both the mean predictions and the explained variances
increased as a consequence of using precipitation data measured at the precipitation
chemistry sites. The explained variances increased by at least 65% and, with the
exception of autumn, mean predicted amounts increased by about 10% or less. This
increase in the predictions is a rather small amount, but it nonetheless is indicative of
one component of the total uncertainty. Because there are differences in the manner
in which each model preprocesses data and computes wet deposition amounts, the
24
-------
TABLE 3.1 THE EFFECTS ON THE 1980 AES PREDICTIONS OF SULFUR
WET DEPOSITION AS A CONSEQUENCE OF USING PRECIPITATION DATA
FROM THE PRECIPITATION CHEMISTRY SITES AS OPPOSED TO THE
GRIDS.
INCREASE IN MEAN VARIANCE EXPLAINED
PERIOD PREDICTIONS (%) SITE GRID INCREASE (%)
WINTER
SPRING
SUMMER
AUTUMN
ANNUAL
4
10
10
37
11
0.12
0.18
0.31
<0.01
0.10
0.20
0.42
0.61
0.06
0.25
65
129
94
2204
160
uncertainties attributed to the averaging techniques will vary.
3.1.4 Climatological Mixing Heights
Some models consider spatial, diurnal and seasonal variations in mixing heights.
Since there is no consensus on the objective method to determine mixing heights From
rawinsonde data, only climatological North American mixing height grids were
included in the standardized data set.
These maximum and minimum mixing height grids were based on January, April,
July and October mixing heights for early evening (00 GMT) and early morning (12
GMT) periods as determined by Holzworth (1967 and 1972) and Portelli (1977) from
several years of rawinsonde data. Grids generated for these 4 months were linearly
interpolated to create grids for the remaining months. A constant 200-m mixing height
was assumed by Holzworth and Portelli over Targe bodies of water.
3.1.5 Anthropogenic Sulfur Dioxide Emissions Data
Since the ISDME models assume linear chemistry, the magnitude of the
uncertainty in the emissions input data will be directly reflected in the predictions of
sulfur wet deposition. That is, if the emissions data were overestimated everywhere by
10%, the predictions of sulfur wet deposition everywhere would be 10% higher than
they would be if the emissions data were not overestimated. This example illustrates
the importance of an accurate emissions data base.
25
-------
3.1.5.1 Canadian Data
The Canadian sulfur dioxide emissions inventory compiled by the Canadian
Environmental Protection Service (EPS) for the MOI Emissions, Costs, and
Engineering Assessment Work Group 3-B (MOI, 1982b) was selected for the ISDME,
since it represented the most appropriate Canadian inventory. The data in this
inventory pertained to the period from 1978 through 1980. However, approximately
85% of the point source emissions (accounting for 80% of the Canadian total)
pertained to 1980.
Emissions from all other anthropogenic sources were summed to area source
totals. Area source emissions, comprising 6% of Canada's total emissions, represented
1978. However, according to Vena (private communications, 1984), these area source
emission rates are not expected to vary significantly from 1980 levels.
Point source data, available on a stack basis, included stack location, height,
diameter, effluent temperature, flow rate, seasonal and annual emission rates, etc.
Seasonal area source emissions, on the other hand, were available on the 127-km
subgrid of the 381-km CMC grid. For the emissions data, seasons were defined as
follows:
Winter: December, January and February
Spring: March, April and May
Summer: June, July and August
Autumn: September, October and November.
The total sulfur dioxide emission rates and the EPS precision estimates for each
Canadian province are listed in Table B.I. Sulfate emissions, estimated to represent
2.7% of the Canadian sulfur emissions total, were assimilated into the sulfur dioxide
inventory.
The precision estimates were determined using a three-step approach. First, a
check for bias errors or omissions was undertaken and corrections were performed
when required. Secondly, systematic errors were then determined on a provincial basis
for each major point source and for each major point source category, based on an
engineering analysis of the parameters that influence the computed emissions. Thirdly,
a weighted sensitivity analysis was performed to calculate the precision of the
nationwide and provincial total emissions. The overall precision of the Canadian
sulfur dioxide emissions total is estimated to be approximately ±6% (MOI, 1982b).
26
-------
3.1.5.2 United States Data
Subsequent to the MOI project, the Man-Made Sources Task Group of the
National Acid Precipitation Assessment Program revised the emissions inventory used
in developing the MOI data base. The data were revised in several stages as updated
source information and advances in emission estimation methodologies became
available. At the time a data base was needed for the ISDME, Version 2.0 of the
NAPAP inventory was available.
The Version 2.0 annual national emissions rate was 4.7% higher than that for the
MOI inventory. The revised total was a consequence of the improvements of the MOI
inventory, which included
1) Updated 1980 sulfur dioxide emissions data from twelve states,
2) Updated EPA sulfur dioxide emission factors,
3) Updated 1980 power plant emissions data from the U.S. Department of
Energy and
4) Disaggregated annual area source emission rates to seasonal rates.
More recently, Version 5.0 of the NAPAP emissions inventory was completed. A
comparison of the Version 2.0 and Version 5.0 inventories indicated that
1) The annual sulfur dioxide emission rates across the 48 contiguous states was
0.4 % greater in the Version 2.0 inventory and
2) The annual sulfur dioxide emission rates across the northeastern industrialized
states differed by less than 6%.
Like the Canadian inventory, NAPAP point source data were available on a stack
basis for each season, as defined for the Canadian inventory. By definition, a United
States point source was a stationary source emitting at least 100 tons per year of any of
the five primary criteria pollutants. Approximately 94% of the nation's 1980 sulfur
dioxide emissions were released by point sources.
Not all point source data pertained to one year. However, over 95% of the
nation's annual sulfur dioxide emissions from point sources pertained to 1980. Much
of the remaining emissions data pertained to years preceding 1980.
27
-------
Seasonal area source emission rates were available for 64 source categories for
each county (or air pollution control district, as was the case for Massachusetts,
Connecticut and Maine). Total emission rates are listed by state in Table B.2. These
sulfur dioxide emission rates account for primary sulfate emissions, which contribute
an estimated 1.6% of the national annual sulfur emissions (MOI, 1982b). The
precision of the Version 2.0 inventory was not estimated, since it was an interim
inventory. The precision estimates of the state emission rates in the MOI emissions
inventory would not apply here, since many data records were updated.
3.1.53 Aggregation Methods
For each ISDME grid configuration and for each season, the Canadian and United
States point source emissions were aggregated to three-dimensional grid cells
according to the location of the stacks and their estimated effective stack heights. Due
to the numerous instances of missing or incorrect stack coordinates and stack heights
in the United States inventory, approximations were used when necessary to determine
locations and effective stack heights.
Discrepancies in the stack coordinates or missing coordinates were encountered
for approximately 20% of the nearly 200,000 United States point sources. These
discrepancies, which accounted for approximately 10% of the total United States sulfur
dioxide emissions from point sources, were identified by comparing the inventory
source coordinates with the coordinates along the border of the smallest rectangle
encompassing the county containing the stack. If the inventory stack location was
found to be outside this rectangle, the coordinates of the county centroid replaced the
inventory source coordinates.
The problem of missing stack heights was approached by using the average stack
heights computed for each of the Standard Industrial Classification (SIC) codes in the
NAPAP inventory. Approximately 25% of the sulfur dioxide point sources had
incomplete stack parameters accounting for approximately 15% of the total United
States sulfur dioxide emissions.
United States and Canadian point source emissions were aggregated to grid cells
containing the sources. Since pollutant transport and dispersion are dependent on
release height, the point source emissions were further aggregated to three layers, 0-
200, 201-500 and greater than 500 m, after doubling the physical stack heights (Figure
3.1). These layers were selected so that approximately the same emissions would be
emitted in each of the lowest two layers and that the emissions from the very tall stacks
would be separate. The frequency distribution of the effective stack heights and the
total emissions for 50-m layers are listed in Table B.3.
28
-------
500 mi
500 m
Om
0 m
Figure 3.1 The three layers into which sulfur dioxide emissions from point and area
sources are injected.
Release heights were estimated for dominant source categories and representative
stack parameters. Climatological data and Briggs' plume rise equations were used
(Briggs, 1969, 1971 and 1972). The United States emissions, predominantly from
power plants, were subdivided into three categories based on physical stack heights of
190, 220 and 340 m. Canadian emissions were subdivided into two categories of
1.__._1___ . 1 j A . ft -t nt * . .
primary metal plants, pulp and paper plants, four categories of oil refineries and three
plants based on physical stack heights of 91, 150 and 198 m.
§- i *. i .M.** J. J -1.1. A^r b. W. f-r-l. MJ..L V
categories of power
An average effective stack height for each category was computed for each season
using stability frequency distributions and associated mean wind speeds on a monthly
basis. These analyses, derived from surface data and the STAR program (Martin and
Tikvart, 1968), were available for the North American 127-km grid (Sirois and
Voldner, 1987). Rather than examining each grid cell separately, the stability
frequency distributions over three broad geographical regions were considered. The
regions were coastal, central industrialized and southern.
The range of average stack heights over the year was found to be two to three
times the physical stack heights. As a conservative estimate, an effective stack height
of twice the physical stack height was chosen for all seasons.
29
-------
Area source emissions were aggregated in a different manner. Canadian area
source emissions were already aggregated to the 127-km grid configuration. For the
remaining ISDME grid configurations, these emissions were assigned to the cells
containing the lower left corner of the 127-km cell.
On the other hand, a different approach was required for the United States area
source emissions, which were available on a county or district basis Detailed
population statistics from the 1970 Census were used to allocate county or district
emissions to the ISDME grid cells (National Data Use and Access Laboratories 1971)
ine relative distributions of population, determined from population-weighted
centroids of small enumeration districts, were assumed to represent the distribution of
area source emissions. Since the county dimensions were generally smaller than the
grid cell sizes, the spatial errors were insignificant.
Un1lt.ed States and Canadian area source emissions were assigned to the 0-
-m layer. Figure 3.2 illustrates the annual area source and point source sulfur
dioxide emissions for each of the three layers of the 80-km UMACID erid
configuration. 5
;N
"l 06*23*
EMISSIONS or sox
flREfl SOURCE - uMflCID GRID
ITONNES I
5.00E-04 TO
5.00E-03 TO
S.OOE-02 TO
S.OOE-01 TO
1.82E'00 TO
1.3SE-05
5.00E-04
S.OOE-03
S.OOE-02
S.OOE-01
BROOKHHVEN NRTlONflL LflB
wSjK
*fnu
LONGITUDE
Figure 32 The 1980 annual ISDME grids of sulfur dioxide emissions (tonnes/year) for
the 80-km UMACID configuration from area sources (a) and point sources in
the lowest (b), middle (c) and highest (d) layers.
30
-------
BNNUflL EMISSIONS Or S0»
POINT SOURCCIJMHCID GRID I
LEVEL: 1
I TONNES I
1.90E-05
l.OOE'OS
l.OOC-01
l.OOt-03
l.OOC'02
BROOKMflvCN NHTIONflU LHB
60^2311
Figure 3.2 Continued.
f\
~' ..
rt3»
3^3 H
LONGITUDE
RNNUflL EMISSIONS OT SOX
POINT SOURCEIUMflCID GRID!
LEVEL: 2
ITONNES I
l.OOE-05 TO
l.OOE-01 TO
l.OOE-03 TO
I.OOE*02 TO
3.63E*00 TO
5.38E«05
I.OOE-OS
I.OOE-01
I.OOE*03
l.OOE'02
BROOKHflVEN NflTlONflL LftB
Figure 3.2 Continued.
31
-------
EMISSIONS or sox
POINT SOURCEIUriflCID GRID)
LEVEL! 3
I TONNES I
OH
I.OOE-05 TO
l.OOE-04 TO
l.OOE-03 TO
I.OOE-02 TO
9.0BE-OI TO
7.47E-05
1.00E*05
1.OOE'01
1. OOE-03
l.OOC-02
tfa*
LONGITUDE
BROOKHflVEN NflTIONRL LflB
erfim
Figure 3.2 Continued.
3.2 Sulfur Wet Deposition Data
3.2.1 Data Screening and Calculation Procedures
In the ISDME, seasonal and annual wet sulfur deposition amounts predicted by
the regional sulfur deposition models over eastern North America for 1980, are
statistically compared with deposition amounts derived from observations. The
observed deposition amounts are derived from observations from the Air and
Precipitation Monitoring Network (APN) and Canadian Network for Sampling
Precipitation (CANSAP) in Canada and from the Electric Power Research Institute
(EPRI) Network, Multistate Atmospheric Power Production Pollution Study (MAP3S)
Network and the National Atmospheric Deposition Program (NADP) Network in the
United States. The APN, EPRI and MAP3S networks collect precipitation on a daily
basis, the NADP network on a weekly basis and the CANSAP network on a monthly
basis.
For a meaningful comparison between the predictions of the regional models and
network data, it is important that the wet deposition amounts calculated from the data
of the different networks are compatible, that the quality of or error in the observed
amounts are known and that only regionally representative sites are considered.
32
-------
Since the methods for quality control and quality assurance of the precipitation
chemistry data and the procedure for computing wet deposition amounts are not
always similar for the above mentioned networks, it was necessary to adopt a unified
procedure for screening and correcting the data and computing deposition amounts.
Also, since the wet deposition amounts estimated at the different monitoring sites may
not be of the same quality, a scheme was developed to classify the data into three
levels of uncertainty. The level of uncertainty is based on the deviation of the location
of a monitoring site from ideal siting criteria, on the percentage of data capture,
sampling efficiency and on the contamination from sea spray.
The methods of data screening/correcting, calculations of wet deposition amounts
and assessment of uncertainties, originally proposed for the ISDME by Voldner et al.
(1984), were refined and applied by the Unified Deposition Data Base Committee
(UDDBC) established under the Canadian Federal-Provincial Research and
Monitoring Co-ordinating Committee's Atmospheric Science Sub-Group
(RMCC/ASSG) (Unified Deposition Data Base Committee, 1985). Under the
direction of UDDBC these procedures were applied to the Acid Deposition System
(ADS) data base (Watson and Olsen, 1984), the official repository of North American
precipitation chemistry measurements.
With the cooperation of the UDDBC, overall data quality levels were determined
and sulfur wet deposition amounts were calculated for each season and the year of
1980. The three overall data quality levels were a function of the site
representativeness and data completion ratings.
Based on site characteristics and nearby sources, each site was rated as a 1, 2A, 2B
or 3, where 1 is considered to be regionally representative and 3 is not, rated according
to its assumed regional representativeness (Unified Deposition Data Base Committee,
1985). "Assumed" is used here to emphasize the fact that little is known about the
effects of "local" sources on the precipitation chemistry. The site rating criteria are
discussed in more detail in Appendix C.
In addition to the site representativeness ratings, the seasonal and annual data
were rated as 1, 2 or 3 on the basis of the percentage of the data captured, sampling
efficiency and the contamination from sea spray. The data with a rating of 1 is most
desirable while those with a rating of 3 are considered to be an inadequate
representation of the period. The data rating criteria are also discussed in Appendix
C.
Table 3.2 defines the UDDBC overall data quality levels. The UDDBC
recommended that only the data with an overall rating of 1 or 2 be used for model
evaluations. However, the stringency of the criteria combined with the start-up of
many monitors during the year resulted in an insufficient number of observations for a
statistically meaningful model evaluation. Only observations at five sites in the United
States passed the annual criteria for levels 1 or 2. In winter the situation was slightly
better with 18 observations in the United States and 4 in Canada qualifying. Clearly,
these deposition data with their uneven spatial distribution were inadequate for use in
the 1980 ISDME. Thus, it was necessary to relax the UDDBC criteria and accept
33
-------
TABLE 3.2 UDDBC (U) AND ISDME (I) OVERALL DATA QUALITY LEVELS.
Data Completeness
Level Site Representativeness Level
1 2
1
2
3
U
1
2
3
I
1
2
3
U
2
2
3
I
1
2
3
3
U
3
3
3
I
2
2
3
observations of potentially lower quality.
The deposition calculation procedures were not changed, however. The modified
UDDBC procedures of screening the precipitation chemistry data, the procedure of
calculating wet deposition amounts and an assessment of the uncertainties of the wet
deposition amounts are discussed in Appendix C. The differences between the ISDME
and the UDDBC's screening criteria were in the collection efficiency (ISDME criteria
were lower by 5-20%) and in the overall data quality data levels. The UDDBC overall
data quality level could not exceed a site's representativeness level. For the ISDME
however, sites with a representativeness level of 3 were assigned overall ratings of 2 if
the data completeness levels were either a 1 or 2 (see Table 3.2).
Similar to the UDDBC recommendation, only data with overall ratings of 1 or 2
were used in the ISDME. Therefore, the major difference between the evaluation data
base recommended by the UDDBC and that used in the ISDME was the ISDME
acceptance of those sites with a low regional representativeness rating provided that
their data completeness rating was a 1 or 2.
34
-------
3.2.2 Results of Data Screening and Calculations
Figure 3.3 illustrates the location of the monitors in the five networks and provides
their site representativeness levels. As discussed in Appendix C, level 1 represents a
"regionally representative" site, 2A and 2B "potentially representative" and "potentially
unrepresentative" sites, and 3 "unrepresentative" sites. Many of the southern Ontario
and eastern Canadian sites are level 3. According to Table 3.3, the majority of sites
are either level 2A or 2B.
Figure 3.3 Locations and levels of regional representativeness of the precipitation
chemistry sites operating in 1980 and passing at least the Level 3 criteria.
35
-------
TABLE 3.3 THE DISTRIBUTION OF SITES ACCORDING TO SITE
REPRESENTATIVENESS LEVEL.
2A 2B 3 Total
U.S.
Canada
Total
9
1
10
28
8
36
11
5
16
8
12
20
56
26
82
Since the spatial resolution of the model is of the order of 100 km, colocated or
nearly colocated (within 100 knV) sites were considered as representing one point
whose location was at the midpoint of the group of sites. The seasonal and annual
deposition amounts at these points are determined by averaging those amounts with
the highest overall data quality levels. If only one site in the group has the highest
overall data quality level, only the amount at that site would be used to represent the
point.
Figures 3.4-3.8 display the observed amounts of sulfur wet deposition (in units of
kg S/ha) for each season and the year at only those sites with a 1 or 2 overall data
quality level. These data comprise the ISDME evaluation data set. The figures also
display the overall data quality level for each of the level 1 and 2 sites. The number of
level 1 and 2 sites is provided in Table 3.4 for each season and the year. Depending on
the period, there are 32-46 evaluation sites.
The annual amounts at some sites in Figure 3.4 can differ from the sum of the
seasonal amounts as displayed at the same location in Figures 3.5-3.8. For these cases,
either colocated monitoring sites or sites within 100 km of each other were operating
during the year. Due to inconsistencies in the site representativeness and/or data
capture levels, the data from one site could be used for one season, while the data from
the second site could be used for the remaining seasons. If the levels were identical for
a particular season, the amount would be the mean of the two amounts. Providing that
the data from only the second site passes the annual screening criteria, the annual
amount used in the evaluation data set would be that for the second site. Therefore, in
this example, the sum of the seasonal amounts would not equal the annual amount.
36
-------
C
u 5
> '-3
"oW
Q, CUD
t/5^
in
-
.1=1
-
II
«« 53
S3
«»
C/5
"* S
rn.2
37
-------
-------
Figure 3.8 Annual 1980 amounts of sulfur wet deposition (kg S/ha) at the ISDME
evaluation sites.
TABLE 3.4 THE NUMBER OF SITES IN THE 1980 ISDME EVALUATION
DATA SET FOR EACH SEASON AND THE YEAR.
OVERALL DATA QUALITY LEVEL
Period
Winter
Spring
Summer
Autumn
Annual
1
24
24
30
27
13
2
24
22
15
15
19
TOTAL
38
46
45
42
32
39
-------
3.2.3 Differences in Amounts at Nearby Sites
For periods of a season or year, one might expect that the amounts of sulfur wet
deposition at sites within several hundred kilometers of each other would generally be
comparable. This is not always observed, as is pointed out in this section, which
highlights the unexpected differences in the amounts of sulfur wet deposition at sites
within 200 km of each other. The discussion refers to ISDME site identification codes
as defined in Table 3.5. Figure 3.9 illustrates the location of the ISDME sites.
\
Figure 3.9 The location and identification codes of the ISDME sites.
40
-------
TABLE 3.5 THE IDENTIFICATION CODES AND COORDINATES OF THE
ISDME SITES.
Site Code
U01
U02
U03
U04
U05
U06
U07
U08
U09
U10
Ull
U12
U13
U14
U15
U16
U17
U18
U19
U20
U21
U22
U23
U24
U25
U26
U40
Site Name
Scranton, PA
Giles Co., TN
Fort Wayne, IN
Lewisburg, WV
*
Whiteface, NY
Huntington, NY
Charlottesville, VA
Bradford For., FL
*
Bondville, IL
Champaign, IL
*
S. I. U., IL
Dixon Springs, IL
Hubbard Br., NH
Greenville, ME
Douglas Lake, MI
Kellogg, MI
Wellston, MI
Lamberton, MN
Meridian, MS
Jasper, NY
Washington Cr., NJ
Lewiston, NC
*
Coweeta, NC
Clemson, NC
Piedmont St., NC
*
Clinton St., NC
Finley A, NC
Finley B, NC
Raleigh, NC
Delaware, OH
*
Kane, PA
Leading Ridge, PA
Penn. St. U., PA
Trout Lake, WI
*
Lewes, DE
Indian River, DE
Caribou, ME
Network
EPRI
EPRI
EPRI
EPRI
MAP3S
NADP
MAP3S
NADP
NADP
MAP3S
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
NADP
EPRI
NADP
NADP
NADP
MAP3S
NADP
MAP3S
EPRI
NADP
Latitude(°)
41.6
35.3
41.0
37.8
44.2
44.4
44.0
38.0
30.0
40.1
40.1
40.1
37.6
37.7
37.4
43.9
45.5
45.6
42.4
44.2
44.2
32.3
42.1
40.3
36.1
34.9
35.1
34.7
35.7
35.5
35.0
35.7
35.7
35.7
40.4
41.0
41.6
40.7
40.8
46.1
38.7
38.8
38.6
46.9
Longitude(°)
76.0
86.9
85.3
80.4
74.1
73.9
74.2
78.5
82.2
88.4
88.4
88.4
89.0
89.3
88.7
71.7
69.7
84.7
85.4
85.8
95.3
88.8
77.5
74.9
77.2
83.1
83.4
82.8
80.6
78.6
78.3
78.7
78.7
78.7
83.1
78.2
78.8
77.9
78.0
89.7
75.2
75.0
75.3
68.0
41
-------
TABLE 3.5 THE IDENTIFICATION CODES AND COORDINATES OF THE
ISDME SITES.
Site Code
U41
U42
U43
U44
U45
U46
U47
U48
U49
U50
U51
U52
U53
U54
U55
U56
U57
U58
C01
C03
C04
COS
Cll
C12
C13
C14
CIS
C16
C17
CIS
C19
C20
Site Name
*
Knobit, NY
Stillwell, NY
»
Aurora, NY
Ithica, NY
Brookhaven, NY
Parsons, WV
Wooster, OH
*
Caldwell, OH
Zanesville, OH
Oxford, OH
Rockport, IN
Argonne, IL
**
**
Marcell, MN
Fayetteville, AR
**
Walker Br, TN
**
**
**
*
E. L. A, ON
E. L. A., ON
Chalk River, ON
*
Simcoe, ON
Long Point, ON
*
Kejimkujik A, NS
Kejimkujik B, NS
Trout Lake, ON
Pickle Lake, ON
Moosonee, ON
Dorset, ON
Harrow, ON
Peterborough, ON
Kingston, ON
Fort Chimo, QU
Nitchequon, QU
Chibougamau, QU
Network
NADP
NADP
NADP
MAP3S
MAP3S
NADP
NADP
NADP
EPRI
MAP3S
EPRI
NADP
NADP
NADP
NADP
CANSAP
APN
APN
CANSAP
APN
CANSAP
APN
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
CANSAP
Latitude(°)
41.9
42.4
41.4
42.6
42.7
42.4
40.9
39.1
40.8
39.9
39.8
40.0
39.5
37.9
41.7
40.5
43.5
47.5
36.1
35.5
36.0
37.5
34.2
35.2
49.7
49.7
49.7
46.1
43.0
42.9
43.0
44.2
44.4
44.4
53.8
51.5
51.3
45.2
42.0
44.2
44.2
58.1
53.2
49.8
Longitude(°)
73.8
73.5
74.0
76.7
76.7
76.6
72.9
79.7
81.9
81.8
81.5
82.0
84.7
87.1
88.0
91.0
89.8
93.5
94.2
89.0
84.3
84.0
86.0
84.4
93.7
93.7
93.7
77.4
80.6
80.3
80.8
65.2
65.2
65.2
89.9
90.2
80.7
78.9
82.9
78.4
76.6
68.4
70.9
74.4
42
-------
TABLE 3.5 THE IDENTIFICATION CODES AND COORDINATES OF THE
ISDME SITES.
Site Code Site Name Network Latitude(°) Longitude(°)
C23
C24
C26
C33
C34
Charlo, NB
Acadia FES, NB
Stephenville, NF
**
**
CANSAP
CANSAP
CANSAP
48.0
46.0
48.5
45.5
49.0
66.3
66.4
58.6
75.0
71.0
Site code and coordinates pertain to either colocated monitoring sites or a point
within 50 km and equidistant from two or more monitoring sites.
No monitoring sites are located at this location; modelers were requested to
provide predictions here.
As illustrated by Figure 3.4, the observed 1980 annual maximum deposition of 15.8
kg S/ha occurs at Harrow, Ontario (C15) at the western end of Lake Erie. Although
this value passed the screening criteria, it did seem to be higher than expected. In 1980
the winter and spring deposition amounts at this site were considerably higher than
those at the nearest sites. In the following year, the spring and summer amounts at this
monthly network site were greater than tnose amounts at three event network sites,
which were within 100 km of the former. (The latter network began operations in late
1980, hence data were not available for the ISDME). The Harrow site was classified
as a level 3 site (i.e., one that is regionally unrepresentative) due to its proximity to an
agricultural research station. Its operation was terminated at the end of 1982. Hence,
the value should be viewed with some caution.
The seasonal and annual maps of sulfur wet deposition indicate that the amounts
at some sites, even though they passed the screening criteria, consistently appeared to
be lower than those amounts at the nearest sites. For most seasons, as well as for the
year, the Scranton (U01) and the Lewisburg (U04) sites showed lower deposition
amounts than those at sites within a radius of 200 km. Also, for the three seasons when
data passed the screening criteria, the amounts at Jasper (U17) were also lower than
those at the nearest sites. These apparently lower amounts cannot be explained.
However, the lower amounts at the first two sites, which are EPRI network sites, could
be caused in part by gaps in the monitoring periods, which are not reported in the
event network data base. Also, since climatological precipitation patterns in
mountainous regions show high spatial variability, these three sites, which are located
in the Appalachian Mountain chain, could be located in climatologically low
precipitation areas. The Lewisburg (U04) site was discontinued in 1981.
Summer and annual amounts at Kejimkujik (C05) are considerably lower than at
the two New Brunswick sites (C23 and C24). These differences could reflect
differences in meteorological patterns and network siting and monitoring protocol.
43
-------
Although internetwork data inconsistencies have been observed here and reported
in studies mentioned earlier, data adjustments were not attempted in the ISDME.
Nonetheless, one should be aware that they do exist.
3.3 Air Concentration Data
For 1980, regionally representative air concentrations of both sulfur dioxide and
sulfate were measured simultaneously at only four sites, CO1, CO3, CO4 and CO5.
Obviously, these few measurements are not sufficient for a statistical evaluation.
However, since the climatological conditions at these APN sites and their relative
location to the high emission regions in North America are significantly different, a
simple comparison between model predicted and observed air concentrations provides
an indication of how well the models simulate transport, dispersion and removal
processes. Hence, weaknesses in model parameterization, potentially masked in the
evaluation of the models for only wet sulfur deposition could be revealed.
Seasonal and annual average air concentrations were computed from the daily
samples (Table 3.6). For every season, valid 24-hour samples were available for at
least half the days. Seasonally varying detection limit concentrations, determined by
Barrie et al. (1984) for each site, were used when appropriate. The sulfate data at the
maritime site, COS, were corrected for sea spray by reducing the sulfate air
concentrations by 0.25 of the sodium air concentration, as recommended by Barrie
(personal communication, 1986).
44
-------
TABLE 3.6 MEAN AIR CONCENTRATIONS OF SULFUR DIOXIDE AND
SULFATE AT THE FOUR APN SITES DURING 1980.
Site
ELA
(C01)
Chalk River
(C03)
Long Point
(004)
Kejimkujik
(COS)
Period
Winter
Spring
Summer
Autumn
Annual
Winter
Spring
Summer
Autumn
Annual
Winter
Spring
Summer
Autumn
Annual
Winter
Spring
Summer
Autumn
Annual
Sample Size
SO2 Sulfate
86
75
49
66
276
88
86
77
63
314
84
84
77
90
335
70
70
66
91
297
88
75
52
92
307
90
89
78
64
321
89
91
81
90
351
72
76
88
91
327
Air Concentration
S02
2.9
0.8
0.6
2.0
1.7
16.9
6.6
2.8
6.8
8.6
25.2
11.6
10.4
18.1
16.5
3.3
1.2
1.3
2.2
2.0
(Hg/m3)
Sulfate
1.3
1.0
1.0
1.2
1.2
2.1
2.4
2.8
1.8
2.3
3.9
6.0
7.6
3.0
5.1
3.5
3.2
4.1
1.8
3.1
45
-------
4. MODEL EVALUATION APPROACH
4.1 Introduction
Model development is a continuous process in that models are refined as new
theories of physical and chemical mechanisms evolve and parameterizations and input
data processing techniques are refined. Model evaluations can be vehicles in this
process since they can provide diagnostic information relating to the performance of a
specific version of a model or specific module. Model evaluations can also assess the
state of the art of modeling and identify strengths and weaknesses of the models.
A model evaluation should attempt to answer the following fundamental
questions:
1) Do the model predictions concur with observations?
2) Do the models predict these results for the right reasons? (i.e., were the
predictions obtained by means of credible physical and chemical
parameterizations?).
To answer the first question, quality assured measurements of those predicted
variables of interest are required at the same temporal scales for which simulation
results from the models are applicable (i.e., for seasons and the year). An operational
evaluation addresses only this question.
The second question can be best answered when continuous measurements and
predictions of all model predicted variables are simultaneously available for time
scales comparable to model time steps. This question is answered in a diagnostic
evaluation. The detailed data are required to assess the credibility of the physical and
chemical parameterizations and to compare the variability of the predictions with the
variability exhibited by the measurements in response to the variations of the pertinent
meteorological parameters.
Ideally, models should be evaluated via both operational and diagnostic
approaches. However, this is not always possible. Since in North America many of the
variables predicted by sulfur deposition models are not measured simultaneously, if at
all, a thorough diagnostic evaluation of the ISDME models was not feasible.
Therefore, the ISDME involved an operational evaluation accompanied by diagnostic
information to identify the differences between predictions and observations.
47
-------
4.2 Elements of the Model Evaluation
There is no universal set of evaluation criteria appropriate for all model
evaluations. Each set needs to be tailored to at least three factors:
1) The intended applications of the model results (i.e., how the model results are
to be used);
2) The spatial and temporal scales of the models; and
3) The predicted variables for which observations of sufficient quality, quantity
and spatial distribution are available for a statistically meaningful model
evaluation.
Regarding the first factor, the results from sulfur deposition models have been
used by ecologists and policy analysts to study
1) Patterns of seasonal and annual average air concentrations and wet/dry
deposition amounts;
2) Sulfur wet/dry deposition totals across states and provinces;
3) Source - receptor relationships;
4) Sulfur budgets; and
5) Sulfur fluxes across political and geographical borders.
At this time, the models cannot be evaluated for their source-receptor
relationships, since a reliable means of doing so has not been devised. The models
cannot be evaluated for their sulfur budgets and fluxes because of the lack of
appropriate dry deposition and horizontal flux measurements.
On the other hand, models have been evaluated for spatial and temporal patterns
of air concentrations of sulfur dioxide and sulfate using the 1977 and 1978 Electric
Power Research Institute's Sulfur Regional Experiment (SURE) data base (Perhac,
1978). However, models have yet to be adequately evaluated for sulfur wet deposition.
48
-------
Models can now be adequately evaluated for sulfur wet deposition amounts using
the precipitation chemistry data available for eastern North America after 1979.
Before 1980, the quantity and spatial distribution of the data rendered them
inappropriate for a statistically meaningful evaluation of long-term regional models.
The ISDME models were evaluated for seasonal amounts of sulfur wet deposition
according to the elements listed in Table 4.1. Seasonal periods were chosen since
these temporal scales are of interest to the model user community plus seasonal
evaluations are more rigorous than annual evaluations. Unlike its predecessors (see
Appendix A), the ISDME compares spatial patterns of predictions to those of
observations on commensurate temporal and spatial scales.
TABLE 4.1 ELEMENTS OF THE EVALUATION OF THE ISDME MODELS.
1) Replication of 1980 seasonal spatial patterns and magnitudes of sulfur wet
deposition amounts across eastern North America within data uncertainty limits
2) Prediction of the magnitude and location of the maximum seasonal sulfur wet
deposition amounts within data uncertainty limits
3) Replication of seasonal profiles of sulfur wet deposition amounts
4) Concurrence of aggregated seasonal and annual predictions
5) Reasonableness of predictions of seasonal average air concentrations and dry
deposition amounts of sulfur dioxide and sulfate
The first two elements in Table 4.1 were selected on the basis of the usual
application of the models the definition of seasonal and annual patterns of sulfur
wet deposition amounts. The third element was selected to assess the models'
capability in predicting greater wet deposition amounts for those seasons when the
observed data indicate greater wet deposition occurs. The fourth element is significant
if one is interested in both the seasonal and the annual results. For example, if both
the seasonal and the annual effects of emission fluctuations are to be assessed, it is
necessary that the model results of the four seasons add up to those obtained by an
annual model execution. Otherwise, two sets of results will be obtained.
49
-------
Finally, the last element was included to address, in a very limited sense, the
question, were the predictions of sulfur wet deposition amounts based on reasonable
predictions of other variables influencing the amounts? In other words, if a model
replicates the seasonal and annual observed patterns of sulfur wet deposition amounts,
how credible are the predictions of the air concentrations and dry depositions?
4.3 Model Performance Measures
4.3.1 Basis for Selection
Previous evaluations of sulfur deposition models relied primarily on the model
performance measures recommended by the American Meteorological Society (AMS)
(Fox, 1981). These model evaluations focused almost entirely on the comparison of
statistical distributions of observations and predictions at measurement sites (e.g.,
means, biases, standard deviations). Spatial analyses were generally restricted to
smoothed analyzed maps of data, location of maxima, spatial correlations and cross-
sectional plots.
An evaluation of sulfur deposition models which focuses on the distributional
statistics rather than the spatial analyses has severe limitations in that
1) The models are being evaluated on an incommensurate spatial scale (i.e.,
predictions, which represent averages across 104 km2 areas, are compared to
point measurements of a parameter that is highly variable in space);
2) Global distributional statistics (e.g., means, medians, variances, etc.) cannot
reveal the spatial characteristics of the data. Hence, two completely different
fields can have identical distributional properties;
3) The usual pairwise comparison of observations and predictions, in terms of
average differences and/or correlations, cannot reveal the capabilities of a
model to replicate spatial patterns of the observations;
4) When distributional statistics are used exclusively, those models which predict
flat fields (i.e., smooth patterns with small gradients) for parameters which
exhibit high spatial variations will often appear to perform well for an
incorrect reason;
5) In general, the regions of poor or good agreement between the predictions and
observations cannot be identified because of the widely spaced measurements.
50
-------
The AMS model evaluation recommendations and the spatial analyses used in
previous model evaluations alone are not sufficient to address the criteria presented in
Table 4.1. Consequently, the ISDME required the development of a new approach
with a greatly increased emphasis on spatial analysis. The ISDME approach uses
distributional statistics from the more traditional approach plus measures relating to a
robust spatial analysis (discussed in the next section) with an emphasis on the latter.
Spatial analyses can provide information not obtainable from distributional
statistics, such as:
1) An assessment of model performance in regions where observations are
unavailable, which is important when observations are widely spaced;
2) Specific features of the spatial distribution of the data that are important for
the models to replicate (i.e., magnitude, position and shape); and
3) The degree to which the predicted spatial pattern differs from the observed
pattern (i.e., by how much the predicted spatial pattern is translated or rotated
and how well the predicted location and magnitudes of the gradients and
maxima concur with the observed spatial pattern).
The ISDME model performance measures were selected on the basis of their:
1) Relevance to the criteria listed in Table 4.1,
2) Interpretability, and
3) Ability to objectively discriminate the behavior of the models.
These measures, listed in Table 4.2, quantify the differences between the spatial
patterns (specifically magnitude, position and shape), location and magnitude of
maxima and seasonal contributions of the observed and predicted amounts of sulfur
wet deposition. As discussed later, values of some of these measures are calculated
from predictions and observations either at ISDME evaluation sites or at interpolated
grid points. Statistical significance tests are used whenever applicable. For reasons
explained in Section 4.3.2 some of the measures in Table 4.2 are more reliable than
otners and have been designated as primary performance measures.
51
-------
TABLE 4.2 THE ISDME MODEL PERFORMANCE MEASURES.
I. Replication of Spatial Pattern
A. Magnitude
1. Area of significant differences
2. Difference between the medians
3. Difference between the dispersions
4. Bias in the high deposition region
5. Bias in the low deposition region
6. Mean square error
B. Position
1. Distance between the centroids
2. Difference in the orientation of the major axes
3. Distance of the predicted pattern shift to
achieve the maximum spatial correlation
C. Shape
1. Maximum spatial correlation resulting from
predicted pattern shifting and rotation
2. Spatial correlation without predicted pattern shifting
II. Maxima
A. Location
1. Distance of separation
B. Magnitude
1. Absolute difference
III. Replication of Temporal Pattern
A. Differences in seasonal contributions to the annual total
IV. Concurrence of Aggregated Seasonal and Annual Results
A. Relative differences between the aggregated seasonal
and annual predictions
* Primary model performance measures subsequently used to cluster the models
according to performance.
52
-------
4.3.2 Description of Measures
4.3.2.1 Spatial Patterns
The ability of the models to replicate the spatial patterns of the observations is
assessed by comparing the magnitudes, positions and shapes of the observed and
predicted patterns. The primary measure of pattern magnitude is the percentage of area
in each of the four subregions depicted by Figure 4.1 where the interpolated predictions
are significantly different from the interpolated observations. Interpolated values are
used since neither the observations nor the predictions from most of the ISDME
models are available with sufficient resolution to define the spatial patterns. The
subregions are defined on the basis of the geographic regions across which the models
tend to significantly under/overpredict. The unique advantage of this measure is that
subregions where the models perform best and worst are readily identified by a set of
quantities.
Figure 4.1 The four subregions where significant differences between interpolated
predictions and observations are identified.
53
-------
The percentages of subregional areas where significant differences occur are
determined by examining pairs of interpolated observations and predictions at each
half-degree grid cell within each subregion. Significant differences are detected when
the prediction falls outside the uncertainty range about each interpolated observation,
as determined via a technique discussed in Section 5. The number of cells with
significant differences is determined and then divided by the number of cells in the
subregion. The percentage of cells in the composite region where significant
differences occur is also determined.
The next two measures of pattern magnitude relate to the distribution of the
observations and predictions at the ISDME evaluation sites. The medians and
variances, which are measures recommended by the AMS (Fox, 1981) are used to
characterize the central tendency and dispersion of these values. Medians are better
measures than means of the central tendency of the distributions because of the
potential effect of outliers on the asymmetry of the distributions.
Statistical tests are normally applied to assess the significance of the differences in
the medians and dispersions of the predicted and observed distributions. The "p value"
associated with these tests is the probability that the differences occurred by chance.
Thus, a p value of 0.05, which signifies that the difference is extremely unlikely by
chance, indicates that the difference is significant at approximately the 95% con-
fidence level.
The Mason-Whitney test (Conover, 1971) is used to assess the differences in the
medians. The nonparametric test avoids the necessity for making assumptions
concerning the form of the distribution. For the comparisons of the dispersion of the
distributions Bartlett's test (Snedecor and Cochran, 1967) is used. However, both
significance tests assume that the data are statistically independent, which is not the
case here, since the sulfur wet deposition data are spatially correlated. Therefore,
these significance tests cannot be applied to determine the probability that the
differences between the observations and predictions were obtained by chance.
However, they provide information on the relative performance of the models.
A statistical test is also used to assess the significance of the biases at the ISDME
evaluation sites in both the high and low deposition regions illustrated by Figure 4.2.
The high deposition region was arbitrarily defined as that region containing ISDME
evaluation sites at which the annual sulfur wet deposition amount generally exceeds 10
kg S/ha. This value is chosen in an effort to partition the data with regards to the
amount of sulfur wet deposition while retaining enough sites to characterize the
distributions of the amounts in the two regions.
The bias, also recommended by the AMS, measures the general tendency of the
model to under/overpredict. It is defined as
54
-------
Figure 4.2 The ISDME sites in the "high deposition region" as denoted by the circles.
N
N
where Oj and Pj are the observation and prediction at site i and N is the number of
sites. The assessment of the statistical significance of the bias is normally based on the
Student's t statistic with N-l degrees of freedom. However, due to the spatial
correlation of the data, this test cannot be used as was originally intended. Like the
results of the significance tests for the median and dispersion, the results of the
Student's t test are presented as a means of qualitatively comparing model
performances.
55
-------
The mean square error,
N
_
N
also recommended by the AMS, measures the squared differences between the
observations and predictions at the ISDME evaluation sites. As a consequence, this
measure shows a sensitivity to outlier predictions, or those that are very different from
the observations. The mean square error was selected as a measure of model
performance in anticipation of the possibility that a set of predictions could have a very
low bias, yet have large, but counterbalancing, differences between the predictions and
observations. Since this measure calculates the mean square differences, no
counterbalancing effects can occur.
The pattern position of the predictions is compared to that of the observations via
three measures. The first measure, namely, the distance between the observed and
predicted centroids, characterizes one aspect of the spatial distribution of the sulfur wet
deposition amounts. If the spatial distribution of the data is represented by a three-
dimensional surface, where the height of any point on the surface is proportional to the
magnitude of the parameter, the center of mass, or centroid, is the point projected
onto the geographic plane at which the surface is balanced. The comparison of the
observed and predicted centroids not only provides information on the relative
location of the center of mass of the predicted patterns, but it quantifies one aspect of
the differences in the observed and predicted patterns.
The centroids are computed from a grid of interpolated values across the region of
high deposition. Site values are not used since values are available only at points of
measurement clustered in scattered pockets across eastern North America. The
spatial distribution of the points would therefore greatly influence the location of the
centroid. Since the interpolated values are available at evenly spaced grid nodes, the
centroids determined from these values would be more representative of the spatial
distribution of the data.
From the N pairs of interpolated observations and predictions, the centroids are
computed as deposition-weighted averages of the longitude (L WjXj) and the latitude
(E Wjyj), where the weights are defined as
N
D,
and Dj is the amount of sulfur wet deposition at point i.
56
-------
This subset of the modeling domain was selected to focus on the high deposition
region. Consideration of the lower amounts of sulfur wet deposition across the regions
to the north, south and east would have influenced the location of the centroids by
virtue of their higher percentage of the total domain rather than their magnitude.
The second measure, the difference between the angles of the major axis of the
observed and predicted patterns, quantifies the degree of concurrence in the
orientation of the observed ridge of high amounts of sulfur wet deposition. From the
observations and predictions at all ISDME sites but the six northernmost Canadian
sites (Cll, C13, CIS, C19, C20 and C22), the orientation of the major axes is
determined via principal component analysis. The six northern Canadian sites are not
considered here since the focus of attention of this measure was the high deposition
region and the immediate surroundings.
The third measure involves the distance of the predicted pattern shift to maximize
the spatial correlation between the observed and predicted patterns. Since visual
comparisons of interpolated observed and predicted patterns indicate that the
predicted patterns are shifted to the south and east of the observed patterns, this
measure is included to objectively quantify the apparent shift in predicted patterns.
This measure is calculated as follows: First, each of the interpolated grids of
predictions is shifted in all cardinal directions in one-degree increments. After each
incremental shift, the spatial correlation is computed, resulting in a grid of spatial
correlations with the coordinates representing the direction and magnitude of the shift.
Finally, the distances of the pattern shifts to maximize the spatial correlation are
determined. The effects of pattern rotation are not considered here. Further
exploration of this measure is needed before it is recommended as a model
performance measure. The correlations were computed from the interpolated values
rather than from the site values to compensate for local variability and errors in the
measurements.
The pattern shape of the predictions is compared to that of the observations via the
maximum spatial correlations resulting from pattern shifting. The spatial correlation
computed before pattern shifting is also a measure to quantify the degree to which the
models replicate the observed pattern shape without shifting the pattern. Both the
Pearson product moment and Spearman rank correlations are computed. Since the
results are very similar, only the values of the Pearson correlation coefficients are
provided in this report.
When relying on the spatial correlation as a measure for pattern shape replication,
pattern shifting may be necessary. The shapes of the predicted and observed patterns
can be identical, yet an offset in the patterns can yield very low spatial correlations. By
shifting one of the patterns an appropriate distance, the resulting correlation of 1.0
would indicate the perfect pattern shape replication.
As mentioned earlier, pattern rotation is not considered. Consequently, greater
correlations would be expected for those patterns of predictions with large differences
57
-------
in the orientation of the major axis. Because the effects of pattern rotation arc not
considered, the maximum correlation is not used as a primary model performance
measure.
4.3.2.2 Maxima
The differences in the location and magnitude of the maximum observed and
predicted amounts are additional measures chosen to assess model performance for
two reasons. First, the magnitude and location of the maximum deposition amounts
are of interest to model users and secondly, the location and magnitude of the
maximum amounts are quantifiable, within uncertainty limits.
The actual location of the observed maximum is uncertain because of the widely
spaced monitoring sites and the unavoidable errors in the measurements and
interpolations. This uncertainty must be considered in computing the distance
between the observed and predicted locations of the maxima.
Two approaches, both of which consider uncertainties, were considered in
determining these distances. One approach relies on spatial analyses, while the second
approach relies only on the values at the ISDME sites. The first approach was more
desirable because of its intrinsic advantages over the alternative approach.
First of all, since it interpolates the values between sites, this approach uses a
representative analysis of the regional pattern, while the second approach considers
only values at a very limited number of points that may or may not be regionally
representative. Furthermore, the configuration of the monitoring network does not
influence the distances calulated from the spatial analyses, since values are available at
evenly spaced grid nodes.
As an example of the differences between the two approaches, if there are no
monitoring sites between the site of maximum prediction and the site of maximum
observation, separated by 200 km, the second approach would conclude that the model
was in error by 200 km. The interpolated approach would determine a geographic
domain within which the maximum observed value is likely to occur. The error or the
model in predicting the location of the maximum would then be the shortest distance
between the site of maximum prediction and this domain. The computed distance
would not be dependent on the network configuration.
Another advantage of the interpolation approach is that it objectively quantifies
the uncertainty about each grid node. Since the uncertainty assessment is based on the
spatial variability of the data across the entire domain, as well as measurement errors,
this approach is intrinsically capable of estimating local effects (e.g., orographical
effects) on individual site values. The alternative approach assumes that the
uncertainty at each site is the same.
58
-------
One must be careful in applying this rigid approach since the interpolation maps
could yield a predicted maximum which approximates the magnitude of a distant
secondary peak value. Both values could conceivably be within the uncertainty bounds
of the maximum observation, yet this approach would consider only the location of the
absolute maximum value. For example, say the absolute maximum value is predicted
over Ohio and a secondary maximum value is predicted over Nova Scotia, which in this
case coincides with the maximum observed value. Even if the difference between the
magnitudes were insignificant, this approach would calculate the distance between the
Ohio and Nova Scotia locations. Given the uncertainties of the observations, it would
be more appropriate to conclude that the model predicted the location of the
maximum. Consequently, it is necessary to search the field for other peaks falling
within the uncertainty bounds of the maximum observed value. This was done for each
period of the ISDME; distances between the observed and predicted maxima were not
affected.
Two approaches were also considered in quantifying the differences in the
magnitude of the maxima, A mx. As before, both approaches consider estimated
uncertainties. The interpolation approach computes
Pmx - (°
mx
mx
0 if P
mx
IfP
mx
mx
- 2°i) - pmx or
0 if P
mx
where Omx and Pmx are the magnitudes of the interpolated maximum observation and
prediction, respectively, and the 2oj is the uncertainty estimate at the point of the
maximum observation.
Considering data limitations, it is highly uncertain that the differences are real
when the interpolated maximum prediction falls within the uncertainty envelope of the
interpolated maximum observation. For these cases, the model is deemed to be on
target.
The second approach computes the percentage difference between the maximum
predicted site value and the maximum observed site value ±20%. The model is
deemed to be on target if the predicted maximum falls within ±20% of the observed
maximum.
The results of both approaches, which normally are very similar, are compared in
Section 6.3. Since the uncertainty estimate of the interpolation approach is derived
objectively for specific points, as compared to the subjectively determined universal
59
-------
uncertainty estimate of the alternate approach, the former approach is thought to be
more reliable. Therefore, the interpolation approach is used to assess the performance
of the models.
4.3.2.3 Temporal Patterns
Finally, since predictions should replicate the observed seasonal temporal pattern
of sulfur wet deposition amounts (discussed in Chapter 3), for each model the average
percentage of the annual amount of sulfur wet deposition is computed for each season.
Only the data are used from those 30 ISDME evaluation sites where both observations
and predictions are available for every season. The locations of these sites are denoted
by the larger symbols in Figure 4.1.
4.4 Model Clustering
As discussed in Section 1.2, one of the ISDME goals is to determine which models
perform best in replicating the seasonal patterns of sulfur wet deposition. A set of
model performance measures has been identified to assess the performance of each
model. This section discusses the approach of utilizing the measures to cluster the
ISDME models according to their overall performance in replicating observed 1980
seasonal patterns of sulfur wet deposition.
There are several approaches to clustering the models according to overall
performance. One approach, referred to as model ranking, would be to first rank the
models for each performance measure. For example, the model with the lowest gross
error would receive a ranking of 1 and the model with the highest gross error would
receive a ranking of N, where N is the number of models. Absolute differences in
magnitudes would be ignored.
The overall performance of the models would be based on the rankings for each
performance measure. Those performance measures thought to be most significant
could be weighted more than the others, or the performance measures could be
weighted evenly, as was the case for the recent DOE/AMS evaluation of mesoscale
atmospheric dispersion models (Weber and Kurzeja, 1985).
A second approach, referred to here as the relative performance approach, would
be to first cluster the models into several categories of similar values for each
performance measure. The discriminating values could be determined subjectively or
on the basis of statistical significance tests. The overall performance of each model can
then be determined by adding scores for each measure, where a score of 1 could be
assigned to the models in the "best" category, a score of 2 to the models in the "second
best" category, and so on. Performance measures with greater significance can be
weighted more than other performance measures.
60
-------
Finally, a third approach, referred to as the performance standards approach, would
be to first determine whether each model passed or failed a standard for each
measure. The standards would be based on modeling expectations. The overall
performance of the models could be determined by the number of performance
measures for which the model passes the standard. Like the other two approaches, the
performance measures could be weighted.
Each approach has its advantages and disadvantages. For the model ranking
approach, there are nonparametric statistical tests available to test the significance of
the differences in the overall rankings. However, since the model ranking approach is
strictly ordinal, it is impossible to interpret the meaning of the ranks with no other
information. More specifically, the ranks are not commensurate for each performance
measure. For instance, suppose N models are ranked for each of M performance
measures. For Measure A, me differences between the values are very small. On the
other hand, for Measure B, the range of values is very large. Given only model
rankings for each measure, one would not be able to interpret the differences between
the pair of models with overall rankings of 1 and N.
The disadvantage of the second approach is that the discriminating value of the
performance measures are chosen subjectively. Therefore, the conclusions might not
be unique. Also, in cases where there is an even distribution of the values,
discrimination is difficult. If, in addition, the distribution is such that the range of
values is large, one would be reluctant to assign the same score to all the models.
Another disadvantage of this approach is that one cannot distinguish whether the
models in the "best" category are exhibiting acceptable behavior for any measure. As
an example, consider the extreme case where all models significantly overpredict in
80% of the evaluation region. This behavior is unacceptable, yet all models would be
assigned a score of 1.
Finally, the performance standards approach has the advantage that the
performance of the models is assessed and can be intercompared on the basis of a
known standard value. Therefore, one can relate the two groups of models to the
model expectations. Furthermore, the weaknesses and strengths of each model are
more easily identified. Although the standards are subjectively defined, they can
reflect the expectations of the model users (e.g., NAPAP has declared that predictions
of sulfur wet deposition should be expected to be within ±50% of the observations).
For the ISDME, the last approach is used to cluster the models according to
overall performance. Model performance standards are prescribed for each season
using the primary performance measures in Table 4.2. The other measures of Table
4.2 were not used to cluster the models because of intrinsic weaknesses or because they
are in an exploratory stage.
Table 4.3 lists the criteria values for the primary performance measures. The
criteria values of the last five measures in Table 4.3 pertain to maximum allowable
differences between the predictions and observations. These values approximate the
61
-------
medians of the distribution of the models' seasonal values of each measure.
Approximately half of the ensemble seasonal values are either less than or greater than
these criteria values. Since the criteria values are the same for each season, it is
therefore possible to compare the seasonal performances of the models as a whole.
There are no criteria values for annual performance, since the overall model
performance is based strictly on the seasonal performances.
The seasonal performance of the models is determined by the number of primary
model performance measures passing the criteria. All but one of these measures
receive a weight of one. The measure of the area of significant differences receives a
weight of two or four by virtue of its assessment of
1) The significance of the differences between predictions and observations at
every half-degree cell and
2) More than one attribute of the spatial pattern (i.e., it assesses differences in
pattern magnitude, position and snape).
62
-------
TABLE 4.3 THE CRITERIA VALUES OF THE PRIMARY MODEL
PERFORMANCE MEASURES USED TO CLUSTER THE MODELS.
MEASURE
CRITERIA VALUE
SCORE
1. Area of Significant Differences
2. Centroids
3. Major Axis Orientation
4. Maximum Location
5. Maximum Magnitude
6. Seasonal Pattern
<. 20% in composite 4
region and < 25%
in each subregion
OR OR
<. 20% in composite 2
region and _>. 25%
in one subregion
<_ 75-km discrepancy 1
<. 10-degree discrepancy 1
<_ 100-km discrepancy 1
<. 30% discrepancy 1
_<. 15% discrepancy 1
Maximum Score
The overall performance is determined by a weighted average of these seasonal
scores. This set of weights reflects the observed contribution of each season on the
annual sulfur wet deposition. Finally, the models are subjectively clustered into several
groups according to their overall model performance scores.
63
-------
5. SPATIAL INTERPOLATION
5.1 Introduction
In evaluating the performance of an air pollution model, it is essential to assess its
ability to replicate validation data not only in probability distribution, but also in
spatial pattern. In an effort to obtain information regarding spatial accuracy, past
evaluations have utilized residuals, computed as differences between model
predictions and validation data taken at monitoring sites. The analysis of residuals
alone may be misleading for several reasons. First of all, field data have errors
associated with them which models can not (and should not) be expected to mimic.
This uncertainty should be taken into account in comparisons with model outputs. In
addition, there is the problem of the "regional representativeness" of the sampling sites.
A site may vary considerably from its neighbors because of a local emissions source, or
the local variability associated with meteorological events. This variation is often large
enough to reduce one's confidence in an individual site's ability to accurately represent
a larger area.
Because of the uncertainty in individual site data, a more reasonable approach for
evaluating model performance is to compare the spatial patterns derived from model
outputs and from observed data. These interpolated patterns retain the general spatial
features of the data by smoothing through random measurement error and local spatial
variation. Associated with the estimation of spatial patterns, there is an interpolation
error which results from these sources of variability. Unfortunately, most of the
available interpolation procedures are undeterministic and provide no estimates of the
error associated with their estimated spatial patterns. However, the statistical
techniques of regression and kriging provide uncertainty estimates for interpolated
surfaces. In the case of kriging, the estimates of error, as well as the smooth pattern
produced, are based on the spatial covariance structure of the data. The approach
makes use of weighted estimates at each interpolation point which are a function of an
estimated covariance. Linear regression, on the other hand, presupposes an
interpolation function whose parameters are estimated from the available data. The
method produces as well a weighted estimate at each interpolated point whereby the
weights are a function of its actual location as well as the estimated parameters.
One of the principal uses of kriging in the model evaluation process is to provide
estimates of spatial patterns for validation and model prediction data which are then
compared relative to the uncertainty in the patterns. However, this comparison does
not distinguish between the components of spatial pattern, namely, shape, positioning
(orientation and location) and magnitude.
It is important to disaggregate these spatial pattern features as much as possible in
order to fully evaluate a model's performance. For example, the shape of the spatial
surface produced by a model's outputs may be identical to the validation data pattern,
65
-------
but the overall comparison is unfavorable due to an error in scaling. Similarly, a model
may replicate the shape and magnitude of the validation data pattern, but fail in an
overall pattern comparison because of a translational shift of its pattern. In order to
investigate these types of model behavior, correlational measures have been utilized.
5.2 Selection of an Interpolation Method
In the analysis of a model's spatial accuracy, this evaluation concentrates on the
similarity of spatial patterns which characterize validation data and model outputs.
Since model outputs are relatively smooth, the patterns produced by different
interpolation algorithms are similar. However, characterizations of the validation data
can vary dramatically. The validation data exhibit a considerable amount of
nonsystematic variation that can result in different characterizations among
interpolation methods. Clearly, it is extremely important to utilize an interpolation
procedure that provides the best possible characterization of the data in order to fairly
evaluate the models. The procedure chosen should preserve the main characteristics
of the data by smoothing through random variability. It should not be unduly
influenced by single points, but should retain important spatial features. Ideally, the
degree of variability in the data should influence the degree of smoothing, and some
means should be available to represent the uncertainty in the interpolation estimates.
Most interpolation techniques do not have these characteristics. They are in
general deterministic, and do not use data weighting schemes that utilize the degree of
variability in the data. Most importantly, the user of these methods has no way to
ascertain the accuracy of the characterizations that they produce. For this evaluation,
we investigated two statistical techniques, standard least squares regression and a
geostatistical method known as kriging.
Least squares regression was rejected for several reasons. First, it was felt that the
method oversmoothed the data and lost a great deal of spatial detail (see Figure 5.1,
for example). In addition, it would be difficult to justify the critical assumption of
spatial independence of regression errors. Failure to meet this assumption would cast
doubt on the variance estimates that were produced. In particular, a positive distance-
related correlation could result in serious underestimates of variance and
inappropriately narrow confidence intervals for the estimated pattern. If this occurred,
models would be compared against an overly restrictive standard. The net effect
would be negative assessments of model performance when model predictions were
within the true uncertainty limits associated with the validation data. However, this
deficiency might be offset by the enhanced residual error associated with an
oversmooth regression surface. Indeed, this overestimate of interpolation variance
could far overshadow the underestimates arising from spatially correlated data,
thereby leading to a lenient standard. In order to better understand the magnitude of
interpolation errors associated with the least squares and kriging, the interpolation
variance estimates for the two methods were compared. Kriging produces lower
estimates of variance in nearly all regions.
66
-------
LEAST SQUARES REGRESSION
INTERPOLATION BY KRIGING
Figure 5.1 Comparison of the interpolated patterns for annual 1980 amounts of
sulfur wet deposition (kg S/ha/yr) using a kriging technique and a standard
least squares regression technique.
67
-------
This inability of standard least squares regression methodology to sufficiently
replicate the spatial surfaces is responsible for these large variance estimates. This
serious drawback precluded the use of standard regression techniques as a means to
estimate spatial patterns. Weighted least squares was considered as an alternative to
address trie correlated nature of the data. However, its results would also be
questionable due to the instability of the variance-covariance matrix estimated from
the small amount of available data.
A geostatistical interpolation method, known as kriging, provided an attractive
alternative. It is based on G. Matheron's theory of regionalized variables, which was
designed to mathematically describe phenomena distributed in space and/or time and
provide an appropriate means for solving spatial estimation problems. Matheron
named his estimation procedure in honor of D. H. Krige, a South African mining
engineer specializing in ore estimation. This technique has been used most extensively
in the mining industry but has also been applied in a wide variety of earth sciences
research (see Barnes, 1980, for descriptions). Analysts of acid precipitation data have
found the technique particularly useful. Recent investigations using kriging include an
analysis of spatial trends in hydrogen ion concentration in New York State (Bilonick,
1983), an evaluation of the performance of kriging in interpolating acid precipitation
parameters in the eastern United States (Finkelstein, 1984), and a study of
temporal/spatial patterns of pH and rainfall (Eynon and Switzer, 1983).
Kriging departs from traditional deterministic interpolation methods in that the
determination of interpolation weights depends on the cpvariance structure of the
regionalized variable (in this application, wet sulfur deposition). A basic and critical
part of kriging is the estimation of the model of this structure, which is known as the
semi-variogram. It is defined as the difference between the underlying variance of the
spatial process and the covariance between values at given distance. The semi-
variogram is given by
Y(h) = Var - Cov ( z(xj + h) - z(xj) )
where z(xj + h) and z(x:) are measurements at points separated by a distance of h units,
"Var" is the variance of the regionalized variable, and "Cov" is the covariance.
The semi-variogram is estimated from the sum of the squared differences of all
sample data values. The estimated semi-variogram is given by
N(H)
Y(H) = 1 £( z(xi + H) - z(Xi) ) 2
2N(H) i=i
where H is a distance category dictated by the range of data; and N is the number of
points in the distance category. The spatial configuration of the data locations and the
interpolation site is used in combination with the semi-variogram to obtain optimal
interpolation weights. The estimator that results from this procedure has several
68
-------
appealing characteristics. First, it is an exact interpolator: If a known point is
interpolated, the exact value of the point will be reproduced. This is not true of all
interpolation procedures (e.g., least squares). In statistical terms, the kriging estimator
is also the best linear unbiased estimator. This implies that the estimator gives no
systematic error (over or underestimation) and has the smallest estimation variance
among unbiased estimators that are linear functions of the data. In addition, kriging
provides an estimate of the variance between the true and interpolated values. By
comparison, regression provides the best linear unbiased estimation of the parameters
of the regression equation.
Kriging requires that the semi-variogram be modeled in order to estimate the
weights. The semi-variogram is usually an increasing function of distance. If the
variogram's value is not zero at the origin, then the intercept on the y-axis represents
an estimate of the measurement error variance. In other words, it is the variance
obtained from co-located sites. When kriging is applied to the predictions, the error at
a co-located site is zero since a model cannot predict different depositions at the same
site. On the other hand, when kriging is applied to measured depositions, it is possible
to exhibit variability at the same grid site. This in fact is the case at sites such as
Kejimkujik, Nova Scotia (COS).
Kriging is not a single method, but rather a family of interpolation techniques.
"Simple" kriging is dependent upon the assumption that the underlying process is
viewed as stationary, that is, the expected value of the process does not vary radically
over the distances between observed and interpolation points. However, if the variable
contains a strong trend component, simple kriging can lead to a biased estimate of
interpolation variance. To redress the nonstationary problem and to account for
trends, "universal" kriging has been developed. The method involves the introduction
of polynomial trend terms into the system of equations used to determine optimal
interpolation weights. Unfortunately, this technique relies on correctly predefining the
polynomial form of trend and using a semi-variogram that does not include the trend
component in its estimation. The estimation of trend-free semi-variograms has been
addressed (Huijbregts, 1970; Sabourin, 1975), but has not been satisfactorily resolved.
The necessity of having to preselect trend forms is also a definite drawback to universal
kriging. If the wrong form is chosen, the bias in the interpolation variance may be
worse than that obtained from simple kriging. For a general discussion of kriging, see
Delhomme (1978) and Clark (1979).
Because of the difficulties in applying universal kriging, we have chosen to use
simple kriging. As indicated above, me presence of a trend in the data may result in a
biased estimate of the interpolation variance. However, in situations of moderate
trend, as in the validation data, the actual interpolation estimates are not severely
affected, and the seriousness of the bias in the interpolation variance is not great
(Journel and Huijbregts, 1978). In addition, the direction of the bias in the
interpolation variance estimates is clear. If anything, the variances produced by simple
kriging will be too large. Therefore, comparisons that rely on the interpolation
variance estimates will be potentially less stringent than they should be. Thus, if a
model's output deviates from the confidence limits of the interpolated validation data,
one can be quite certain that the deviation is significant.
69
-------
A further point which needs to be mentioned is that the spatial resolution, or grid
size in interpolation must be prescribed in advance. Care must be taken to ensure that
it is neither too large so as to smooth the data too much, nor too small so as to produce
a map with more detail than the data permit. The final choice is usually a matter of
subjective judgement though it is based in part on the resolution of the validation data.
The degree of smoothing in kriged analyses is, however, less dependent on grid
spacing, since the weighting scheme is determined separately for each interpolated site.
In this study, grid sizes of l/4° by 1/4°, 1/2° by 1/2°, and 1° by 1° yielded very
similar patterns of the validation data.
Finally, in order to ensure that kriging produces a faithful representation of the
grid patiern, a comparison was made between the gridded predictions provided by
TVA, RELMAP and RTM and the kriged values determined from the model
predictions at only the ISDME sites. Ratios of gridded to kriged values were
computed for the annual data. In general, there was excellent agreement (i.e., less
than a 10% overall error) except in those regions where there were no ISDME sites (as
was the case in the vicinity of Sudbury, Ontario). On the basis of this information, we
deduced that kriging provides a reasonable representation of the grid pattern.
5.3 The Use of Kriging in the Model Evaluation
The primary purpose of kriging in evaluating models is to develop spatial patterns
for the observed data against which model predictions could be compared. Monitoring
site data were utilized to estimate the semi-variograms. Using these variograms, a
surface representing these data was estimated by interpolating the data values to a
1/2° by 1/2° grid covering eastern North America. This region is confined to 32° N -
58° N latitude and 54° W - 94° W longitude and includes approximately 4000 grid
cells.
Ideally, one would compare the predictions to interpolated observations at each
grid node. Since many ISDME models did not provide gridded predictions, the
ISDME site predictions provided by each model were kriged in the same manner to
produce consistently derived surfaces. At each grid node, the interpolated observed
and predicted values were compared, relative to the uncertainty in the interpolated
values. The relative magnitude of the uncertainty across the high deposition region
(depicted by Figure 4.2) was within ±50%, as shown in Figure 5.2. Measurement
errors accounted for approximately one-fourth to one-third of the total uncertainty
(Dennis and Seilkop, 1986). The estimates of the measurement errors, which were
based on the semi-variogram fit to the observed data, are consistent with the
differences observed in colocated sampling data (see Appendix C).
70
-------
100
Figure 5.2 The uncertainty estimates (%) of the kriged values of annual 1980
observations of sulfur wet deposition.
A confidence interval for the difference in the observed and predicted values was
constructed as follows:
Ypred - Yobs i1-96 t Var (Ypred) + Var(Yobs) ^
where
Ypred = kriging estimate for the model prediction at the grid node
Yobs = Arising estimate for the observed data at the grid
node
Var = kriging variance estimate.
71
-------
If the deviations of the kriged values from the true values that they estimate have a
normal (Gaussian) distribution, this is a 95% confidence interval. Failure of the
confidence interval to include zero can be viewed as strong evidence that the model
prediction would not be the same as the observation at this grid node. The percentage
of a geographic region in which the confidence limits at the grid nodes excluded zero
was used to assess the model's performance.
In order to verify the assumption that the kriging residuals of the observed data
were normally distributed, a cross-validation analysis was performed. That is, each
observed value was estimated by kriging the remaining values. The empirical
distribution of residuals was then compared to the normal distribution using the
Shapirq-Wilk test (1965). In all cases, the p-values for these statistics were larger than
0.25, giving little evidence to suggest a departure from normality. Histograms
depicting the distributions of kriging residuals for the summer and annual validation
data are shown in Figure 5.3. For the high deposition area, the histograms clearly
suggest a normal distribution. Normality is not as clear for the low deposition area,
where interpolation "edge" effects are likely. Based on the results of the statistical tests
for normality of the residuals from kriging both observations and model predictions, it
is not unreasonable to view the confidence intervals that were used as being at
approximately the 95% level.
The same procedure was carried out for the model predictions. With the
exception of RELMAP and ASTRAP, there was little evidence to suggest that the
residuals were not normally distributed. For RELMAP and ASTRAP the departure
from normality was relatively minor. The distributions of the residuals arising from
kriging these model predictions were symmetric but longer tailed than expected in a
normal distribution.
A problem which could arise from kriging is associated with the fact that some
models do not produce "smooth" patterns of predictions. For these cases, the proposed
confidence interval will be inflated in size because of disproportionately large
interpolation errors for the model predictions. Therefore, the more variable model
prediction pattern will be less likely to significantly deviate from the observed pattern.
The magnitude of acceptable model deviations in this case is not only influenced by the
uncertainties of the observations, but also by the uncertainties in the estimates of the
model predictions. This clearly represents an unfair advantage over models with
smooth predictions.
According to Table 5.1, interpolation errors in characterizing annual prediction
grid values for most models did not exceed 25% of the interpolation errors associated
with estimating the observation pattern. The magnitude of the interpolation errors for
the annual RELMAP predictions were similar to, while those for ASTRAP were
approximately 50% of, the magnitude of the interpolation errors for the annual
observations. Judging from the results discussed in Section 6, the large annual
interpolation errors for RELMAP could be attributed to two seasons - summer and
spring. On the other hand, the relatively large annual interpolation errors of ASTRAP
could be attributed to each season. Consequently, the results of the spatial analysis of
RELMAP predictions in summer and spring and those for ASTRAP for each season
were interpreted in light of the inflated size of the confidence interval.
72
-------
r
ui
rr
O
Q_
UJ
I
O
I
O
00
en
a:
UJ
5
00
f;
CO
O
CL
UJ
Q
O
I
o
03
cn
or
UJ
5
GO
00
00
c
o
o
ex
-------
TABLE 5.1 COMPARISON OF KRIGED VARIANCES FOR 1980 ANNUAL
AMOUNTS OF OBSERVED AND ESTIMATED SULFUR WET DEPOSITION.
SITE OBSERVATION
(KG S/HA)
U01
U03
U04
U05
U06
U08
U10
U19
U20
U21
U22
U23
U24
U26
U41
U42
U43
U44
U45
U46
U48
U52
C01
C03
C04
COS
C13
C14
C15
C16
C17
C19
C20
C22
C23
C24
5.57
8.63
5.33
7.47
7.03
9.93
7.13
6.23
8.53
10.07
6.23
14.40
9.25
6.93
6.27
8.33
6.23
14.53
11.00
13.47
8.67
2.67
2.05
9.00
11.67
5.83
2.80
11.03
15.77
12.20
13.50
2.77
5.13
3.53
9.20
9.13
KRIGED
ESTIMATE RESIDUAL
(KG S/HA) (KG S/HA)
7.65
13.17
11.34
9.08
8.83
7.81
6.98
6.25
9.25
6.51
7.48
11.24
10.25
6.23
6.58
8.42
5.97
8.53
15.12
11.99
9.70
3.70
2.38
10.31
12.67
8.45
4.62
10.92
11.00
11.70
9.62
4.14
5.60
4.80
7.33
7.44
-2.08
-4.54
-6.01
-1.61
-1.80
2.12
0.15
-0.02
-0.72
3.56
-1.25
3.16
-1.00
0.70
-0.31
-0.10
0.26
6.00
-1.00
1.48
-1.03
-1.03
-0.33
-1.31
-1.00
-2.62
-1.82
0.11
4.77
0.50
3.88
-1.37
-0.47
-1.27
1.87
1.69
ESTIMATE,
OF ERROR
-1.39
-2.23
-3.37
-0.86
-1.03
0.87
0.07
-0.01
-0.27
1.85
-0.64
2.22
-0.54
0.31
-0.17
-0.07
0.12
3.62
-4.31
0.92
-0.41
-0.40
-0.12
-0.67
-0.52
-1.03
-0.57
0.06
2.61
0.29
2.38
-0.44
-0.18
-0.29
0.77
0.85
Determined by dividing the residual by the standard deviation at that site.
74
-------
6. MODEL EVALUATION RESULTS
6.1 Introduction
With few exceptions, each of the eleven ISDME models predicted seasonal and
annual totals of sulfur wet and dry depositions and seasonal and annual averages of
sulfur dioxide and sulfate air concentrations at most of the 66 ISDME sites depicted in
Figure 3.9. Table 6.1 summarizes the 1980 seasonal and annual predictions provided
by each model. Since some of the northern Canadian sites were outside their domain,
not all models provided predictions at all 66 sites. UMACID predicted air
concentrations and deposition amounts at 47 sites, most of which are included in the
ISDME evaluation data set. All models provided predictions at the fifteen sites in the
high deposition region (Figure 4.2).
TABLE 6.1
MODEL.
SUMMARY OF THE PREDICTIONS PROVIDED BY EACH
MODEL
NUMBER OF SO2/SULFATE SO2/SULFATE SO2/SULFATE
SITES WET DRY AIR
DEPOSITION DEPOSITION CONCENTRATION
AES
ASTRAP
MOE
RELMAP
RIVAD
RTM-II
SERTAD
SIMPMOD
STAT.MOD
TVA
UMACID
65
66
66
62
65
60
65
64
64
66
47
both
bulkS
bulkS
both
both
both
both
both
both
both
both
both
both
bulkS
both
both
both
SO2 only
both
both
both
both
both
both
both
both
both
both
both
both
both
both
both
Annual results are obtained from a separate model run rather than summing or
averaging seasonal results.
Appendix D contains three sets of tables consisting of the site values and
distributional statistics (see footnote). The seasonal and annual site predictions and
observations of sulfur wet deposition are provided in Tables D.1-D.5. Hereafter,
observations are those amounts of sulfur wet deposition passing the screening criteria
as discussed in Chapter 3. Standardized residuals (i.e., residuals divided by the
75
-------
observations) at each of the ISDME evaluation sites are provided in Tables D.6-D.10.
Finally, Tables D.11-D.15 compare the distributional statistics of the predictions and
observations in the low and high deposition regions as well as the entire region.
The remainder of Chapter 6 presents and discusses the model evaluation results
for each season of 1980. Since overall model performance is based on only the
seasonal evaluation results, the annual 1980 results are presented in Appendix E. The
site predictions and observations and the spatial patterns of the predictions and
observations of sulfur wet deposition amounts are compared for each season and the
year of 1980 via the model performance measures discussed in Chapter 4.
6.2 Seasonal Temporal Patterns
The observed amounts of sulfur wet deposition exhibited a noticeable seasonality
as a result of the seasonal variations in pertinent physical and chemical processes (e.g.,
sulfate scavenging and sulfur dioxide transformation rates). This seasonality aspect
should not be ignored when evaluating sulfur wet deposition models, even if they are to
be applied for annual periods. A model should replicate the observed temporal
pattern.
The temporal patterns of the predictions and observations were determined by
averaging the sulfur wet deposition amounts at the 30 ISDME sites where both
predictions and observations were available for each model and for every season. The
relative contribution of each season to the annual total was computed for the set of
observations and predictions (see Table D.16). The mean seasonal deposition
amounts predicted by each model were compared with the mean seasonal observed
amounts to qualitatively ascertain whether any of the ISDME models failed to
reasonably replicate the seasonal temporal pattern (Figure 6.1).
The seasonal pattern of the observations, represented by the bold lines in the
graph, indicated that approximately two-thirds of the annual amount of sulfur wet
deposition occurred in tne summer (38%) and the spring (30%). The winter and
autumn amounts accounted for approximately 16% each.
Footnote: Just prior to publication, the authors were notified of a SERTAD
programming error which understated the seasonal and annual values of sulfur wet
deposition at site U42, near central New York. This error explained the artificially
strong gradients apparent in the winter, spring and summer spatial patterns across the
New York area.
76
-------
1980 TEMPORAL PATTERN
OF SULFUR WET DEPOSITION
OBSERVED
RIVAD
SERTAO
10-
WINTER
SPRING
SUMMER
AUTUMN
SEASON
Figure 6.1 The mean percentage of annual 1980 sulfur wet deposition observed and
predicted for each season at 30ISDME sites.
77
-------
Several models poorly replicate the observed seasonal pattern. SIMPMOD winter
predictions account for nearly one-third of its annual prediction, more than double the
observed contribution. Meanwhile, its spring, summer and autumn contributions are
virtually equal. STATMOD's pattern is virtually flat with the contribution of each
season being virtually the same. Both the spring and summer contributions of
SIMPMOD and STATMOD are much lower than those observed. AES overpredicts
the winter contribution at the expense of that for spring. UMACID approximates the
summer maximum, but does not differentiate between the other seasons. RELMAP
slightly overpredicts both the summer and spring contributions at the expense of the
other two seasons. The patterns of the remaining models are more similar to the
observed pattern.
6.3 Seasonal Model Evaluation Results
The remaining discussions of this chapter focus on the model evaluation results for
the seasons of 1980. The four-part discussions for each season begin with a summary
of the distributional statistics of the predictions and observations at the ISDME sites.
Next, the predictions of the seasonally averaged air concentrations of sulfur dioxide
and sulfate are compared to those measured at the four APN sites. The third
discussion focuses on the comparison of the spatial patterns of the predictions and
observations. Finally, the performance of each model is assessed using the model
performance measures discussed in Chapter 4. The order of the seasons reflects the
relative importance of the season to the annual sulfur wet deposition as illustrated in
Figure 6.1 (i.e., summer, spring, autumn and winter).
For each model, seasonal values of the model performance measures are
compared to the criteria values presented in Table 4.3. These comparisons are used in
Chapter 7 to cluster the models according to their overall performance in predicting
the patterns of sulfur wet deposition for each season.
6.3.1 Summer 1980 Results
6.3.1.1 Summary of Distributional Statistics
Unpaired statistical features of the model predictions of sulfur wet deposition
amounts at up to 45 sites are illustrated by the quartile box plots in Figure 6.2. Only
the predictions are considered from those ISDME sites where valid observations are
available. The two vertical rectangles for each set of predictions and the observations
represent the range of the second (lower) and third (upper) quartiles. The vertical
dashed lines, plus any symbols above or below, represent the range of the first (lower)
and fourth (upper) quartiles. The "o" and "*" symbols represent values which would
occur about once in 20 ("o") and once in 200 times ("*") if the distribution were normal.
The mean value is indicated by " + ", while the median is represented by "* *".
Standard deviations of the distributions are provided to the left of the first quartiles.
78
-------
TOTAL SULFUR WET DEPOSITION, kg S/h»
SUMMER 1980
-I-
1 O.l
-I-
AES ASTRAf MOE RELMAP RIVAD HTM SERTAD SIMPMOO STATMOD TVA
1 2 I
UMACIU
Figure 6.2 Summer 1980 box plots of the predictions and observations of sulfur wet
deposition (kg S/ha).
20
SULFATE CONCENTRATION. ,,g m
SUMMER 1980
9 81
39 I
38
33,
ASTRAP
AIVAO
RTM
SERTAD SIMPMOD STATMOD TVA
Figure 6.3 Summer 1980 box plots of the predictions of sulfate air concentrations
79
-------
0
60
40
20
0
MODU
Figure 6.4
(ng
10
8
6
4
2
0
0 o
0
0 .
I
I
0
I 0
I
g 61 --- 18 1
I 1001 ---
78 I
-J
.-
._ 89
. _ , . _^_ . _
SO2 CONCENTRATION, ^g m3
SUMMER 1980
0
0
0
I I
0 I -.-
1 1 1
-. 1 .... 1
--- III
1 1 -- 1 8 3
27 1 37' ' 7'4I 72I 561
AES ASTRAP MOE REIMAP RIVAD RTM SERTAD SIMPMOD STATMOD TVA UMACID
Summer 1980 box plots of predicted sulfur dioxide air concentrations
/m3).
i
°
.
0
0
0
0
;--
TOTAL SULFUR DRY DEPOSITION. \>g S ha
SUMMER 1980
0
0
0
-. 0
0
0 -.-
..--. 0 .- -.
1
1
1
0 , , . .
.
1
1
1
i
3 2
2 6
. . 22 1 .---
1 81 1 1 Ol
1 1 1
-.- 0
- I
I
I
-- < I
I
I
2 2
I I
I I 2 1 20
! 1 1 iT'
86 1
RIVAD RTM
SERTAD SIMPMOD STATMOO
Figure 6.5 Summer 1980 box plots of predicted sulfur dry deposition (kg S/ha).
80
-------
This figure indicates that for most models, the distribution of sulfur wet deposition
predictions resembles that of the observations. However, both RIVAD and RTM
substantially underpredict and their standard deviations are much lower than that of
the observations. On the other hand, RELMAP tends to substantially overpredict and
its standard deviation is twice that of the observations.
Figures 6.3-6.5, which are quartile box plots of the predicted summer average air
concentrations of sulfate and sulfur dioxide and sulfur dry deposition amounts at all
ISDME sites where predictions are available, illustrate the behavior of the models.
This information is of limited use since sufficient measurements of these quantities are
not available. However, the figures provide the means to
1) Intercompare model results for which there are no measurements,
2) Assess the magnitude of the model process rates, and
3) Assist in explaining model performance.
Figure 6.3 illustrates the very high SERTAD sulfate air concentrations, while
Figure 6.4 shows that SERTAD sulfur dioxide air concentrations are considerably
lower than those of the other models. These differences can be explained by
1) The very high SERTAD conversion of sulfur dioxide to sulfate near the
sources (40% of the total mass of sulfur dioxide is converted to sulfate)
2) The lack of consideration of sulfate dry deposition.
The fact that RIVAD and RTM underpredict the amounts of sulfur wet deposition
(Figure 6.2) can be explained by the tendency of both models to predict low sulfate air
concentrations (Figure 6.3) and for RTM low sulfur dioxide air concentrations as well.
These low predictions are attributed in part to the relatively high predictions of sulfur
dry deposition (Figure 6.5), which tend to be factors of two to four higher than their
predictions of sulfur wet deposition (Figure 6.3).
Finally, the relatively high RELMAP sulfur dioxide air concentrations of the third
and fourth quartiles (Figure 6.4) prompted an investigation whichdiscovered that all
the emissions were inadvertently injected into the surface layer, rather than each of the
three layers. An additional model application for a one-month period indicated that
the injection of all emissions into the surface layer did not have a substantial effect on
RELMAP predictions of sulfur wet deposition, but did cause higher air concentrations,
particularly for sulfur dioxide.
81
-------
6.3.1.2 Comparison of Predictions and APN Measurements
During 1980 daily average air concentrations of sulfur dioxide and sulfate were
measured at four Air and Precipitation Monitoring Network (APN) sites in Canada.
These sites, where sulfur wet deposition was also monitored, are located in western
Ontario (C01), southern Quebec (C03), southern Ontario (C04) and southern Nova
Scotia (COS) (see Figure 3.9). Seasonal and annual means of these daily
concentrations were computed and compared to the corresponding means predicted by
the models at each site. Since data are available from only four sites, the general
tendencies of the models deduced from these figures are not and should not be
extrapolated to other sites. The comparisons are used here to identify outlier models
at these sites only.
Figure 6.6 compares the predicted and observed air concentrations of sulfate and
sulfur dioxide for the summer at these four sites. Three of the four statistical models,
SERTAD, SIMPMOD and STATMOD, substantially overpredict sulfate air
concentrations. The SERTAD predictions are as much as six times greater than the
observations. Similarly, Figure 6.3 indicates that the means and standard deviations of
the SERTAD sulfate air concentrations are more than twice those of any other model.
RTM tends to underpredict by a factor of two the sulfate air concentrations at the
APN sites. Since Figure 6.3 also indicates that the RTM predictions are much lower
than those of other models, it is likely that this model substantially underpredicts the
sulfate air concentrations elsewhere.
For sulfur dioxide air concentrations, all of the models, with the exception of RTM
and SERTAD, tend to overpredict at the APN sites. RIVAD tends to overpredict by
as much as a factor of nine. However, the distribution of its predictions at the ISDME
sites (Figure 6.4) is similar to that of most other models.
6.3.1.3 Spatial Analyses
The comparisons of the kriged spatial analyses of predictions and observations in
Figure 6.7 enable one to assess the ability of the models to replicate observed patterns.
In the upper left corner of each page in this series of plots is the kriged analysis of the
summer observations. Highlighted are the isopleths that are multiples of 4 kg S/ha.
The kriged analysis of the observations shows that the greatest amounts of sulfur
wet deposition are found across Ohio and western Lake Erie. A ridge of large
amounts is apparent from Illinois and Wisconsin to Ohio and continuing northeastward
to the Maritime provinces. Several secondary peaks are apparent to the north and
northeast of Lake Ontario and in the vicinity of southern Lake Michigan. Because of
the uncertainties of the measurements, however, the actual existence of all of these
peaks is uncertain. Also apparent is the region of minimun amounts along the East
Coast of the United States, which corresponds to the region of minimum summer
82
-------
60
50
40
O 30
O
20
10
0
11111 n 1111
SULFATE
SO2
Figure 6.6 Comparison of the summer 1980 observations and predictions of the air
concentrations (|ig/m3) of sulfur dioxide and sulfate at the APN sites.
83
-------
84
-------
1C
UJ
2
5
M
Q.
I
DC
111
5
0)
T
5
. E
z
g
1
DC
UJ
M
m
o
cc
UJ
5
(0
oc
UJ
5
D
-------
ec
UJ
5
5
u
(0
Q
CC
UJ
v>
z
g
I
cc
CD
O
CC
UJ
5
5
3
86
-------
K
Ul
5
5
=>
w
to
CQ
O
OC
UJ
(A
87
-------
precipitation amounts (Figure B.5).
In contrast to the observed pattern, the predicted patterns exhibit much less detail
in their structure, since the predictions represent spatial averages across grid cells as
opposed to the measurements at a point. These predicted patterns are similar to each
other in that
1) They tend to consist of nearly concentric ellipses;
2) They exhibit a single area of peak wet deposition amounts in the high
emissions region of Ohio, West Virginia and Pennsylvania (Figure 3.2), as
opposed to the multiple peaks exhibited in the more detailed observed
pattern; and
3) The predicted maxima are located 130 to 350 km to the south and east of the
region of maximum observed amounts.
In addition, each predicted pattern exhibits a ridge of high values, however the
orientation of this ridge varies from west-to-east (most models) to southwest-to-
northeast (e.g., AES, ASTRAP, TVA and UMACID).
The interpolated values of the predictions at each of the half-degree grid cells,
from which the patterns in Figure 6.7 are derived, can be compared to the uncertainty
range of the observations (i.e., the observed value plus and minus two standard
deviations) to determine where the predictions significantly differ from the
observations. Figure 6.8 illustrates those cells where the predictions significantly differ
from the summer observations. The " +" and "-" symbols on the maps represent cells of
significant overprediction and underprediction, respectively. Based on patterns of
under/overpredictions, the models appear to cluster into one of the three categories
described in Table 6.2.
When interpreting the measure for the area of significant differences, one should
bear in mind one potential drawback of this measure: The uncertainty envelope, used
to identify those cells where the prediction significantly differs from the observation,
can be inflated for those models which exhibit relatively high variances of their
predictions (compared to the variance of the observations). Consequently, these
models would receive an unfair advantage. As discussed in Sections 5.1, the RELMAP
variances for summer and spring and ASTRAP variances for each season are relatively
high and, therefore, could benefit from the unfair advantage. Since the extent of this
advantage has not been quantified, the area of significant differences for these two
models for the specified seasons represent the Tower limit of the estimate. The
possible effects on the performance score for pattern magnitude are discussed in the
Model Performance Measures sections that follow.
88
-------
DC
Ul
5
2
w
a.
oc
(0
cc
UJ
5
2
(A
(A
UJ
89
-------
V .
E !
UJ
i *
5 - - ^ -^
i ^
1
H
K r~
j
r
C r
\ ^
/
) '
'
^J
1
S
r^
t 1 1
fN!
J::u
rl 1 1 1,
f i i i a
i lit
i i t
i i i
i i i \ '-IT
LU *
/1 14
i^tfri,
y 1 1 1 i-
r ^
v t-
s^
r v--^-
r ^ "~ ~
,.'-
T
\
J
t*
~T-
,
..--
A
"*L
-*
-
-
.'
^
-
H
-__
"^
T
.*~*
-t-
i
^
:\
i
i
i
i
1
t-
*~%
^7%
i i
<
n
/
t-*' \
*
\
*
-vi.
_>
W
Q
O
0.
5
GC
UJ
D
M
oc
UJ
^
\ "t-
, \
'Sjv( ^
S 2
LJ^
§2
ca
-------
cc
UJ
5
5
D
(A
tr
ai
CO
Q
O
s
25
>s
o 9
-------
TABLE 6.2. TYPES OF PATTERNS OF SIGNIFICANT DIFFERENCES FOR
SUMMER 1980.
TYPE DESCRIPTION MODELS
I Significant differences nowhere SERTAD
or limited to small widely
scattered areas
II Significant overpredictions ASTRAP
across areas to the east and/or RELMAP
south of the Ohio River Valley
III Significant underpredictions AES
across continuous or sporatic MOE
areas from the Midwest to Nova RIVAD
Scotia RTM
SIMPMOD
STATMOD
TVA
UMACID
See footnote in Section 6.1.
6.3.1.4 Model Performance Measures
The differences between the predictions and observations of sulfur wet deposition
that are illustrated in the preceeding figures are quantified by the model performance
measures in Table 6.3. These measures are subsequently discussed in the order as they
appear in the table. Those measures designated in Table 4.3 as primary performance
measures are emphasized.
Pattern Magnitude - Apparent from the measure of the area of significant
differences (Table 6.3) is the wide range of success of the models in replicating the
spatial patterns. On one hand, SERTAD does not significantly under,/overpredict in
more than 5% of the cells in the composite region (i.e., all four subregions illustrated
in Figure 4.1) or in more than 5% of the cells in any of the four subregions. On the
other hand, RIVAD, RTM and STATMOD significantly underpredict in nearly 40% of
the composite region and more than 40% in one or two subregions.
Four models (AES, MOE, SERTAD and TVA) pass the more stringent criterion
for the area of significant differences (i.e., significant differences in no more than 20%
92
-------
.
o
5£r
c^
fH
tf
5
^^
jj
g
S
.
O
rT
HH
S
Q/
I-M
P
oo
W
S
w
u
!Z
^1}
^^
S
PM
o
HM
DH
(X
p*^
^J
W
Q
0
S
O
?*
Pji
S
S
00
Uj
CO
g
«
S
1
5
>-
at
ee
UJ
ac
ac
>
Of
*
W
ac
u>
*
V)
O «- IA
o N> in
Kl
O IA
fMsOON-4- rao«-ooooorg NN oo sr-
«coco^^ i;i§§f5S8:o§ M«o ^ts s^01 -*fc
K. s« n\ ^ nv OOOfVJOOOW ^O*^ OO |JS>*^ N.I
*
iA *» KS-gS-S§3 o«g CS
fs) 00 *O -O «O «-OOO(MOOOv* -O - OO -O
ru ? ^F (M Pt + KI
§o
M S «M r- i , r- +
eoSeoS-j-QK-o^- «-N.
- iO «* co o MO-J-OOOOOIM ^eoRi oo K> * * >»""
«* + M
O IA
<< + ^«MK1W>'*»-OKlCp ^O CO
O>> ro IA O'- * O'«ONOoOKO*-foO» o oo o meo h-O> w 'o» »«* M OI "* ?"8 5 5 5 ^ >< 3 C^
C WtClAVti* (/> 'v ^ OI TJ ,2 ., T) Ct iS C
a owjwoaijt. w ^ j* * %- ^ SS "2 * ^ *-
C H- V t. U 3 3 ' 3 (fl 5 3o3o" <~] Oil . _- W ' 4* ?'^£^5 °° -222S£
'E*° v«v«^ Q. ° ° « 'c *C in"
oi g fl) uooacoo
ara> < < 0 . .c . . g o ffl fl t- fti a.
3 x E*
a. . m v
v>< « uac4:mi-<
-
^
.1
i
2
41
t-
O
(A
4»
1
5
|^
o
I
4)
Q.
|
i
41
> O
« *
l>
«->
Jl
o E
1! S
D. '=
j; .i!
1|
TJ
« * 5
4) M
3 O O
re Z o
U U 3
o ^ 5
i &^
Si"
^° $
41 <~
i- S! &
il:
i_ > 4>
0. 4> T5
i_ B
A> O. E
O TJ
U TJ £
W C « >
i- re 41 i_
cu 5 o
? 3* « JS
3 3 > O
« *
93
-------
of the composite region and no more than 25% in each subregion). Each receives the
maximum score of four for pattern magnitude. With the exception of SERTAD, these
models have a tendency to underpredict, especially in the northeast. The significant
differences for SERTAD are limited to small widely scattered areas.
MOE is only one of two models, SERTAD being the other, which superimposes a
background contribution to the predictions of sulfur wet deposition (Tables 2.1 and
2.2). Although each model considers background contributions, the definitions of the
background levels for the two models are quite different. The MOE background
contribution attempts to account for the effects of the sulfur emissions from biogenic
sources inside the model domain, which were not included in the anthropogenic
emissions inventory, as well as from sources outside the model domain. A total annual
background contribution of 0.7 kg S/ha is added to the predicted amounts across each
cell in the model domain. This amount is about 30% of the observed annual amounts
at the remote ISDME sites, where background levels are relatively significant, and less
than 10% of those at sites in the high deposition region.
On the other hand, the SERTAD background contribution is comprised of two
components; one is similar to the MOE background. The other component assumes
that a minimum of 80% of the sulfur emissions from stacks exceeding 250 m is wet
deposited across the model domain. The amount deposited across a cell is
proportional to only the precipitation amount for that cell; source-receptor distances
are irrelevant to the amount deposited. Hence, the spatial pattern of SERTAD
predictions of sulfur wet deposition, which for a statistical model would normally be
rather smooth, would tend to resemble the precipitation pattern. Unlike the
background levels for MOE, those for SERTAD were much higher. For example,
based on typical precipitation rates across Ohio during the summer of 1980, the
SERTAD background contributions for summer are 2.4 kg S/ha, or about 40% of the
observed amount.
With the exception of ASTRAP, all of the other models under/overpredict in
more than 20% of the composite region and in more than 25% of two or more
subregions. These models, therefore, receive a score of zero for pattern magnitude.
ASTRAP satisfies the less stringent criterion and, therefore, receives a score of two.
The relatively high ASTRAP variance probably is not a factor in its score, since it is
unlikely that the model would have otherwise significantly overpredicted in twice as
many cells in the composite region (i.e., 20% rather than 10%).
For this season, the assessment of model performance based only on the measure
of the area of significant differences generally concurs with that based on only the
descriptive statistics. Three of the four models with the maximum score for the pattern
magnitude, MOE excluded, have medians and dispersions closest to those of the
observations and have the lowest biases and mean squared errors. Furthermore, those
models failing the criteria for the area of significant differences also perform least
favorably according to the descriptive statistics alone.
However, one must not rely on descriptive statistics alone. There are two good
examples for this season. First, the spatial analysis of the MOE predictions indicates
94
-------
that the model performs rather well in replicating the pattern, but the descriptive
statistics give the appearance that the model performs relatively poorly (i.e., the
median and dispersion are lower and the biases are higher than those for AES,
SERTAD and TV A). Secondly, the descriptive statistics for ASTRAP indicate that
this model performs very well, however, the spatial analysis indicates that this model
significantly overpredicts across nearly half of the southeast subregion.
Pattern Position - Four models (ASTRAP, RIVAD, RTM and UMACID) pass
both criteria for pattern position. Their centroids (Figure 6.9) are within 65 km of the
observed centroids and the orientation of their major axes differs from that of the
observations by less than 10 degrees (Table 6.3). Figure 6.9 indicates that the
centroids of all models but MOE and SERTAD are located to the south and east of
the observed centroid. This shift is indicative of the tendency of the models to predict
the greatest values to the south and east of the greatest observations. The MOE
centroid is the only one to the north of the observed centroid, while the SERTAD
centroid is virtually colocated. The difference in the location of the MOE centroid
relative to those predicted by the other models is consistent for all seasons and could
reflect the method in which this model simulates transport. This is the only model
which uses geostrophic wind velocities calculated from surface pressure fields to
simulate transport.
SUMMER 1980
Figure 6.9 Summer 1980 locations of the centroids.
95
-------
Four models (RELMAP, SIMPMOD, STATMOD and TVA) pass neither
criterion for pattern position. Their centroids are at least 90 km from the observed
centroids and the orientation of their major axes differs from that of the observations
by as much as 20 degrees.
The four statistical models (SERTAD, SIMPMOD, STATMOD and TVA) exhibit
the largest differences in the orientation of the major axis. Unlike that for the other
models, the orientations for the statistical models tends to be more west-to-east than
the observed southwest-to-northeast orientation. The greater differences for the
statistical models could be an artifact of the methods in which the transport winds are
determined. These models compute wind roses to represent the long-term tendencies
of the boundary layer wind flow throughout the simulation period.
A potentially serious problem with this approach is illustrated. During the
summer, the long-term average flow across much of the northeastern United States
tends to be from the west. However, prior to and during precipitation events, if the
flow in the boundary layer tends to be from the southwest, the statistical models would
tend to transport and scavenge sulfur to the east of the major source regions, while the
deterministic models, which use actual wind measurements, would tend to transport
and scavenge sulfur to the northeast of the sources.
The final measure for pattern position is the distance the predicted patterns are
shifted to maximize the correlation between the observations and predictions. All
predicted patterns are shifted 110 to 453 km (or two to nine half-degree grid cells) to
the west and northwest, once again reflecting the tendency of the models to predict the
higher wet deposition amounts too far east and south.
Pattern Shape - A wide range in the correlations between the predictions and
observations (0.30 to 0.81) is apparent for summer. The correlations are highest for
RIVAD and RTM predictions. Over 60% of the variance is explained by these two
models, while just over 50% is explained by MOE, SERTAD and UMACID. The
correlations are lowest for RELMAP and SIMPMOD.
Pattern shifting substantially enhances the correlation for most models, especially
that of RELMAP and SIMPMOD (both of which nearly doubled) and AES, ASTRAP
and TVA. STATMOD is the only model which does not exhibit a substantial increase
in the correlation as a consequence of the shift. The correlation could increase by
rotating the pattern counterclockwise. As a consequence of the shifting, all models but
SIMPMOD and STATMOD explain over half of the variance.
Maxima - As illustrated by the ISDME site values in Figure 6.10 and the spatial
patterns depicted in Figure 6.7, the models generally predict the maximum in or near
West Virginia, while the maximum observation is located in central Ohio. Only TVA
passes the criterion for both the location and magnitude of the maximum value. Its
maximum prediction was within 85 km of the location and within 15% of the
magnitude of the observed maximum. Conversely, RELMAP, SIMPMOD and
STATMOD pass neither criterion.
96
-------
MAXIMUM SUMMER SULFUR WET DEPOSITION, 1980 (kg S/ha)
/0= OBSERVED
MODELS{
TVA
12= AES
/3= SIMPMOD
/4= STATMOD
/5= MOE
/6= SERTAD
/7= RTM II
/8= RIVAD
/9= AST RAP
/A= UMACID
/B= RELMAP
Figure 6.10 Summer 1980 locations and magnitudes (kg S/ha) of the maximum
amount of sulfur wet deposition at the ISDME sites.
Three models (MOE, RIVAD and RTM) pass the criterion for the location of the
maximum, but each underpredicts the magnitude by 30 to 60%. These
underproductions are consistent with the general tendencies of these models to
underpredict everywhere.
Four other models (AES, ASTRAP, SERTAD and UMACID) pass the criterion
for the magnitude, but fail the criterion for the location. These models predict the
location of the maximum value at least 250 km from the nearest probable location.
Temporal Pattern - Only AES, SIMPMOD and STATMOD fail the criterion for
the summer contribution to the annual predicted total. As illustrated by Figure 6.1, the
percent contribution for each of these models is 15 to 40% lower than that observed,
partly as a result of their high winter contributions.
97
-------
6.3.2 Spring 1980 Results
6.3.2.1 Summary of Distributional Statistics
Illustrated by Figure 6.11 are the statistical features of the distributions of the
predictions at 41 to 46 evaluation sites and the observations at the 46 evaluation sites
for spring sulfur wet deposition amounts. The spring features are similar to the
summer features illustrated by Figure 6.2. For example, both RIVAD and RTM
underpredict and show a much smaller standard deviation, while RELMAP
overpredicts the higher values and has a greater standard deviation than that of the
observations. STATMOD and UMACID underpredict as well, but not to the extent of
RIVAD and RTM.
The box plots of the spring predictions of sulfate (Figure 6.12) and sulfur dioxide
(Figure 6.13) air concentrations and sulfur dry deposition amounts (Figure 6.14) are
similar to those of summer. Again, SERTAD sulfate air concentrations are
significantly higher while its predictions of sulfur dioxide air concentration and sulfur
dry deposition are lower than most other models. As in summer, RIVAD and RTM
sulfur dioxide air concentrations and RTM sulfur dioxide and sulfate air
concentrations for spring are substantially lower than the others while their sulfur dry
deposition amounts are about twice as high as those of other models. Finally, the third
and fourth quartile RELMAP predictions of sulfur dioxide air concentrations again
tend to be much higher than the others.
6.3.2.2 Comparison of Predictions and APN Measurements
Illustrated in Figure 6.15 are the spring averages of the predicted and observed air
concentrations of sulfur dioxide and sulfate at the APN sites. Similar to the summer
comparisons, the four statistical models predict the highest sulfate concentrations and
overpredict the sulfate concentrations for spring. SERTAD sulfate concentrations are
as much as six times too high, while those of the other models are generally within 50%
of the observations.
Also similar to the summer comparisons, most models underpredict the sulfate
and overpredict the sulfur dioxide concentrations at C05, the Nova Scotia site. The
reason for this is not apparent. For the sulfur dioxide concentrations, RIVAD
overpredicts at each site and RELMAP overpredicts at C04 and C05, the sites which
are more often downwind of the major source regions. This is also the case for
summer.
98
-------
TOTAL SULFUR WET DEPOSITION, kg S h>
SPRING 1980
1 1 I
I I I I
24
AES ASTRAP
...I...
MOE
I I II
I <
36 I
I 93 I
RELMAP RIVAD RTM SERTAO SIMPMOD STATMOD TVA UMACIO VALIDATION
Figure 6.11 Spring 1980 box plots of the predictions and observations of sulfur wet
deposition (kg S/ha).
20
SULFATE CONCENTRATION ,,g m3
SPRING 1980
36 I
78 i
24
MODEL AES ASTRAP MOf RELMAP RIVAD RTM SERTAD SIMPMOD STATMOD TVA UMACID
Figure 6.12 Soring 1980 box plots of the predictions of sulfate air concentrations
(Hg/m3).
99
-------
60
40
20
0
MODEL
y
SO2 CONCENTRATION. ,ig/m3
0 SPRING 1980
0
0 00
0
; I
0
I 0
i ' .---. ,~ . , . ! ,
1 1 1
1 1 1 1 -
- - - 1 1
1 III 1
--- '"'II .1
1 1 1 1 1 t 1 . I ,
| 181 °| II!' ' ' ' 78 '
791 861 79 i .---4 ,--, ' | '66 ] [
1 1 3 61 4 51 ° 7 491 1
AES ASTRAP MOE RELMAP RIVAD ATM SERTAD SIMPMOD STATMOD TVA UMACID
Figure 6.13 Spring 1980 box plots of the predictions of sulfur dioxide air
concentrations (n,g/m3).
10
8
6
«
2
0
TOTAL SULFUR DRY DEPOSITION, kg S-ha
SPRING 1980
0
0
I
0
0
0
0
0
0
!
0
-.- . . .- . 0
t t t 1
. . . . 271
! . --- 1
. . | . .---. 1
221 161151 l'sP
1 61 141 101 9'gp j
i 8'8""
MODEL AES ASTRAP MOE RELMAP RIVAD RTM SERTAD SIMPMOD STATMOD TVA UMACID
Figure 6.14 Spring 1980 box plots of the predictions of sulfur dry deposition (kg
S/ha).
100
-------
18
16
14
12
O 10
O 8
6
4
2
0
o
o
U>
O
O
15
12
9
6
3
0
mm mi
tt.
<
W
<
Q. Q
< <
O O O
2 ii
E 11
-------
6.3.2.3 Spatial Analyses
Unlike the summer pattern, the kriged pattern of spring observations (Figure 6.16)
has only one peak of sulfur wet deposition. The ridge of maximum values stretches
from eastern Illinois to eastern West Virginia. Areas of minimum deposition are
located across eastern North Carolina, the northeastern United States and the
Maritime provinces, reflecting the areas of minimum precipitation (Figure B.4).
The reasons for the eastern shift of higher observed amounts is not apparent. In
both summer and spring, relative precipitation maxima are located across Indiana and
Ohio (Figures B.4 andB.5), so seasonal precipitation patterns do not seem to be
factors.
The kriged patterns of the predictions (Figure 6.16), which generally resemble
concentric elipses, also show maximum values across Ohio, West Virginia and western
Pennsylvania. The STATMOD pattern shows a second area of maximum values across
the western segment of the Ohio River. All predicted patterns replicate the observed
ridge of high values along the Ohio River as well as the relatively high values along the
Atlantic coast from Virginia to New York.
Based on the number of half-degree cells in Figure 6.17 where the predictions
significantly differ from the observations, the predicted spatial patterns compare more
favorably for spring than for summer. Like the summer case, the models can be
classified into three types of patterns of significant under/overpredictions (Table 6.4).
Although ASTRAP and RELMAP significantly overpredict in portions of the
region to the east and south of the Ohio River Valley for both summer and spring, the
geographic extent in spring is smaller. The same is true for the extent of the RIVAD,
RTM, SIMPMOD and STATMOD underpredictions.
6.3.2.4 Model Performance Measures
Pattern Magnitude - According to the values of the measures for pattern
magnitude (Table 6.5), six models perform very well for spring and receive the
maximum score of four. The predictions of four of these models (MOE, SERTAD,
SIMPMOD and TV A) significantly differ from the observations virtually nowhere.
The other two models (AES and ASTRAP) significantly underpredict in a small area
of western Ontario and significantly overpredict across Pennsylvania and New Jersey,
respectively.
102
-------
103
-------
e>
z
E
a.
V)
o.
IU
cc
(S
z
£
a,
w
co
O
I
oc
UJ
00
03
o
CD
Z
oc
0.
M
O
Z
E
a
w
O
I
oc
104
-------
o
z
E
Q.
V>
oc
UJ
(A
\
(A
O
K
UJ
(/)
CD
O
O
Z
E
a.
105
-------
tc
0.
(0
g
oc
01
CD
O
o
z
E
Q.
(A
106
-------
;S
T1
P*
-'N/ i
^ -* »»
i r~>»* » *
j ,»»**»»
!* ^ A + * +
~~<~*;Si»I«
* + +^ +
X^^* *>-^
r***^-^-"
L ; I .?£#?#
. ««*»**< r
» * * *^+ * +
d
_o
^^»
1-H
«
o
c±
-------
\
111II
108
-------
109
-------
TABLE 6.4 TYPES OF PATTERNS OF SIGNIFICANT DIFFERENCES FOR
SPRING 1980.
TYPE
DESCRIPTION
MODELS
II
III
Significant differences nowhere
or limited to small widely
scattered areas
Significant overpredictions
across areas to the east and/or
south of the Ohio River Valley
Significant underpredictions
across continuous or sporatic
areas from the Midwestern
United States to Nova Scotia
AES
MOE ,
SERTAD
SIMPMOD
TVA
ASTRAP
RELMAP
RIVAD
RTM
STATMOD
UMACID
See footnote in Section 6.1.
Three other models (RELMAP, STATMOD and UMACID) do not significantly
under/overpredict across more than 20% of the composite evaluation region.
However, RELMAP significantly overpredicts in nearly 60% of the southeastern
subregion and STATMOD and UMACID significantly underpredict across nearly a
third of the north central and southwestern subregions, respectively. These models
receive a score of two for the magnitude measure. Since RIVAD and RTM
significantly underpredict across more than 20% of the composite evaluation region
and more than 55% of one subregion, they receive a score of zero.
The high variance of the RELMAP predictions is not likely a factor in its scoring,
since the number of cells where the predictions significantly differ from the
observations would have to nearly double in either the southwest subregion or the
composite region. On the other hand, the ASTRAP score could drop to one if the
effects of the high variance of its predictions caused a 25% drop in the number of cells
where the predictions significantly differed in the southeast subregion.
One final point, as is the case for summer, SERTAD predictions are based largely
on a "correction" term (calculated from the precipitation amounts), rather than
physical and chemical parameterizations, as described in Section 6.3.1.4 (see Concord
Scientific Corporation, 1985). For example, based on the typical spring precipitation
amount of 270 mm, 45% of the SERTAD predictions of sulfur wet deposition across
Ohio is attributed to the background contribution.
110
-------
o
Cs
T-H
O
o.
Cfl
C*
S
MM
j»y?
Sj
C/3
S
W
U
Z
3
g
o
fe
OH
PM
^J
u
Q
OF MO
1
*|
S
D
on
§
S
u
««
at
*
QC
UJ
8
X
R
«1
U
as °:
~ ° K R
o *
w- M *- -f fM
OOOOO *O»*Or'- n
° ft *t t- tt v-OOOv-Op-OfM fM S OO ii K
m fMKI
"~ f\i *"..*" . d d ^
^- -o f*- .J S oo)^- fM
dodod CModddddod tv £ dd id ^
fM
I
Ot * H^ *-.
s« 7- £^ Hi Stt
B S . g-SIe 5 s*~l.ssi.
s ^s t .5 i « " a i «5 Or St.5^s
- w ** e ** ** O> J w BTi m ^m O »* -* a C
£ 2 "f f tf * M w "* 9 T? JDw ., -n C p. J t
Q OyinowMi- -+ M 0 «^ C C T» t* IT - t.
4J <^4io 2X °*^*6rt W«!B WS 00 COVCKI-AI
t Ol-OOOO-^ C ^ U - - Z ^ ' W Ol _. ^ U < * 4) U O
w««iacatun)vt< I n «i n ra . * u u *- «t> t» «* S
w -Q «fl *- >!l>«i>M>CC^'O«- i_i_ c -Q -Q *- *^
S22:t |cx.2a.2a.2a!.5£«^ fe& -£222?£
flb"~ j« ff o O O to i O « £**gOA]IDl-4>a.
X«- '
S « r
^ « - J£<«l<
>
1
Kl
s
8
(B
s
ij
w
5
1
SL
s
I
*
x S
1 °
s *
8 >
U V
JC
Ii
u 'i
1 1
i- u
i 1
.Is
11°
» ^ *
v u 5
t) C >
g t- j=
c > *J
1^1
& . £
is »re primary
the prevalertC'
lach ndel rep
t. tt
II*
8 * ?
? "! S
t_ X
o
U ~Q
U C Vt M
1 : s 3
e + « >
*
«
VO
in
-------
Pattern Position - With few exceptions, the models perform equally well in
predicting pattern position. The distances between observed and predicted centroids,
illustrated by Figure 6.18, are less than 85 km. All models but RELMAP and
STATMOD pass me criterion for the centroid.
The orientation of the major axis for the observations (52 degrees) is more west to
east in spring than for any other season when the orientation tends to be south-
southwest to north-north-east. The orientation of the major axes of the predictions
deviates from that of the observations by no more than 8 degrees. Therefore, all
models pass the criterion for the orientation.
For all but three models (AES, ASTRAP and RTM), the distance of the pattern
shift to maximize the correlation is less than 160 km, slightly larger than one model
grid cell. The patterns of the three models are shifted from 350 to 550 km to maximize
the correlation.
SPRING 1980
NUMBER
1
2
3
4
5
6
7
8
9
10
11
12
MODEL
AES
ASTRAP
MOE
RELMAP
RIVAD
RTM-II
SERTAD
SIMPMOD
STATMOD
TVA
UMACID
VALIDATION
Figure 6.18 Spring 1980 locations of the centroids.
112
-------
Pattern Shape - As in the summer, MOE, SERTAD and UMACID have spring
correlations in excess of 0.7, indicating a consistency of relatively good model
performance in predicting the pattern shape and position for those two seasons. On
the other hand, RIVAD and RTM correlations for the spring are considerably lower
than the 0.8 correlations for the summer. RELMAP and SIMPMOD correlations are
again low (i.e., less than 0.6), as are those for AES, ASTRAP and RTM.
Pattern shifting increases all correlations to nearly 0.7. The magnitude of the
increases appear to be correlated to the distance the patterns are shifted to obtain the
maximum correlation (i.e., the correlations for those models with patterns shifted the
most increased by the greatest margin). Those models with the highest correlations
before pattern shifting tend to require minimal shifting to obtain the maximum
correlation.
Maxima - As illustrated by the maximum ISDME site values in Figure 6.19 and
the kriged patterns of Figure 6.16, the predicted and observed maximum values are
located within 150 km of the southwestern corner of Pennsylvania. The good
agreement is another indication of the better model performances in the spring
compared to those of the summer.
MAXIMUM SPRING SULFUR WET DEPOSITION, 1980 (kg S/ha)
A> OBSERVED
MODELS
/1- TVA
AES
SIMPMOD
STATMOD
MOE
/6= SERTAD
17= RTM II
/8= RIVAD
/9= ASTRAP
/A= UMACID
/B= RELMAP
5.3/O»U44
0/5 45/37^
3.2,*
Figure 6.19 Spring 1980 locations and magnitudes (kg S/ha) of the maximum amount
of sulfur wet deposition at the ISDME sites.
113
-------
Considering the uncertainties of the observations, it cannot be stated that any
model fails to accurately predict the location of the maximum value. Therefore, all
models pass the criterion for the location of the maximum value.
Both approaches comparing the magnitudes indicate that for eight models the
magnitude of the predicted maximum is within the criterion value (30%) of the
observed maximum. RELMAP overpredicts by over 50%, while RIVAD and RTM
underpredict the magnitude by more than 30%.
Temporal Pattern - The seasonal contribution of only ASTRAP, MOE, RELMAP,
RIVAD, RTM and TVA is within 15% of the observed spring contribution of 29%. As
is the case for summer, SIMPMOD differs the most from the observed contribution
and has a lower seasonal contribution than observed.
6.3.3 Autumn 1980 Results
6.33.1 Summary of Distributional Statistics
Figure 6.20 shows the tendency of RIVAD and RTM to underpredict sulfur wet
deposition amounts. This is consistent with the summer and spring seasons. On the
other hand, AES, ASTRAP, SERTAD and SIMPMOD tend to overpredict in the
autumn. Contrary to the summer and spring, RELMAP does not show a tendency to
overpredict for this season.
The SERTAD and SIMPMOD overpredictions could be a consequence of the
manner in which precipitation is considered and, in the case of the former, how
background contribution is determined. These two models, as well as TVA, use
statistical representations of the seasonal precipitation rather than actual precipitation
rates for periods of the order of a day or less (see Section 2). If the precipitation data
used by these two models do not correspond to the much lower precipitation amounts
observed across the high emission regions, the models could substantially overpredict
there.
It is interesting to note that the distribution of STATMOD predictions agrees
much better with that of the observations than does SIMPMOD, even though the
models are identical with the sole exception of their consideration of precipitation.
The lower precipitation amounts across the high emission areas, where RELMAP
tends to overpredict air concentrations of sulfur dioxide could alone be responsible for
this model's good agreement with the distribution of the observations.
Figures 6.21 - 6.23 indicate, as those for summer and spring, that SERTAD sulfate
air concentrations are much higher than those predicted by other models. RIVAD and
114
-------
" TOTAL SULFUR WET DEPOSITION kg S I
FALL 1980
US AITBAP MOf MIMA* MWO HTM IIHUD SMWOO IUTMOO TW UMACW MUOATIUN
Figure 6.20 Autumn 1980 box plots of the predictions and observations of sulfur wet
deposition (kg S/ha).
SULFATE CONCENTRATION
FALL 1980
I
.--
1
'.-.-
1
3 2t
'
1
[
--
i
'
( "
] (
1 7 r' 3*
7«
'
..
116* 38:^:
i
i i
i i
i i
i
... j- j
I '! 27
t t
t t
1
2
I
!
'!
All AITMP HOf
SERTAD immOO STATMOO TW UMACID
Figure 6.21 Autumn 1980 box plots of the predictions of sulfate air concentrations
115
-------
BO, CONCENTRATION , g i
FALL18BO
Figure 6.22 Autumn 1980 box plots of the predictions of sulfur dioxide air
concentrations (tig/nr).
IS ASTRAP
TOTAL SULFUR DRY DEPOSITION kg S ha
FALL 1980
STAIVOO i» UMACID
Figure 6.23 Autumn 1980 box plots of the predictions of sulfur dry deposition (kg
S/ha).
116
-------
RTM sulfur dry deposition amounts are much greater, while their air concentrations of
sulfur dioxide and sulfate are much lower than those of other models. This partially
explains why these models underpredict sulfur wet deposition amounts. RELMAP air
concentrations of sulfur dioxide again are much higher than other predictions.
6.3.3.2 Comparison of Predictions and APN Measurements
In general, with the exception of the statistical models, predictions of sulfate air
concentrations are within a factor of two of the measurements at the APN sites (Figure
6.24). Similar to summer and spring, the four statistical models tend to both
overpredict and predict the highest sulfate air concentrations. SERTAD, which once
again overpredicts by the greatest margin, overpredicts by a factor of ten at the two
APN sites closest to the major source regions. Also similar to other seasons, RTM has
a tendency to underpredict the sulfate air concentrations by 50%.
With few exceptions, the predictions of sulfur dioxide air concentrations are within
a factor of two of the measurements. Despite the doubling of the measured sulfur
dioxide air concentrations for autumn (compared to summer and spring), the RTM
predictions for autumn are very similar to the RTM predictions for the other two
seasons. Consequently, its predictions are low by as much as a factor of four.
As in the case for every season, RELMAP predictions of sulfur dioxide air
concentrations at C04 and COS are two to three times greater than those measured.
Also, for all models but SERTAD and MOE, the autumn predictions at COS, the Nova
Scotia site, are much higher than the measurements. SERTAD and MOE predictions
at this site are very close to the measurements for every season.
6.3.3.3 Spatial Analyses
The observed pattern of sulfur wet deposition for autumn (Figure 6.25) is quite
different than those for summer and spring. The pattern for this season shows two
distinct and distant areas of maximum values. Both are in Canada, one across
southwestern Ontario and the other across the Maritime provinces. In summer and
spring, the maximum observed values are typically found across Ohio, West Virginia
and western Pennsylvania.
Across the United States the values are relatively low compared to those in
Canada. Moreover, the gradients are much flatter than those for summer and spring.
Both of these features seem to reflect the shifting of the precipitation pattern
northward. Rather than the high precipitation amounts being over the major source
region of the United States (as is the case for summer and spring), the higher
precipitation amounts in autumn are located across Canada (see Figures B.4 - B.6).
117
-------
o
o.ujtt.QSOOO
-------
119
-------
o
\
120
-------
<
Q
V /
- g
^7// .
-- IVJ;/ J /?
(0
O
I
E
CO
O
tn
-121
-------
(A
O
I
cc
111
(A
00
O
z
5
D
0
-a
/>
cs
^d
-------
In general, the predicted patterns differ from the observed patterns more for the
autumn than they do for the summer or spring. Also, for autumn there are more
dissimilarities between predicted patterns.
Rather than a shift of the highest amounts to the north, as is observed, the highest
predicted amounts are located either
1) To the northeast of the area where the highest summer and spring amounts
are predicted or
2) Across the same general area as summer and spring.
The general shift of some patterns to the northeast could be, in part, a response to the
shift in maximum precipitation amounts over southern Canada.
With respect to the location of highest values, the autumn patterns of the
predictions from most models are more similar to their summer and spring patterns
than the observed pattern for autumn (Table 6.6). ASTRAP, MOE, RELMAP,
SIMPMOD and TVA still show their highest predictions across the high emissions
region.
The STATMOD pattern for autumn is quite similar to its summer and spring
pattern in that all exhibit a ridge of high values along the Ohio River and from West
Virginia eastward to the Atlantic Ocean. This model is consistent in predicting two
distinct areas of maximum values, one across the western segment of the Ohio River
and the other across West Virginia.
The patterns for the other models are quite different from their summer and
spring patterns. AES broadens and shifts its area of highest values northeastward
across much of Pennsylvania and New York. Likewise, RIVAD and RTM shift their
highest values northeastward. The location of the RIVAD area resembles that of
AES, while the RTM area is across Maine and Nova Scotia. Meanwhile, SERTAD
shifts its area to the north, across Lake Ontario and eastward to the Atlantic Ocean.
Finally, UMACID predicts its highest values across West Virginia, as before, but it also
extends this area northeastward to Lake Ontario.
The areas where each model significantly under/overpredicts are illustrated by
Figure 6.26. Again, the models can be categorized by their patterns of significant
differences using the same three types discussed previously (Table 6.7).
123
-------
TABLE 6.6. COMPARISON OF THE GEOGRAPHICAL REGIONS WHERE
EACH MODEL PREDICTS THE HIGHEST AMOUNTS OF SULFUR WET
DEPOSITION FOR SUMMER/SPRING VERSUS AUTUMN 1980.
MODEL
SUMMER/SPRING
AUTUMN
OBSERVED
AES
Ohio, Pennsylvania,
West Virginia
Ohio, Pennsylvania,
West Virginia
SW Ontario,
Maritime Provinces
West Virginia to
N New York
ASTRAP
MOE
RELMAP
RIVAD
RTM
SERTAD*
SIMPMOD
STATMOD
TVA
UMACID
W Pennsylvania
Ohio, W Pennsylvania,
Lake Erie
West Virginia
Ohio, Pennsylvania,
West Virginia
Ohio, Pennsylvania,
West Virginia
Ohio, Pennsylvania,
Lake Erie
Pennsylvania, West
Virginia, W Maryland
Ohio, West Virginia
Ohio, Pennsylvania,
West Virginia
Ohio, Pennsylvania,
West Virginia
no difference
no difference
no difference
Pennsylvania to
New Hampshire
Maine, Nova
Scotia
Lake Ontario
no difference
West Virginia to
the Atlantic Ocean
no difference
West Virginia to
Lake Ontario
See footnote in Section 6.1.
124
-------
z
5
3
<
Q.
cc
£
<
J&T
z
5
<
o.
Ul
c
z
5
D
UJ
O
5
11
f-^
11 i v~ i \
B
O
m
O
OH
-------
z
5
I
2
z
2
3
<
Q
O
s
c7>
5
D
<
O
Z
5
^
<
Q
g
ec
UJ
M
126
-------
z
5
I
z
I
<
0
o
5
127
-------
TABLE 6.7 TYPES OF PATTERNS OF SIGNIFICANT DIFFERENCES FOR
AUTUMN 1980.
TYPE
DESCRIPTION
MODELS
II
III
Significant differences nowhere
or limited to small widely
scattered areas
Significant overpredictions
across areas to the east and/or
south of the Ohio River Valley
Significant underpredictions
across continuous or sporatic
areas from the Midwestern
United States to Nova Scotia
RELMAP
STATMOD
TVA
UMACID
AES
ASTRAP.
SERTAD
SIMPMOD
MOE
RIVAD
RTM
See footnote in Section 6.1.
6.3.3.4 Model Performance Measures
Pattern Magnitude - Unlike the summer and spring, the autumn predictions of all
models do not significantly differ from the observations in more than 20% of the
composite region (Table 6.8). In addition, the predictions of all but three models
(AES, ASTRAP and SIMPMOD) do not significantly differ across more than 25% of
any subregion. Therefore, eight models receive the maximum score of four for pattern
magnitude. The other three models receive a score of two, since they each significantly
overpredict in more than half of the southeastern subregion.
SIMPMOD exhibits quite dissimilar behavior for each season. In summer, it
significantly underpredicts in more than 29% of the cells in three subregions, in spring
all predictions are insignificantly different than the observations, while in autumn, it
significantly overpredicts in nearly 80% of the cells in the southeastern subregion. The
inconsistent behavior could likely be a consequence of the fact that this model does not
directly use actual precipitation amounts, as explained previously.
The ASTRAP score for pattern magnitude could actually be zero if the effects of
the high variance of its predictions increase by 25% the number of cells in the
composite region where the predictions significantly differ from the observations.
Since the variance of the RELMAP predictions in autumn is small compared to that of
the observations, its score would not likely change as a result of these effects.
128
-------
g
oo
as
e|
p^
P"*
§
P>
2
£3
3
s?
MJ
IS
8
ee
-
_j
g
§
a
H i* o
^ d R -o
»
O 0 O £ £ fcK»K?K^oS5°-3 ID N. KI K)8 «0 O O K j?J
OOOfMO *-OOOOOOOO «0 s» OO KI O +
o* t*
rsjO'imiM «*>o**"*«O
tooAidtn «-dddddddd JC ^ do Kt ^
o rw o tn ^ R5SKt3Sfe83P K. M rg 5t tJ MOO rw o-
ooo*« -oooooooo ^ a « oo a .«.»
4.
o A 5 * A S§S5SSS3S «*. S3 So« »»
» S - *" * fM ' CV
O f fZN^* «~ P- mMog * m
rsiRddsr odod--oddd r%J S od to ^ KI>
s K
*"^^R^ ~OOOTOOO- .£., oo - 0; +
*
+ + + * *Vw<>tn»n'^^^K> O* m
iniAOav »o. £
I °
"^ )
o 5
2 *
||
o 'i
'§ |
« -
5 1
Q.
So5
3 5 'o
* W tt
j! u 3
8l-
1 1 f
t: £
X " 1
1 1 -
j £. 1
"Is
V 4>
3 £ u
MOO.
Il5l
p < «-> n
«> . M >
C. 3 »
O ^ T3
{*§!
Jt w V
y * ~* B
C + 4
31 > O
* :
129
-------
Based on the descriptive statistics alone, which do not consider the uncertainties of
the observations, five of the eight models receiving a score of four (MOE, RELMAP,
STATMOD, TVA and UMACID) could be identified as better performers. However,
RIVAD, RTM and SERTAD could receive a lower score if the descriptive statistics
were used alone. Those models receiving a score of two show the poorest results based
on the descriptive statistics.
As mentioned previously, a large percentage of the SERTAD predictions of sulfur
wet deposition are attributed to a background contribution. As an example, about
50% of the magnitude of the SERTAD predictions across Ohio are attributed to its
background contribution. This is based on a typical autumn precipitation amount for
Ohio of 120 mm (Figure B.6).
Pattern Position - Although the patterns of predictions visually appear to differ
from the pattern of observations for autumn (Figure 6.25), most models receive the
maximum score for pattern magnitude. This can be explained by the small range in the
observed values compared to the uncertainty of the observations. Even though the
positioning of the patterns is obviously different, much of the time, the predictions are
within the uncertainty envelope. The measures for pattern position, however, quantify
the pattern differences, underscoring the necessity for a balanced and comprehensive
set of model evaluation measures.
Only five models (ASTRAP, MOE, RIVAD, SERTAD and UMACID) have
centroids within the criterion value of 75 km from the observed centroid (Figure 6.27).
The centroids for the other models are as much as 180 km southeast of the observed
centroid.
The difference between the orientation of the major axes of the predictions and
observations does not exceed 10 degrees, the criterion value, for only four models
(AES, RELMAP, RTM and UMACID). The orientation differs the most (in excess of
20 degrees) for three of the four statistical models (SIMPMOD, STATMOD and
TVA). (As discussed previously, the attributes of the pattern for the other statistical
model, SERTAD, can be dominated by the precipitation pattern). The orientation of
the major axes for each model is rotated clockwise from the observed major axis. This
is consistent with the summer, but not the spring, results.
The differences in pattern positioning is apparent from the distances the predicted
patterns are shifted to maximize the correlation. In contrast to summer and spring, the
shifts in the autumn patterns exceed 400 km for all but three models, while for the
other two seasons, the shift distances exceed 250 km for only two or three models.
Pattern Shape - In contrast to summer and spring, and in concuurence with the
previously discussed values of the measures for autumn, the correlations for autumn,
ranging from -0.6 to 0.6 (Table 6.8), reflect the relatively poor agreement between the
spatial patterns of the predictions and observations. For six models (AES, RELMAP,
RTM, SIMPMOD, STATMOD and TVA) the correlations are negative, indicating
that these models tend to predict their highest values where the lowest values are
130
-------
NUMB!
Figure 6.27 Autumn 1980 locations of the centroids.
observed and vice versa.
Consistent with the summer and spring, the MOE and SERTAD correlations for
autumn, although they are less than 0.6, are the highest (in the positive sense) than
those of the other models. However, the correlations for UMACID, which has
correlations greater than 0.7 for summer and spring, has an autumn correlation of only
\JfJt
Pattern shifting dramatically increases the correlations for all cases but one (that
for RTM remains less than 0.3). This indicates that the RTM pattern shape is so
dissimilar to that of the observations that pattern shifting fails to significantly improve
the correlation. With the exception of AES, RELMAP and RTM, at least half the
variance is explained by the models after the patterns are shifted.
Maxima - As seen from Figures 6.25 and 6.28, the locations of the maximum
values are widely scattered, more so than for any other season. Only two models
(RTM and SERTAD) predict their maximum values in Canada. RIVAD locates its
maximum prediction in New Hampshire where the observed pattern shows a trough of
low values. Most other models locate their maximum predictions in or near West
Virginia.
131
-------
MAXIMUM AUTUMN SULFUR WET DEPOSITION, 1980 (kg S/ha)
"==3f
70= OBSERVED
MODELS!
= TVA
/2= AES
/3= SIMPMOD
/4= STATMOD
/5= MOE
/6= SERTAD
/7= RTM II
/8= RIVAD
79= ASTRAP
/A= UMACID
/B- RELMAP
2.8/UU44
2.8/2 fj
2.9/4'
4.5/3
.8/A
2.8/B
Figure 6.28 Autumn 1980 locations and magnitudes (kg S/ha) of the maximum
amount of sulfur wet deposition at the ISDME sites.
The values of the measure for the location of the maxima (Table 6.8) indicate that
only SERTAD passes the criterion that the predicted maximum be within 100 km of
the observed maximum. The location of the maximum predictions of the other models
is over 280 km from the nearest possible location of the observed maximum.
It is interesting to point out a feature of the kriging approach to compute the
distance between the predicted and observed maximum. The observed pattern for
autumn (Figure 6.25) shows two distinct and distant (i.e., greater than 1000 km) peaks,
the largest one near Toronto and the other one (about 30% lower in magnitude) over
Nova Scotia and New Brunswick. Even though the locations of the RTM and
SERTAD maxima are over 1000 km apart (the former being in Nova Scotia, while the
latter is near Toronto), the kriging approach does not ignore the fact that each location
is very near a possible location of the observed maximum (Table 6.8).
With the exception of RIVAD and RTM, the magnitudes of the predicted maxima
are within 25% of that observed. Consequently, all but these two models, which
underpredict the magnitude by about 50%, pass the criterion for the magnitude of the
maximum.
Temporal Pattern - The autumn contribution of the predictions are within 10% of
the observed autumn contribution (16% of the annual total) for only three models
132
-------
(MOE, RTM and TV A). Therefore, only these models pass the criterion. SERTAD,
SIMPMOD and STATMOD each overpredict the autumn contribution by nearly 40%.
6.3.4 Winter 1980 Results
6.3.4.1 Summary of Distributional Statistics
As seen from Figure 6.29 AES, ASTRAP and SIMPMOD tend to overpredict,
while RIVAD and RTM tend to underpredict both the magnitudes and the standard
deviation of the sulfur wet deposition amounts for winter. The second quartile
predictions of ASTRAP and SIMPMOD are greater than the third quartile
observations.
The winter precipitation pattern (see Figure B.3) is similar to that of autumn in
that precipitation amounts across the high emission regions are relatively low. As in
the case of autumn, this could partially explain the higher SIMPMOD predictions.
SERTAD predictions, which for the autumn were nearly as high as SIMPMOD's, are
much lower than SIMPMOD winter predictions.
Figures 6.30 - 6.32 indicate that for the winter, as for the other seasons, SERTAD
sulfate air concentrations, RELMAP sulfur dioxide air concentrations and sulfur dry
deposition predictions of AES, RIVAD and RTM are substantially higher than those
predicted by the other models. Consistent with other seasons, RIVAD and RTM
sulfate air concentrations are the lowest. In addition, MOE sulfate air concentrations
are relatively low.
6.3.4.2 Comparison of Predictions and APN Measurements
Also like the other seasons, the statistical models tend to both overpredict and
predict the highest sulfate air concentrations at the APN sites (Figure 6.33). Again,
SERTAD predictions are as much as six times higher than the measurements. With
the exception of the predictions of the statistical models and RELMAP predictions at
C04, the predictions are within a factor of two of the measurements.
It is interesting to note the small seasonal variations in sulfate air concentrations at
these sites. In contrast, the winter sulfur dioxide air concentrations increase by factors
ranging from 2.5 at C04 to nearly 6 at both C01 and C03.
With the exception of the RELMAP predictions, there are only six out of a
possible forty cases when the predictions of sulfur dioxide air concentrations differ
from the observations by more than a factor of two. The tendency for RIVAD to
133
-------
-I-
-I-
TOTAL SULFUR WET DEPOSITION kg S ha!
WINTER 1980
42 I 76 I
77'
I.
MODEL AES ASTRA? MOE RELMAP RIVAD RTM SERTAO SIHPMOO STATMOO TVA UMACIO VALIDATION
Figure 6.29 Winter 1980 box plots of the predictions and observations of sulfur wet
dc position (kg S/ha).
7
0
SULFATE CONCENTRATION, ^g/m3
WINTER 1980
i t
i i
I i
1 ~ ! 30
. « * 1 1
1 1 -'-
It III
26' 1 ' 43 -161
2 4i i - -* i i |
i l 1 7 t i
i i 1 88'
2 9 i
1
1
1
I
i
l
i
i
^T1
l
1 -,--
l 2 5|
AES ASTRAP MOE RELMAP RIVAD RTM SERTAD SIMPMOD STATMOD TVA UMACID
Figure 6.30 Winter 1980 box plots of the predictions of sulfate air concentrations
134
-------
w
80
60
40
20
0
0
1
1
1 1
1
...
.--
18 7| 1257' - 26
I | 105 |
I I 1
i 1 1
i
{
-
2
i-
)
.
SO2 CONCENTRATION. Mg/m3
WINTER 1980
0
0 0
0
1 0
1 1 1
-'- ""j . !., - '" ._ , ;
. i !
I ! ! 13^1 ' ---
s'eT' 9*71" ' '-- 127 i '--- !
1 1 861 1 9 01
III II
AES ASTRAP MDE RELMAP RIVAD RTM SERTAD SIMPMOD STATMOD TVA UMACID
Figure 6.31 Winter 1980 box plots of the predictions of sulfur dioxide air
concentrations (ng/m3).
2
B
AE
5
0
0
0
0
....
-
,7j" IB"' ,!-;-!
1 1 1
1
ASTRAP MOE REIMAP
7
Rl
/A
TOTAL SULFUR DRY DEPOSITION, kg S/ha
WINTER 1980
0
0
0
0
0
0
0
-_
«
2*21
. 15* 1*9
1 51
--- 1 81
1 41 I
I I
I I
) RTM SERTAD SIMPMOD STATMOO TVA UMACID
Figure 6.32 Winter 1980 box plots of the predictions of sulfur dry deposition (kg
S/ha).
135
-------
W
0-
<
4 e
(A
uj o. Q 5 Q QO<
0<
S >
ui *
oc
5 5
a. K
S <
M M
o
o
SULFATE
S02
Figure 6.33 Comparison of the winter 1980 observations and predictions of the air
concentrations (jig/m3) of sulfur dioxide and sulfate at theAPN sites.
136
-------
overpredict the sulfur dioxide air concentrations is not apparent at the APN sites in
winter, when its predictions are comparable to the measurements. On the other hand,
RELMAP predictions at C04 and COS are still three to four times higher than the
measurements.
6.3.4.3 Spatial Analyses
Like the autumn pattern of the observations (Figure 6.25), the pattern of the
winter observations (Figure 6.34) shows two distinct and distant areas of maximum
values, one of which is located across southern Ontario. However, for winter, the
second area is located across the southeastern United States. The southern area of
maximum values coincides with the peak amounts of precipitation (40 cm). The winter
precipitation amounts to the north of the Ohio River and Maryland are half as great.
Minimum values are found across eastern Canada, the northeastern United States and
Indiana.
Although the patterns of predictions show maximum values across Ohio, West
Virginia, western Pennsylvania and western Virginia, all patterns but those for MOE,
SERTAD, SIMPMOD and TV A show a secondary maximum or a ridge of high values
extending southwestward, similar to the observed pattern. The four exceptions
resemble concentric circles that, like the other models, are anchored near West
Virginia. However, each pattern has a ridge extending westward, where the observed
pattern shows a trough of low values.
A peculiarity is evident in the SERTAD pattern. The tight gradient across western
New York results from a very low prediction (i.e., lower than neighboring sites by an
order of magnitude). This is caused by a SERTAD programming error (see footnote
in Section 6.1). The depicted SERTAD pattern in Figure 6.34 demonstrates an
intrinsic attribute of the simple kriging technique applied to generate the spatial
patterns: the predicted value at the site is not ignored and the pattern exhibits the
resulting gradient.
The subregions where the models significantly under/overpredict are illustrated by
Figure 6.35. The patterns of signifcant differences are categorized in Table 6.9 using
the same three types as before. A fourth type is necessary for this season to describe
several models which significantly underpredict across the southern United States,
where the amounts of sulfur wet deposition are not normally comparable to those near
the high emission regions.
Finally, SERTAD does not consider a background contribution to its winter
predictions of sulfur wet deposition. In the absence of the influence of the low
prediction at the western New York site, the SERTAD pattern (Figure 6.34) would
resemble concentric circles and would show little detail. The comparison of this winter
pattern with those of the other seasons demonstrates the level of detail introduced by
the precipitation pattern, which is used in determining the relatively large background
contributions.
137
-------
CC
UJ
LU
o
bfi
o
I<
-4»
14
C/3
O
CX
V
o
3
C/D
M-l
O
on
o
-a
i
o.
<
CC
(A
T3
<0
u.
CX
-------
c
UJ
g
&
4
5
UJ
oc
z
o
I
cc
UJ
V)
to
O
oc
UJ
z
i
139
-------
EC
LU
H
i
O
2
CC
III
W
2 6
I
DC
LU
w
00
o
tr
LU
h-
z
5
140
-------
i
141
-------
fet
142
-------
z
5
5
tc
Q
O
z
i
Q
143
-------
I-
-a
-------
TABLE 6.9 TYPES OF PATTERNS OF SIGNIFICANT DIFFERENCES FOR
WINTER 1980.
TYPE
DESCRIPTION
MODELS
II
III
IV
Significant differences nowhere
or limited to small widely
scattered areas
Significant overpredictions
across areas to the east and/or
south of the Ohio River Valley
Significant underpredictions
across continuous or sporatic
areas from the midwestern
United States to Nova Scotia
Significant underpredictions
across the southern United
States
SERTAD
TVA
UMACID
AES
ASTRAP
RELMAP
SIMPMOD
STATMOD
RIVAD
RTM
MOE
RIVAD
See footnote in Section 6.1.
6.3.4.4 Model Performance Measures
Pattern Magnitude - The predictions of nine models (all but AES and SIMPMOD)
significantly differ from the observations in less than 20% of the composite region
(Table 6.10). Further, the predictions of MOE, SERTAD, TVA and UMACID
significantly differ in less than 5% of the composite region and not more than 25% of
any subregion. Only these four models receive the maximum score of four for pattern
magnitude, since the others have significant differences in more than 25% of at least
one subregion.
Three of these nine models have predictions which significantly differ from the
observations in over 25% of only one subregion and, therefore, receive a score of two.
RELMAP significantly overpredicts across half of the southeast subregion. Both
RIVAD and RTM significantly underpredict across the north central subregion.
145
-------
o
oo
rH
£
td
M
Zi
S
Qi
S
MH
oo
rV
D
t/1
M
UJ
**
V
t-
V)
*
(A
U(
S3 ^ ~
- d fc 2
O O «- <* CM *-OOOOOOOO Olft OO N -*u^fin^'r5Srst«O M -^ h* »- 3 h- O O K» -
OOOO^KI vOOOOOOOO O* JF OO »~ -O
1 + 1 »-
dr^ftid*-' do'do'ddddd ^dC "" . oS^fe *^fc
rg^ **ooru>iNJ«o rJo'do«-ddd<- «-* od ** * o***
ONfti/^rt. S§ gfcoo^SiS ««:« ^^ aS°* °1
o'ov-mo rOotNjdoooON *~ i/% S oo W + + «"» +
-* ao *M . + N
I
1 !. I I S f 1 I! -L.lft
u'^tf*' - f n w* iu «"» J7 o 5 «- '^ ^ y
'c*^«**Wl*'w M *rf ^1 O> TI ^*^ t . V^T3C "i £ $
C*<*>*<«J ^'^"l!a'&'$!aL "S X 5 *j ** 1- C C **"
£ *ouoo33 e- 2 -r 3 w 'o*" 5 i *^ 1* t S S
illlj^j-d^ Us £ !£ ! Illl Is. 55 Irl^fl
^ I . 8...1..,2ffisSI
*v Q.
-------
The remaining four models (AES, ASTRAP, SIMPMOD and STATMOD) scora
zero points for pattern magnitude. AES and SIMPMOD significantly overprediu
across more than 40% of the southwest and southeast subregions. ASTRAP and
STATMOD significantly overpredict across more than 25% of both the southwest and
southeast subregions.
For winter, the descriptive statistics alone can generally distinguish between those
models receiving four and zero points for pattern magnitude. With few exceptions, the
four models scoring four points for pattern magnitude (MOE, SERTAD, TVA and
UMACID) have the highest p values for the median, dispersion and biases (i.e., the
lowest probability that these measures for the predictions are significantly different
from those of the observations) and the lowest mean squared errors.
However, on the basis of the descriptive statistics alone, RELMAP, despite its
significant overpredictions across half of one subregion, and STATMOD, despite its
significant overpredictions across a third of two subregions, would appear to perform
as well as those models which significantly under/overpredicted across only 5% or less
of the composite region.
For STATMOD, the bias across the high and low deposition regions (based on
ISDME site values) is low (Table 6.10), yet at the few sites south of the Ohio River the
residuals are much greater (see Table D.7). The overpredictive nature of this model
across this large region (Figure 6.34) is masked simply by the sparseness of the
monitoring networks.
There are no affects on the pattern magnitude scoring of ASTRAP, since it
receives a score of zero for the winter. Likewise, the RELMAP score is not affected,
since the variance of its winter predictions is small compared to that of the
observations.
Pattern Position - Similar to autumn, the pattern position of the winter predictions
generally does not compare favorably with that of the observed. However, two models
(ASTRAP and SERTAD) pass the criteria for both the centroid and the orientation of
the major axis. Both predict the centroids within 75 km of the observed centroid
(Figure 6.36) and orient their major axes within 10 degrees of that of the observations.
Two other models (RIVAD and TVA) pass only the criterion for the centroids.
On the other hand, AES, RELMAP and UMACID pass only the orientation criterion.
The remaining models (MOE, RTM, SIMPMOD and STATMOD) pass neither
criterion for pattern position. The relatively large differences in the orientation of the
major axes of the MOE, SIMPMOD and STATMOD predictions and observations
reflect the west-to-east ridges of high values depicted in the predicted patterns
illustrated in Figure 6.34.
147
-------
NUMBER
1
2
3
4
5
6
7
8
9
10
11
12
MODEL
AES
ASTRAP
MOE
RELMAP
RIVAD
RTM-II
SERTAD
SIMPMOD
STATMOO
TVA
UMAdD
VALIDATION
Figure 6.36 Winter 1980 locations of the centroids.
Similar to the other three seasons, as Figure 6.37 illustrates, the difference
between the orientation of the major axes of the winter observations and predictions is
greatest for STATMOD. Also, similar to the summer and autumn, the orientation of
the major axes of two other statistical models (SIMPMOD and TVA) as well as MOE
is rotated clockwise 11 to 21 degrees.
Since the orientation of the major axis likely reflects the mean transport vectors
during precipitation events, the approach of the statistical models in simulating
transport (i.e., long-term wind roses) apparently is not as accurate as the deterministic
approach of the other models that use rawinsonde data. These long-term wind roses
could represent the frequency distribution of wind velocities during the entire period,
but could differ for those periods consisting only of precipitation events. The MOE
model is the only deterministic model not using rawinsonde data.
Figme 6.37 also illustrates the seasonal variability of the orientation of the major
axes of the observations. It is not known how the orientation varies seasonally for
years other than 1980. In both the winter and summer of 1980 the major axis is
oriented approximately 40 degrees from North. However, in spring the orientation is
50 degrees while for autumn it is 30 degrees. With the exception of autumn, UMACID
duplicates this pattern. AES, RELMAP, RIVAD and RTM mimic general features of
the pattern. However, for the other models, the seasonal variations of the orientation
of the major axes are confined to a narrow band of 5 degrees.
148
-------
WINTER
SPRING
SUMMER
AUTUMN
SEASON
MODEL +++ AES
EH3E) OBSERVED
IMMtRTIHI
STATHOO
X--X-X ASTRAP
RELMAP
SERTAD
TVA
AA-ARIYAD
SIMPMO
UMACIO
Figure 6.37 Seasonal variation of the orientation of the major axes of the annual 1980
predictions and observations of sulfur wet deposition.
The high values for the performance measure of pattern shift to maximize the
correlations provide additional evidence of the poorer winter comparison of predicted
to observed pattern positions. Predicted patterns are shifted in excess of 450 km for all
but two models (MOE and TVA). One factor explaining the higher winter values is
the fact that the effects of pattern rotation are not considered here. As can be seen
from Figure 6.34, most predicted patterns appear to be substantially rotated from the
observed pattern. Because of its potential to yield inconsistent results, this measure is
not used to score the performance of the models. It can, however, yield results that
confirm what is apparent from the patterns (e.g., the winter pattern of UMACID).
149
-------
Pattern Shape - Like autumn, the winter correlations are very low. The MOE
correlation again is the highest, although it is less than 0.4. All other correlations are
less than 0.2, indicating the poor agreement in the shape and/or position of the
patterns of predictions and observations.
The maximum correlations resulting from pattern shifting, all lower than 0.8, are
lowest for winter than any other season. The correlations of only AES, RELMAP,
RIVAD, RTM and UMACID explain at least half of the variance.
Maxima - As the case for spring, the models locate their maximum predictions
across the area within 150 km of the southwestern corner of Pennsylvania (Figures 6.34
and 6.38). The STATMOD pattern exhibits a second area of maximum values across
eastern Tennessee and Kentucky. Conversely, the location of the observed maximum
value is much further to the northwest, across southwestern Ontario.
There are only two models (ASTRAP and MOE) that predict their maximum
values within 100 km of the location of the observed maximum value, the criterion
value. However, the locations of the maximum predicted values of the other models
are within 260 km of that observed.
MAXIMUM WINTER SULFUR WET DEPOSITION, 1980 (kg S/ha)
/O= OBSERVED
MODELS
TVA
/2= AES
/3= SIMPMOD
STATMOD
MOE
/6= SERTAD
77= RTM II
/8= RIVAD
ASTRAP
4.1/9/ 1.7/7* U44
3.5/B/v/
6.1/2
3.9/4
6.3/3
3.2/A
1.6/8
/A= UMACID
/B= RELMAP
Figure 6.38 Winter 1980 locations and magnitudes (kg S/ha) of the maximum amount
of sulfur wet deposition at the ISDME sites.
150
-------
The same two models, as well as RELMAP, SERTAD, STATMOD, TVA and
UMACID, predict the magnitude of the observed maximum value within the
uncertainty range. The magnitudes of the RIVAD and RTM maxima are within 30%
of the uncertainty range as determined by the kriging approach. Both approaches
indicate that the magnitude of the AES and SIMPMOD maximum predictions is more
than 30% greater than the upper limit of the uncertainty range. Therefore, nine
models pass the criterion for the magnitude of the winter maximum value.
Temporal Pattern - The range of the relative contributions of winter predictions,
12.5 to 32.0%, is highest for winter. The SIMPMOD winter contribution doubles the
16% observed contribution, while AES and STATMOD winter contributions exceed
that of the observed by over 50%. Only the MOE, RELMAP, RIVAD and SERTAD
winter contributions are within 15% of that observed and, therefore, only these models
pass the criterion for this measure.
6.4 Consistencies of Seasonal and Annual Results
The eight deterministic models of the ISDME compute annual predictions by
summing or averaging predictions for the four seasons. The statistical models, which
rely on mean durations and frequencies of wet and dry periods, compute seasonal and
annual quantities independently. Thus, for the latter models, the aggregation of the
seasonal predictions are inconsistent with the annual predictions.
If model predictions are to be used for the evaluation of control strategy scenarios
and for the assessment of ecological effects on a seasonal as well as an annual basis, it
is essential that the seasonal predictions are consistent with the annual predictions. A
simple check of seasonal and annual amounts of predicted wet deposition is performed
to identify those models which have annual results inconsistent with the seasonal
results. At each of the ISDME evaluation sites, the difference is computed between
the sum of the seasonal and the annual sulfur wet deposition predictions. Minor
differences can be attributed to round off errors.
All models except SERTAD, STATMOD and TVA (all of which are statistical
models) have a mean difference of less than 3% and at any site a difference of less
than 10%. The mean differences for SERTAD and STATMOD are within 5%, but
that for TVA is 11.3%. The site differences for these models are as great as 15 to 25%.
151
-------
7. Model Clustering
7.1 Introduction
The preceeding section discussed the values of the model performance measures
for each season of 1980. As mentioned in Section 4.4, the values of six of these model
performance measures, referred to as primary performance measures, namely,
1) Area of significant differences (pattern magnitude)
2) Difference in the location of the centroids (pattern position)
3) Difference in the orientation of the major axis (pattern position)
4) Difference in the location of the maximum value (maximum)
5) Difference in the magnitude of the maximum value (maximum)
6) Difference in the seasonal contributions (temporal pattern)
were compared to the criteria values in Table 4.3.
As a means of comparing model performance, seasonal scores were assigned to
the models on the basis of the number of these primary model performance measures
passing the prescribed criteria (see Table 4.3). With the exception of the first primary
measure in the list above, a model received one point for each primary measure
passing the criteria. For the measure of the area of significant differences, a model
received a score of four points when the more stringent criteria were satisfied or two
points when the less stringent criteria were satisfied. The fact that this measure
assesses
1) The significance of the differences between the predictions and observations at
every half-degree cell and
2) The differences of more than one attribute of the spatial patterns (i.e., it
actually assesses, to varying degrees, pattern magnitude, position and shape)
153
-------
justifies the higher weight, since the other primary measures focus on a single attribute
of the patterns.
The maximum score a model can receive for the seasonal and overall performance
is nine. For the annual performance, the maximum score is only eight, since the
primary measure for temporal pattern is not considered in the assessment.
The remainder of this section intercompares the seasonal, overall and annual
performances of the models. The subsequent discussions focus on the overall model
performance and the degree of consistency of the seasonal performances of each
model. In addition, explanations of the seasonal and overall performances are
discussed.
7.2 Overall Model Performance
The assessment of overall model performance is based on overall scores calculated
by summing weighted seasonal scores. The set of weights, which reflects the
seasonally of the observations of sulfur wet deposition, is determined by spatially
averaging the relative seasonal contributions 01 the observations (see Figure 6.1).
Accordingly, the set of weights is
0.39 for summer
0.29 for spring
0.16 for autumn
0.16 for winter.
Consequently, those models which perform well for summei and spring, the seasons
when the amounts are greatest, will tend to excel in their overall performance.
As mentioned in Section 4.4, the overall performance is a better gauge of model
performance and reliability than only the performance of the model in replicating
annual patterns of sulfur wet deposition. An assessment of model performance based
on its overall performance, as opposed to only its annual performance, at least partially
addresses the question, did the model predict the right values for the right reason? In
other words, did the model predict the highest seasonal deposition amounts when they
were observed? An assessment based on overall model performance, therefore, will
address an additional attribute of the model the seasonally of its parameterizations.
If a model does not mimic the observed seasonality, then its annual predictions, even
though they concur with observations, were probably obtained for the wrong reasons.
154
-------
Table 7.1 presents the seasonal, overall and annual model performance scores for
pattern magnitude and position, maximum value, and temporal pattern, as well as the
total score. As illustrated by the distribution of overall scores in Figure 7.1, there
appears to be three clusters of models with similar scores. Group A models have
overall scores greater than 7, Group B models have scores between 5 and 7 and Group
C models have scores lower than 5. The following subsections discuss the performance
of the models in each cluster.
7.2.1 Group A Models
MOE, SERTAD and TVA each have overall performance scores greater than 7 as
a result of their consistently high seasonal scores. Each model scores the maximum
points (4) for the pattern magnitude for each season and MOE and TVA scored the
maximum number of points (9) for spring (Table 7.1).
Based only on its performance scores, SERTAD appears to be one of the better
performing models (see Footnote). However, the following aspects of the model and,
in particular, its application in this "blind test" evaluation study can lead one to wonder
if its relatively good performance was achieved for the wrong reasons:
1) The contribution of its seasonal "correction" terms to the prediction of sulfur
wet deposition was substantial (e.g., 40 to 50% of the magnitude of its
predictions across the high deposition region of North America are attributed
to the correction terms). These correction terms were proportional to the
product of the seasonal/annual precipitation amounts and a constant
seasonal/annual concentration of sulfate in precipitation that was
approximately three times greater than those observed at remote sites. In
addition, the values of these correction terms were computed from data
collected during a span of several years, including the ISDME evaluation year.
2) Its high initial conversion of sulfur dioxide to sulfate near the sources (i.e., 20
to 40% of the mass of sulfur dioxide is immediately converted to sulfate) was
much greater than the other models.
3) The model ignores the dry deposition of sulfate.
4) Its predictions of seasonal and annual mean air concentrations of sulfate at the
four APN sites are greater than those observed by factors of two to ten. In
addition, these predictions at the other ISDME sites are at least a factor of
two greater than those of the other models.
Footnote: The effects on the performance scores of a SERTAD postprocessor
programming error, brought to the authors' attention just prior to the publication of
this report have not been quantified (see footnote in Section 6.1)
155
-------
GO
W
O
u
u
w
Q
O
2
Q
g
O
O
CO
U
GO
s
w 9> oocr9<»-*
a- Ceo o
< ° < r r "
x < (- £ f- u
0861 M31NIM
E -
i-rooo^oo-*o
e * a a
= a. a. C o O O
|s <xoig<<<
l! <
-------
Figure 7.1 The distribution of the seasonal, overall and annual 1980 model
performance scores.
157
-------
Several additional paints concerning the Group A models are noteworthy. First, a
statistical model (TVA) performs better or as good as the deterministic models. The
former models simulate mass transport on the basis of long-term wind roses (i.e.,
probability distributions of the wind velocity), as opposed to the deterministic models,
which base their mass transport on series of short-term trajectory segments determined
from actual wind data. The fact that a statistical model performs rather well indicates
that seasonal average wind roses can be used to successfully simulate, within the
uncertainty bounds of the data, seasonal and annual patterns of sulfur wet deposition,
provided that the other processes are appropriately simulated.
Secondly, it should be restated here that, by virtue of the fact that statistical
models predict annual values independent of their seasonal predictions, the annual
TVA predictions are inconsistent with their accumulated seasonal predictions (see
Section 6.4). Therefore, one needs to be cautious when using both seasonal and
annual predictions from statistical models.
Finally, MOE, which is the only deterministic model in Group A, is also the only
deterministic model in this model evaluation that does not use rawindsonde data to
simulate mass transport. Instead, MOE transport is based on geostrophic winds
computed from surface pressure fields. Also, it is only one of two models to use a finer
resolution for the precipitation data (3-h amounts versus those for 12 or 24 h). Since
the rates of the physical and chemical processes are very similar to those of the other
deterministic models, perhaps the method in which transport and precipitation are
simulated can explain the relatively higher score of this model.
7.2.2 Group B Models
Unlike the Group A models, AES, ASTRAP and UMACID perform well for
several seasons, but perform rather poorly for one or two of the other seasons. Their
inconsistent seasonal performances precludes these models from earning overall scores
as high as those for the Group A models.
Both AES and ASTRAP perform very well for spring (8 and 9 points, respectively)
and relatively well for summer (6 points each). However, primarily because they
significantly overpredict across more than half of the southeast subregion, both models
score only 4 or less points for autumn and winter. The overpredictive nature of AES
for autumn and winter can be explained by the lack of seasonal variation of its wet
deposition rates of sulfur dioxide and sulfate. As Figure 6.1 illustrates, there is a strong
seasonality in the observations that AES does not mimic.
On the other hand, ASTRAP has a tendency to significantly overpredict across
much of the southeast subregion regardless of season. In addition to the autumn and
winter, it significantly overpredicts across nearly half of the same subregion in summer.
Since this subregion is typically downwind of the high emission regions, it seems that
this model removes sulfur at a rate significantly higher than the actual rate. However,
this model does not appear to significantly underpredict across the areas further
158
-------
downwind, which is what one might expect. ASTRAP does tend to underpredict at
distant sites (see Tables D.7-D.10), however, the differences are insignificant when the
uncertainties of the observations are considered. The magnitude of these uncertainties
relative to the observations are highest where the density of sites is lowest, and for this
case, where the model underpredicts.
Unlike AES and ASTRAP, UMACID performs best in autumn, winter and spring
(6 or 7 points each). Its summer performance (4 points) is marred by significant
underpredictions across 20% of the composite region and particularly across the Great
Lake states (nearly a third of the southwest subregion) and Maritime provinces. This
model also significantly underpredicts across nearly a third of the southwest subregion
in spring. Like AES, the UMACID rates of the wet removal of sulfur dioxide and
sulfate do not vary seasonally. Apparently, the prescribed rates are appropriate for
several, but not all, seasons.
7.2.3 Group C Models
On the basis of their low scores for overall model performance, the remaining five
models (RELMAP, RIVAD, RTM, SIMPMOD and STATMOD) are classified as
Group C models. All but RIVAD and RTM perform relatively well for only one
season. The two exceptions perform consistently poorly for each season.
RELMAP performs best in autumn when the model receives the maximum score
for pattern magnitude. However, the model performs poorly in replicating the pattern
position, location of the maximum value and the autumn contribution to the annual
total.
The low overall performance score from RELMAP is likely attributable to its high
rate of the wet removal of sulfur, as demonstrated by its tendency to significantly
overpredict in the southeast subregion. RELMAP significantly overpredicts across
90%, 60% and 50% of the southeast subregion in summer, spring and winter,
respectively. In autumn, however, the model significantly overpredicts across less than
10% of the same subregion. The inconsistency in the model performance can be
explained in part by the variation in the seasonal precipitation patterns, illustrated by
Figures B.3-B.6. Across the subregion where RELMAP tends to overpredict, the
autumn precipitation amounts are much lower than those for the other seasons
(approximately half those for summer and spring and one-third those for winter). As a
consequence, the effects of the high removal rates diminishes for this season.
SIMPMOD performs very well for spring (8 points), but poorly for the other
seasons (0 to 3 points). In spring, this model does not significantly under/overpredict
anywhere, replicates well the pattern position and predicts well the location and
magnitude of the maximum value. However, for summer, this model significantly
wmterpredicts across 30% of the composite region and across 30 to 40% of each
subregion but the southeast. In contrast, for both autumn and winter, it significantly
overpredicts across nearly all of the southeast subregion (nearly 80% and 100%,
159
-------
respectively) and across large portions of the southwest subregion (20% and 60%,
respectively).
Since SIMPMOD does not directly use precipitation amounts, the reasons are not
apparent for its inconsistent seasonal performances. The fact that, for every season,
SIMPMOD (and STATMOD) wet removal rates of sulfur dioxide are an order of
magnitude lower, while those for sulfate are factors of two to three lower than those of
the other statistical models (SERTAD and TV A) would appear to explain the
underpredictions in summer. However, this explanation is not valid, since for spring
the model performs well.
The tendency of SIMPMOD to overpredict in autumn and winter could be related
to the manner in which the precipitation statistics (i.e., the mean duration, frequency
and amount) were determined. If the statistics reflected the annual period, as opposed
to individual seasons, the lower precipitation amounts observed across the United
States in autumn and winter would not be considered by the model. Consequently, in
autumn and winter the model would overpredict across much of the United States,
which it does.
The performance of STATMOD is similar to SIMPMOD. The two models are
identical with the exception that the former uses seasonal precipitation rates in
simulating wet removal. Like SIMPMOD, STATMOD in summer significantly
Httderpredicts across large portions of the composite region (40%) and every subregion
but the southeast. Also as SIMPMOD, STATMOD significantly ove/predicts across
much of the United States in autumn and winter.
In contrast to SIMPMOD, however, STATMOD in spring significantly
underpredicts across a large portion of one subregion. The reason for this difference is
not apparent.
The low overall performance scores of SIMPMOD and STATMOD can also be
attributed to the fact that their spatial patterns appear to be shifted, as the other
models, but also rotated more than the others. The rotation, which is visible in the
figures of Section 6), is also obvious in the orientation of their major axes. For almost
every season, the SIMPMOD and STATMOD major axes are oriented more west to
east compared to the general south west-to-northeast orientation of the observed major
axes. Another factor in their low scores is that they both fail to mimic the seasonal
variation of the observations (Figure 6.1).
Finally, the poor performance of RIVAD and RTM is attributed to the significant
underpredictions across extensive areas of the evaluation region, especially for summer
and spring. However, for these two seasons, both models receive maximum scores for
the performance measures that do not relate to magnitudes: Their pattern positions,
locations of the maximum values and the temporal patterns correspond to those of the
observations. This suggests that there is a systematic problem with the wet removal
rates. Since their predictions of sulfur dry deposition are higher than those of the
other models, perhaps these two models are removing too much mass via dry
160
-------
deposition. In addition to their significant underpredictions, the predicted patterns of
both models are excessively smooth.
7.3 Annual Model Performance
In contrast to the overall performance, the annual performance is assessed
independently of the seasonal performances. Only annual predictions of sulfur wet
deposition are compared to annual observations. The maximum annual performance
score is 8 versus 9, since the measure for temporal pattern replication does not apply.
The three Group A models, which have the highest overall performance scores,
also have the highest annual performance scores. In addition, MOE scores the
maximum number of points. MOE and SERTAD, at least for 1980, predict with equal
success the seasonal and the annual sulfur wet deposition amounts. However, TVA
does not perform as well for the annual period, which suggests that the wind roses
and/or statistical input data used for the annual simulation might not be as
appropriate as those for the seasonal simulations.
By varying degrees, all but one of the Group B and C models (SIMPMOD) have
annual performance scores lower than their overall performance scores. One
explanation is the fact that there is an incommensurate maximum number of points
(i.e., the maximum number of points for the overall performance score is one greater).
161
-------
8. SUMMARY AND CONCLUSIONS
8.1 The ISDME Approach
The International Sulfur Deposition Model Evaluation (ISDME) project, jointly
conducted by the U.S. Environmental Protection Agency and Atmospheric
Environment Service of Environment Canada, assessed the performance of eleven
linear-chemistry atmospheric models for the entire year of 1980. This year was
selected for the eastern North American evaluation on the basis of the availability of a
spatially and seasonally resolved emissions inventory and an adequate precipitation
chemistry data base.
The approach of the study was to first compile and distribute to the participating
modelers standardized sets of model input data grids consisting of 6-hour 1000- and
850-mb wind velocities, 12-hour precipitation amounts, climatological mixing heights,
and vertically stratified seasonal emissions of sulfur dioxide. The use of this
standardized data set by all modelers would minimize the differences in model results
arising from differences in model input data. Modelers then applied their models in a
"blind" test mode and submitted to the evaluators predictions of wet and dry deposition
amounts and air concentrations of sulfur dioxide and sulfate at up to 65 sites.
The evaluation data set used to assess the performance of these models consisted
of only sulfur wet deposition amounts, since measurements of other parameters were
either nonexistent or sparse. These amounts were calculated from screened
precipitation chemistry data from five episodic, weekly or monthly networks. With
several slight variations, the screening and calculation procedures were those
recommended by the Unified Deposition Data Base Committee established under the
Canadian Federal-Provincial Research and Monitoring Co-ordinating Committee.
The number of sites passing the screening criteria ranged from 32 for the annual to 46
for the spring periods.
The models were evaluated in an operational mode using a new, more rigorous
approach, which involved i) the statistical comparison of spatial patterns rather than
sets of point values, ii) the consideration of the measurement and interpolation errors
of the observations, and iii) the assessment of seasonal rather than only annual
performances. In addition, the more conventional distribution statistics recommended
by the American Meteorological Society were computed.
The unique ISDME approach of assessing model performance elucidated the
behavior of the models in ways that conventional statistical measures alone could not.
For example, in addition to concluding that a model tended to under/overpredict by a
specific amount, as conventional statistical measures would yield, the ISDME results
also identified the regions where the model predictions were significantly greater or
163
-------
lower than the observations.
Spatial patterns of seasonal and annual amounts of sulfur wet deposition were
determined by an interpolation technique known as kriging. This technique quantified
the uncertainties in the observations which were used in the evaluation process to
identify areas where interpolated predictions were statistically significantly different
from the interpolated observations.
Of the many model performance measures that were quantified, six were selected
to evaluate the models. They were the area of significant differences and the
discrepancies between the observed and predicted centroids, major axis orientation,
maximum location, maximum magnitude, and seasonal pattern (i.e., the distribution of
the annual amount across the seasons). These measures were perceived to be
appropriate for ascertaining the differences of major features of the predictions and
observations. Model performance criteria were defined for each of these measures
and used in a scoring system to summarize, as well as compare in relative terms, the
performance of the models. To supplement the evaluation, predictions of dry
deposition amounts and air concentrations of each model were intercompared to
identify apparent peculiarities.
8.2 Comparison of Predictions and Observations
The seasonal patterns of the observed amounts of sulfur wet deposition exhibited
detailed structure with absolute and secondary maxima. Depending on the season, the
location of the absolute maxima ranged from northern West Virginia and central Ohio
to southern Ontario. The magnitude of these maxima, also seasonally dependent,
ranged from 3.2 (winter and autumn) to 7.9 kg S/ha (summer). Although they might
not be significant in light of the data uncertainties, several centers of minimum and
secondary maximum values appeared in the spatial patterns.
In contrast, the spatial patterns predicted for each season by the ISDME models in
general
1) Resembled patterns of concentric ellipses;
2) Exhibited much less detail than the observed patterns (i.e., the predicted
patterns were relatively smoother than the observed patterns and generally did
not show secondary maxima);
3) Anchored their area of seasonal and annual maximum values within 150 km of
the southwestern corner of Pennsylvania, which coincided with the area of
high sulfur emissions. (This indicated that the predictions from these linear
sulfur deposition models were dominated by the emissions pattern); and
164
-------
4) Have major axes that were oriented more west-to-east than those of the
observed seasonal patterns, which generally were oriented southwest-to-
northeast.
Half-degree cells where the interpolated predictions significantly differed from the
interpolated observations were identified as one means of objectively assessing model
performance. Differences were deemed to be significant at approximately the 95%
confidence level when the interpolated prediction fell outside the uncertainty limits of
the interpolated observation. This assessment indicated that
1) Interpolated predictions from SERTAD and TVA were within the uncertainty
range of the interpolated observations virtually everywhere for each season
but autumn for SERTAD and summer for TVA;
2) Interpolated predictions from the remaining models differed significantly from
the interpolated observations across certain regions for three or four seasons:
a) RIVAD and RTM, which predicted very smooth patterns, significantly
underpredicted across much of the region from the midwestern United
States to Nova Scotia;
b) ASTRAP, regardless of season, significantly overpredicted across portions
of the region over and to the east of the Ohio River Valley (the spatial
extent, however, varies seasonally);
c) RELMAP, similar to ASTRAP, significantly overpredicted across portions
of the region to the east and/or south of the Ohio River Valley for every
season but autumn;
d) MOE significantly underpredicted across portions of the southern and
midwestern United States to Nova Scotia for every season but spring when
virtually everywhere its predictions were within the range of uncertainty of
the observations;
e) AES predicted within the uncertainty limits of the observations virtually
everywhere in the spring and significantly underpredicted in summer across
portions of the midwestern United States and New Brunswick. The model
significantly overpredicted in the autumn across the region from Virginia to
New Hampshire and in the winter across the region from northern
Alabama to Pennsylvania;
f) UMACID predicted within the uncertainty limits of the observations
virtually everywhere in the autumn and winter. The model significantly
165
-------
underpredicted across a portion of the midwestern United States in the
spring and summer. In addition, summer values were significantly
underpredicted across the Maritime provinces; and
g) SIMPMOD and STATMOD, which did not replicate the seasonal temporal
patterns of sulfur wet deposition,
o Predicted within the uncertainty range of the observations for one season
(spring for SIMPMOD and autumn for STATMOD);
o Significantly underpredicted across portions of the area from the
midwestern United States to Nova Scotia for some seasons (summer for
each as well as spring for STATMOD);
o Significantly overpredicted across portions of the region east of the Ohio
River Valley for the other seasons (winter for each as well as autumn for
SIMPMOD).
Since at any point the uncertainty range of the interpolated observations is an
inverse function of the density of the monitoring networks within the neighborhood of
the point, the uncertainty range would diminish as the density of the networks
increases. Consequently, for years subsequent to 1980, when more monitoring sites
were operating, the uncertainty range is expected to diminish.
8.3 Model Performance Scores
As a means of assessing and intercomparing model performance, criteria were
established for six of the ISDME model performance measures (area of significant
differences and discrepancies between predicted and observed centroids, major axis
orientation, maximum location, maximum magnitude, and temporal pattern). The
criteria values of these measures approximated the median value of the distribution of
the seasonal values for all models.
For each season, model performance scores were assigned to the models on the
basis of the number of these performance measures passing the prescribed criteria.
The maximum score was nine (i.e., four points for the measure of the area of
significant differences and one point for each of the five remaining measures). Overall
performance scores based only on the seasonal performances were determined from a
weighted sum of seasonal performance scores. The seasonal weights were determined
from the mean relative contributions of each season to the observed annual amount of
sulfur wet deposition (i.e., approximately 0.40 for summer, 0.30 for spring, and 0.15 for
both winter and autumn).
166
-------
The distribution of overall performance scores identified the following three
groups, or clusters, of models:
1) Group A models, namely MOE (deterministic) and SERTAD and TV A (both
statistical), having overall performance scores exceeding 7, performed
relatively well for each season, especially for spring.
MOE performed somewhat better than the other deterministic models. Since
the transformation and removal rates were similar to those of the other
deterministic models, a possible explanation for the differences in the
predictions could relate to the different manner in which transport and
precipitation were simulated by this model.
Although SERTAD predictions of sulfur wet deposition were within the
uncertainty limits of the observations, several unique features of the model
deserve comment. First, the values of the "correction" (background) terms,
used in spring, summer, autumn and annual simulations, were proportional to
the product of the seasonal/annual precipitation amounts and constant
seasonal/annual background concentrations of sulfate in precipitation that
were approximately three times greater than those observed at remote sites.
These terms contributed 40 to 60% of the model's predicted sulfur wet
deposition, which was considerably greater than that of the only other model
that considered a background term. Secondly, SERTAD sulfate air
concentrations were always greater than the predictions of other models and
greater by a factor of two to ten than the observations at the four APN sites.
These effects were attributed to the high initial conversion of sulfur dioxide to
sulfate near the source (20 to 40%) and the lack of a dry deposition sink term
for sulfate.
2) Group B models, AES, ASTRAP and UMACID, having overall performance
scores between 5 and 7, performed relatively well for two or three seasons.
The lack of a seasonal variation in the rates of sulfur wet removal for AES and
UMACID were likely factors in their inconsistent performance. The
ASTRAP rates of sulfur wet removal appeared to be too high, since its
overpredictions of sulfur wet deposition occurred across and to the east of the
major source region along the Ohio River Valley.
3) Group C models, RELMAP, RIVAD, RTM, SIMPMOD and STATMOD,
having overall performance scores of less than 5 points, performed relatively
well for only one or two seasons. Based on its patterns of significant
overpredictions, the RELMAP rates of sulfur wet removal appear to be too
high for at least summer and spring. Its predictions of sulfur dioxide
concentration also were too high. This overprediction was caused by the
inadvertent injection of sulfur dioxide emissions into the surface layer at
night.
167
-------
Although both RIVAD and RTM tended to significantly underpredict
amounts of sulfur wet deposition for each season, the models performed
relatively well for the measures not pertaining to the magnitude of the
predictions. This could be indicative of a systematic error in the models. As
evidenced by the fact that their predictions of the air concentrations were
lower and sulfur dry deposition were higher than those of the other models,
the RIVAD and RTM sulfur budgets were quite different.
Finally, both SIMPMOD and STATMOD scored few if any points for all
seasons but spring for SIMPMOD and both spring and autumn for
STATMOD. The temporal pattern of the predictions of sulfur wet deposition,
which deviated more from that of the observations than the other models,
explained the extensive, significant underpredictions for some seasons and the
extensive, significant pverpredictions for some other seasons. This was
indicative of a problem in the seasonal variation of their sulfur wet removal.
There were several aspects of the predictions that were common to each of the
statistical models (namely, SERTAD, SIMPMOD, STATMOD and TV A), but were
dissimilar to the predictions of the deterministic models. First, the orientation of the
major axis of the spatial patterns of sulfur wet deposition for the statistical models
tended to be further clockwise from the major axis of the observed spatial patterns
than the deterministic models. These differences, greatest for STATMOD, were as
much as +20 degrees for every season but spring. Since the orientation of the major
axis likely reflects the mean transport vectors during precipitation events, the approach
of the statistical models in simulating transport (i.e., via long-term wind roses)
apparently yielded different results than the approaches of the other models. These
long-term wind roses could represent the frequency distribution of wind velocities
during the entire period, but could differ for those periods consisting only of
precipitation events.
Another difference between the predictions of the two types of models was
apparent in the comparison of the seasonal air concentrations of sulfate at the four
APN sites. The predictions of the statistical models generally were higher than those
predicted by the deterministic models and also higher than those observed. The
reason for this was not clear. No systematic differences were apparent for the
predictions of seasonal air concentrations of sulfur dioxide.
Finally, the statistical models, which rely on mean durations and frequencies of
wet and dry periods, computed seasonal and annual values independently. Thus, the
aggregation of the seasonal predictions were inconsistent with the annual predictions.
The prevalent opinion is that the statistical models are somewhat constrained to
annual applications because of the limitations of the statistics to long periods of time.
However, as this model evaluation indicates, some statistical models predict rather
well for seasonal periods.
The scope of a comprehensive model evaluation should include more than an
assessment of the performance of a model. Ideally, one needs to ascertain whether
168
-------
those models replicating the spatial patterns did so for the "right reasons". That is, was
the pattern replication a consequence of the correct simulation of the effects of all
significant processes or can it be attributed to factors which didnot simulate these
effects, yet produce patterns which fortuitously replicate the observed patterns?
Considering these models as regulatory tools, the issue here was model reliability: If a
model is capable of replicating the observed patterns for one year, can it be expected
to do so for another period when the meteorological and emissions scenarios differ?
Since appropriate air concentration and dry deposition measurements were not
available, the ISDME could not comprehensively assess model reliability.
8.4 Concluding Remarks
Model development is an evolutionary process involving iterations of model
evaluations and subsequent model refinements. That is, the results of a model
evaluation can reveal both the strengths and weaknesses of a specific version of a
model and can assist the modeler in improving the model. The ISDME study
represents one step of this continuous model development process. By assessing the
performance of the eleven sulfur deposition models, the results of this study enable the
modelers to advance to the next steps of model refinement and re-evaluation. The
results also provide a benchmark with which the performance of the refined models
can be compared to quantify the effects of the model improvements.
These evaluation results are based on a single application of a specific version of
each of the eleven ISDME models. The performances can be affected by model
application errors and coding errors that were previously undetected. Therefore, on
the basis of a single model evaluation study, one should not dismiss any one of these
models as being unworthy of further consideration. However, the sources of the errors
must be identified and eliminated and the credibility of the model must be
demonstrated before the model results can be used with confidence.
169
-------
9. RECOMMENDATIONS FOR
FUTURE RESEARCH
Model Input Data
1) Improve the accuracy of the temporal and spatial characteristics of the
precipitation data for scales commensurate with those of the models.
2) Estimate the uncertainties and errors in the emission inventories and
meteorological input data.
Model Refinements and Applications
1) Explore the reasons why the models performed as they did for 1980.
2) Use the 1980ISDME data set to improve the model performance.
3) For 1980, develop and intercompare source-receptor relationships.
4) Assess the effects of the uncertainties and errors in the model input data on the
predictions, the model performance and the source-receptor relationships.
Model Evaluation Data
1) Enhance the rigor of future evaluations by simultaneously measuring or
empirically estimating all predicted variables (e.g., air concentrations, dry
depositions and wet depositions of each modelled species) at the same sites in
a optimally configured network.
2) Adopt the Unified Deposition Data Base Committee methods for screening
precipitation chemistry measurements and computing wet deposition amounts.
Model Evaluations
1) Focus on the replication of spatial patterns in evaluating models. It is necessary
to compare predictions and observations on commensurate spatial scales.
171
-------
2) Quantify uncertainties of the patterns of observations arising from interpolation
and measurement errors and account for them in the evaluation procedure.
3) Evaluate long-term models on seasonal rather than only annual scales to
increase the rigor of the evaluations and the detail of the results.
4) Evaluate the latest versions of the ISDME models using the 1988 and 1989
monitoring data to be collected during the Regional Model Field Study.
5) Continue to develop methods of quantifying differences in the spatial patterns
of observations and predictions, namely, rotation and translation.
Interpretation of Observed Data
1) Accompany spatial patterns of observations with spatial patterns of the relative
uncertainties; different sets of data for the same period can produce markedly
different patterns.
172
-------
REFERENCES
Barne ,, M.G. (1980) The use of kriging in estimating the spatial distribution of
radionuclides and other spatial phenomena,TRANSTAT (Statistics for
Environmental Studies) #13, Battelle Memorial Institute, Richland, WA, 22 pp.
Barrie, L.A. and Hales, J.M. (1984) The spatial distribution of precipitation acidity
and major ion wet deposition in North America during 1980, Tellus, 36B: 333-355.
Barrie. L.A.; Wiebe, A.; Anluf, K. (1984) Results of the Canadian Air and
PJ ecipitation Monitoring Network (APN), Technical Report, AQRT-84-005-T,
Atmospheric Environment Service, Downsview, Ontario, Canada.
Bhumralkar, C.M.; Mancuso, R.L.; Wolf, D.E.; Nitz, K.C.; Johnson, W.B. (1981)
ENAMAP-1 long-term SO2 and sulfate pollution model: further applications to
eastern North America, Final Report, EPA-600/7-81-102, U.S. Environmental
Protection Agency, Research Triangle Park, NC, 128 pp.
Bilonick, R. (1983) Risk qualified maps of hydrogen ion concentration for the New
York State area for 1966-1978, Atmos. Environ., 17: 2513-2524.
Briggs, G.A. (1969) Plume rise, USAFC Critical Review Series, TID-26075, NTIS,
Springfield, VA.
Briggs, G.A. (1971) Some recent analyses of plume rise observa-tions, Proceedings of
the Second International Clean Air Congress, H.M. Englund and W.T. Berry,
Editors, Academic Press, New York, NY, pp. 1029-1032.
Briggs, G.A. (1972) Discussion of chimney plumes in neutral and stable conditions,
Atmos. Environ., 6: 507-510.
Clark, I. (1979) Practical Geostatistics, Applied Science Publishers, London, England,
129 pp.
Clark, T.L.; Coventry, D.H.; Eder, B.K.; Benkovitz, CM.; Evans, V.A.; Voider, E.G.;
and Olson, M.P. (1985) International Sulfur Deposition Model Evaluation - 1980
standard model input data set, Paper 85-5.3, 78th Annual Meeting of the Air
Pollution Control Association, Detroit, MI, June 16-21.
Clark, T.L.; Dennis R.L.,; Palka, D.; Olson, M.P.; and Voldner, B.C. (1986) Effects of
gridded precipitation bias on wet deposition predictions. Paper 86-29.5, 79th
Annual Meeting of the Air Pollution Control Association, Minneapolis, MN,
June 22-27.
173
-------
Concord Scientific Corporation (1985) Development of SERTAD: a modeling system
for statistical estimates of regional transport and acidic deposition, Part II, Final
Report for the Canadian Electrical Association.
Conover, W.J. (1971) Practical Nonparametric Statistics, John Wiley and Sons, New
York, NY, 462 pp.
Davies, D. (1967) Three-layer numerical forecasts of precipitation amounts. Canadian
Meteorological Memoirs, No. 25, Canadian Department of Transport.
Delhomme, J.P. (1978) Kriging in the hydrosciences, Adv. Water Res., 1: 251-266.
Dennis, R.L. and Seilkop, S.K. (1986) The use of spatial patterns and their uncertainty
estimates in the model evaluation process. Proceedings of the Fifth Joint
Conference on Applications of Air Pollution Meteorology, November 18-21,
Chapel Hill, NC, 41-44.
De Pina, R.G.; Pena, J.A.; and Bowersox, V.C. (1980) Precipitation Collectors
Intercomparison Study, Department of Meteorology, Pennsylvania State
University, State College, PA.
Eliassen, A. (1978) The OECD study of long range transport of air pollutants: long
range transport modeling. Atmos. Environ., 12: 479-487.
Eynon, B. and Switzer, P. (1983) The variability of rainfall activity, Can. J. Stat.,11: 11-
24.
Finkelstein, P.L. (1983) The spatial analysis of acid precipitation data, /. of Climate
andAppl Meteor., 23: 52-62.
Fox, D.G. (1981) Judging air quality model performance - review of the Woods Hole
Workshop. Bull. Amer. Meteor. Soc., 62: 599-602.
Holzworth, G.C. (1967) Mixing depths, wind speeds and air pollution potential for
selected locations in the United States,/. ofAppl. Meteor., 6: 1039-1044.
Holzworth, G.C. (1972) Mixing heights, wind speeds, and potential for urban air
pollution throughout the contiguous United States, Final Report, AP-101, U.S.
Environmental Protection Agency, Research Triangle Park, NC, 118 pp.
Huijbregts, CJ. (1970) Le variogramme des residus, Int. Rep., CMM/N-200, Ecole
des Mines, Fountainebleau, France, 22 pp.
Journel, A.G. and Huijbregts, C.J. (1978) Mining Geostatistics, Academic Press, New
York, NY, 600 pp.
Klug, W. and Lupkes, C. (1985) A comparison between long term interregional air
pollution models, NR. 104-025-65, Technische Hochschule, Darmstadt, Federal
Republic of Germany.
Lawson, C.L. (1972) Generation of a triangular grid with application to contour
plotting, Technical Memorandum No. 299, California Institute of Technology,
Pasadena, CA, 24 pp.
174
-------
Lehmhaus, J.; Saltbones, J.; and Eliassen, A. (1986a) A modified sulphur budget for
Europe for 1980. EMEP/MSC-W Report 1/86, Norwegian Meteorological
Institute, 19 pp.
Lehmhaus, J.; Saltbones, J.; and Eliassen, A. (1986b) Calculated and observed data
for 1980 compared at EMEP measurement stations, EMEP/MSC-W Report
1/86 (Annex), Norwegian Meteoroligical Institute, 103 pp.
Martin, D.O. and Tikvart, J.A. (1968) A general atmospheric diffusion model for
estimating the effects of air equality of one or more sources, Paper No. 68-148,
61st Annual Meeting of the Air Pollution Control Association, St. Paul, MN.
McNaughton, D.J. (1980) Initial comparison of SURE/MAP3S sulfur oxide
observations with long-term regional model predictions, Atmos. Environ., 14: 55-
63.
McNaughton, D.J. (1984) Use of currently available precipitation data for regional air
quality modeling, J. Air Poll. Control Assoc., 31: 250-252.
MOI (1982a) U.S./Canadian Memorandum of Intent on Transboundary Air Pollution,
Atmospheric Science and Analysis Work Group 2 Phase III Final Report.
MOI (1982b) U.S./Canadian Memorandum of Intent on Transboundary Air Pollution,
Emissions, Costs and Engineering Assessment Work Group 3-B Phase III Final
Report.
Munger, J.W. and Eisenreich, SJ. (1983) Continental-scale variations in precipitation
chemistry,Environ. SclTechnol, 17: 32A-42A.
National Data Use and Access Laboratories (1971) Technical Document No. GT-1, 20
pp.
Norwegian Meteorological Institute (1984) Report of the Meteorological Synthesizing
Centre, West (MSC-W) for 1984", Final Report, EMEP/MSC-W Report 1/84,
Oslo, Norway.
Pasquill, F. and Smith, F.B. (1983) Atmospheric Diffusion: Study of the Dispersion of
Windborne Material from Industrial and Other Sources, Third Edition, Ellis
Horwood Limited, Chichester, England, 437 pp.
Perhac, R.M. (1978) Sulfate Regional Experiment in northeastern United States: the
'SURE' program, Atmos. Environ., 12: 641-647.
Portelli, R. (1977) Mixing heights, windspeeds, and ventilation coefficients for Canada,
Climatological Study No. 31, Atmospheric Environment Service, Downsview,
Ontario.
Ruff, R.E.; Nitz, K.C.; Ludwig, F.L.; Bhumralkar, CM.; Shannon, J.D.; Sheih, CM.;
Lee, I.Y.; Kumar, R.; McNaughton, D.J. (1984) Assessment and evaluation of
three regional air quality models, Paper No. 84-46.1, 77th Annual Meeting of the
Air Pol-lution Control Association, San Francisco, CA.
175
-------
Ruff, R.E.; Nitz, K.C.; Ludwig, F.L.; Bhumralkar, CM.; Shannon, J.D.; Sheih, C.M.;
Lee, I.Y.; Kumar, R.; McNaughton, D.J. (1985) Evaluation of three regional air
quality models, Attnos. Environ., 19: 1103-1115.
Rutherford, I.D. (1977) An operational three dimensional multi- variate statistical
objective analysis scheme, Issue No. 1, Notes Scientifiques et Techniques, RPN,
Atmospheric Environment Service, Dorval, Quebec.
Sabourin, R. (1975) Application of two methods for the interpolation of the
underlying variogram, Proceedings of NATO ASI Geostat. 75, Rome, Italy,
Reidel Publishing Corp, Dordrecht, Netherlands.
Schiermeier, F.A. and Misra, P.K. (1983) Evaluation of eight linear regional-scale
sulfur models by the Regional Modeling Subgroup of the United
States/Canadian Memorandum of Intent Work Group 2. Tranactions of the Air
Pollution Control Asso-ciation Specialty Conference on the Meteorology of Acid
Deposition, pp. 330-345.
Shannon, J.D.; Kleinman, L.K.; Benkovitz, C.M. (1982) Intercomparison of MAP3S
models of long-range transport and deposition, Proceedings of the Third Joint
Conference on Applications of Air Pollution Meteorology, January 12-15, San
Antonio, TX, 29-32.
Shapiro, S.S. and Wilk, M.B. (1965) An analysis of variance tests for nonlinearity
(complete samples), Biometrika, 52: 591-611.
Sirois, A. and Voldner, B.C. Estimation of dry deposition velocities in North America,
Part II: the meteorological data banks (in press).
Snedecor, G.W. and Cochran, W.G. (1967) Statistical Methods, Iowa State University
Press, Ames, IA, 593 pp.
Stensland, G.J. and Bowersox, V.C. (1984) Quality assurance in acid precipitation
through the use of ion balance calculations, Transactions of APCA International
Specialty Conference on Assurance in Air Pollution Measurements, pp. 425-443.
Stewart, D.A.; Morris, R.E.; Liu, M.K. (1983a) Evaluation of long-term regional
transport models of interest to the National Park Service, Final Report SYSAPP
83/215, Systems Application Incorporated, San Rafael, CA.
Stewart, D.A.; Morris, R.E.; Liu, M.K. (1983b) Evaluation of an episodic regional
transport model for a multi-day sulfate episode, Attnos. Environ., 17: 1225-1252.
Thorp, J.M., Scott, B.C. (1982) Preliminary calculations of average storm duration and
seasonal precipitation rates for the northeast sector of the United States, Atmos.
Environ., 16: 1763-1774.
Topol, L.E. and Schwall, R. (1984) Performance of precipitation chemistry
measurement systems, Transactions of the Air Pollution Control Association
International Specialty Conference on Quality Assurance in Air Pollution
Measurements, T.R. Johnson and S.J. Penkala, Editors, pp. 444-454.
176
-------
Unified Deposition Data Base Committee (1985) A unified wet deposition data base
for eastern North America: data screening, calculation procedures, and results
for sulphates and nitrates, (1980), Final Report, Prepared for the Canadian
Federal-Provincial Research and Monitoring Co-Ordinatmg Committee, 79 pp.
U.S. Environmental Protection Agency (1987) Rocky Mountain acid deposition model
assessment: review of existing mesoscale models for use in complex terrain, Final
Report, EPA/600/3-87/013, Research Triangle Park, NC.
Vet, R.J.; Chan, W.H.; and Lusis, M.A. (1981) An intercomparison study of three
precipitation sampling networks in Ontario - APIOS, CANSAP and GLPN,
Ontario Ministry of the Environment, Report ARB-002-81-ARSP.
Vet, R.J. and Onloch, S.G. (1981) Detailed analyses of the Pennsylvania State
University intercomparison of precipitation collectors data, Concord Scientific
Corporation, Final Report #52-80, Toronto, Ontario.
Vet, R.J. and Reid, N.W. (1982) A comprehensive evaluation and integration of
selected wet deposition data from Canada and the U.S.A., Report No. 110-84,
Concord Scientific Corporation, Downsview, Ontario.
Voldner, E.G.; Olson, M.P.; Oikawa, K.; and Loiselle, M. (1981) Comparison between
measured and computed concentrations of sulphur compounds in eastern North
America,./. Geopliy. Res., 86, No. C6, 5339-5346.
Voldner, E.G. and Shannon, J.D. (1983) Evaluation of predicted wet deposition fields
of sulfur in eastern Canada, Transactions of the Air Pollution Control
Association Specialty Conference on the Meteorology of Acid Deposition,
October 16-19, Hartford, CT, pp. 387-399.
Voldner, E.G.; Sirois, A.; Clark, T.L. (1984) Data screening and calculation
procedures for the North American precipitation chemistry data to be used in the
International Sulfur Deposition Model Evaluation, Proceedings of the Air
Pollution Control Association/American Society for Quality Control Specialty
Conference on Quality Assurance in Air Pollution Measurements, October 14-18,
Boulder, CO.
Watson, C.R. and Olsen, A.R. (1984) Acid Deposition System (ADS) for statistical
reporting: system design and user's code manual, Final Report, EPA-600/8-84-
023, U.S. Environmental Protection Agency, Research Triangle Park, NC.
Weber, A.H. and Kurzeia, R.J. (1985) Proceedings of the DOE/AMS Air Pollution
Model Evaluation Work Shop, volume 2, statistical analysis of results, Savannah
River Laboratory, E.I. du Pont de Nemours and Co., DP-1701-2, Aiken, SC.
177
-------
APPENDIX A
SUMMARY OF PREVIOUS EVALUATIONS
OF SULFUR DEPOSITION MODELS
A-l
-------
o
HH
H
K
O
w
Q
0>1
D
^^
D
C/5
i-)
£
O
o
w
^1^
UH
O
C/D
O
H
D
^
>
W
i,
H»
O
HH
^^
W
OH
ft
O
>
»*"*
^^
S
S
D
T~H
^J *
LlJ >-J
c
o
4-i o
CO *^l
3 ki
H 01
a a.
u
c E
0 0
H 1-1
4-1 00
CO 01
CO 01
> £
u u
U-l TJ
0 C
CO
OI CO
XI 01
Z en
c
o
rf *
U CO
31
0
H O
a H
00
O 0>
U 4J
v I g
o -*
0) CD
U >
cu to
«H
01
*O
g
CO 0)
v^ O
u c
O 0)
U l-J
«0 <1)
bo*M
-* a>
4J Q£
n
> c
C RJ
1
to
Q)
U
H
CO
C
CO
a cu
c a
to o
C 3
0)
4-) i-H
CO (Q
3 u
C
O OI
in o
CO
s
u
u
a n
B u
CO 0
ki a.
a)
bo k<
<0 n
kl CO
01
> 01
CO u CO
CO E
f-l lu O
3 3 u
B m co
B ki
^- c e
2-" 8
u CM e
n o o
D V3 o
to «
tO ki 4J
i e o.
10 0> -r^
3 0 O
C C 0)
6 O kf
co u a.
£t
s
o
u U
3 ° rx
3 B O
< CO -1
B
01
4_l CO
eo 0
Id 01 ki
ad oi
3 B B
CO .H <
00 CO .C
pn Q) w
1 4-1 kt
<""» -»-i O
CS CO Z
'o
"
A *U
c o
0 CO
CO B B
1H 0 CO
X m *J tt
« a. o o
S S 01 u
O O CO CO
* ej -*
t, "" -c
o X AJ 4-1
CO CO CO
U-l kl
* 3 B
£:
u u
c -a B
oca
E co o
co
-j
Z
a.
B
O
.C
bO"^
30
CO 00
Z CT*
£ %«x
CM
B
CO 0)
OI JJ
*! CO
CO OI
en u
n ki
Oj o
< B
I en
B
2
~
j3
Q DO
a) o
H i-l
1 CO
kl (X
O --.
c o
4J CO
a) AJ
* §
u 0
3 B
u <9
3 6
a o
r-< 0 1H
I Z M
'a
~
o
o.
H^ C
-T4 CO
^ CO
t) M
f-H J3 rH
4J fi 4J
C Q B
O OI O
CO
0)
a
ki
0)
cd
0
*tf °
C CO CO
O kl
g Ol 4-1
. 4J C
>, CO 01
-^ u-i o
CO 3 o
^ CO o
CO
4t
r-4
CO
u
01
u
eu ^x
B ~"«
"O CO
r-l CTS
O i
t*>
C
CO
T3
co cn
B
CO =>
O
B
E M
kl 01
CU U
4-1 CO
m co
CO 01
^£
u P
3 0
O C CO
CO 01
en C -H
-^ CO CO
2
*"
4-1 f-H
C
^^ o
co e
CO O CO
co co IM -a
o o
i-l CU ^.H
ki e J ki
CO CO E 0)
kl CO O 0.
t
*d co
g
O -H E
01 4J O
4J CO ^ O O
« 1 O 0)
£ u
4-1 o> a.
B u
O CO B
B u-i i-t
^
4J
E
01
H
u
U4
U-l
w
o
u
E
O
H
kl
CO
01
o
0
kl
CO
o
B
CO
U-l
O
B
O
AJ
^H
E
i-l
U-f
U
4(
CO
CO
H
JO
U-l
o
E
O
' i-<
4J
Q
> E
01 O
0 -1 C
u O
T3 10 -r4
CO kl O
T3 0) iH
E M T3
CO -O OJ
CO CX
s I §
co -H e
X -H
E CO X
O S rt
CO B *O
H CO £
CU B
a o E
H O
T3 4-1 *H
M «0 U
« Id L. ^
C CO C 0) T>
CQ to ct) co eu
Q) ^H 4J ^ M
Q J* CO O (X
2
3. J3 O o O.
OJ
O
OJ
/~\
O
1
a.
o
0
ki
kl kl
O 01
kl
kl 01
a> 0
B
o g
oi B
ki C
30
u
cr ki
CO 0)
a.
E
CO -H
01 01
!1
§
M
CO
£*
A-2
-------
Q
D
P
O
U
r-H
G
o
4-1
31
< 1 -H
a M
> 01
U Cu
c e
0 0
4J DO
* S
.H
CO 01
> J=
"o "c
0)
u
01 CO
XI 0)
§4J
-1
z en
e
o
H
4J
CO «
3 CO
CO 0
> O
U H
to
u -o
0 CU
U 4J
H 3
T3 f-H
01 CO
Cu [d
O
O
£
to
^ 01
M O
O C
u OI
CO IJ
00 01
t-4 *4«*
CO Dd
01
> -o
c c
1-1 eg
f-. CO
^ t ^ **"
C -i to 3 -i
eo 3 i->
u c Li
4J 4) tfl 4)
§X» -> t-4 jQ
0 ^ 0
3 o c a c u
< O «) -tf (Q O
jC
u
(D O
V Z
iH C
CO I-*
V
UJ J-> a)
oi w o
=3 (0 ^
01 0) U
0)
* C B
m -H M
CO CO U
>,e* X 0
W -* 01 -H
CO 4J
3 >, 10 CO 00
e J 3 -i r^
CO 3 rJ o ej\
-> ^ o. o. -
j:
^j
en o
01 Z
* e
CO Li
01
U3 4J CO
as eo ej
^D eo -^
en oi u
-f c a
in ^-« ^
'eo
a.
s:
^ Xl^
D &
O
c 'E xT
CO JC IM
u e; o
u -^ co
CO U 01 Li
oi co e
CM T3
-< 0 - C
^4 rH t? CO
CO
CM
O
en
01 M
no-rt to
CO CO C
M O
OI 0) «-(
CO CO CO
t-H 3 C
x: eo oi
4-1 CJ
c -o c
o e o
g CO 0
_^
CO
7
CU Cu
< 5;
* ?
e/3 H *C CU M
td en Z M o
<: < u z s
O r-i
00
§ -
C
CO 4J
U C
§01
4J
in
tn cu
Cu en
z <
O to
m oi
^4 -a 4J
1 C *
-» eo eo
XI
^^
CO
CO
eo
01
e
CO
CO
XI
4-1 CA
* §
3 i
4-1 S
r-4
3 e
to o
rH
>^ 4«l
x: "tn
u o
c a.
O 01
e -o
^^
x>
o
0
o
u l-l
H Z O
z Q a
o u |
3- o- ?3
CO
er\
3
0) CO
4) ^j
" C fci
eo Li 01
01 S
W 4J <
fv; (o
=1 CO Xi
en cu 4J
* C 0
m f-i z
eo
CL
T3
01
N
CO
C
eo
e->j
0
en
01 14
bo -4 to
co eo C
K 0
01 41 i-t
CO CO CO
IM U
^? 3 C
x; to oi
4J ej
c -o c
o c o
e eo o
x CM
o <
en CM H
CO
4J
01
c
O '^
C CN
C CD
x; ^^
en x^
xO
oo
o>
T>
C
eo
0
CO
u)
01
.^
LO
DM
xO O CO
° o >> o
C '>x, CJ 14-1 CJO
CO IM Q. c O O
01 o oi n i
H * 3 M CO
Li O. cr C O OS
o ij x; LI j:
* 4J M-i 4J T3 T3
. X1»H ^ C C
^ o s x ;» co co
5
m
u e
* 3
0
Li B
3 CO
U-l
-I C
3 O
CO 1H
4J
r-t T*
CO CD
3 O
c a
C 01
.^
£
H
tn
C
eo
Li C
01 0 ^
c d f>
a B to
^H co ejs
O x; <
rx.
09
0
o.
A-3
-------
Q
W
P
Z
f^
O
u
i (
<
d
o
4J
g -a
3 0
-H Tl
Cd kl
> 01
u a.
d c
o o
1H 1H
4J DO
5 cS
_t
CO 01
> £
tu u
«u -o
o d
k, "
O> 0)
XI 0)
g 4J
3 -4
Z tn
d
o
H
4J
S«
CO
1 t
CO 0
> o
U H
00
ki -0
0 0)
4J 4J
U CO
H D
T) 1
U CO
kl >
a. id
f-«
o>
a
£
y».
CO
wx 0)
kl 0
O d
4-1 Ol
Q U
DO OJ
H U-l
£2
u
> T)
d C
1-1 CO
^i
r^
9*
1 kl
kl £ XI
01 bo B
XI 3 OJ
O O 4J OO
4-1 ki a.r»
u s: oj u>
O 4-1 tO -H
J3 d
tO 4-1 kl
01 ki 01
u O u
H SZ CO
tn co
B OJ
U kl CO
K 0) n~i B
2 u co a, 1-1
C/3 CO U -rf
CO -H JS tO
« 01 ki 0) .
»n oj r-*. 4J to
IBB 1 -I .
O* -rl < -tf CO O
s-\ ^-^
CO XI
AJ 4-1
CO CO
Ol OJ
£ £,
DO DO
kl _ »H
0 J= J= .
<4-l U-J (0
O .C CO .C -
« 1 l-i AJ - w O
CO CO -H O -H
3 - U i 1* M_,
C W 1-4 o. IM o
C * co ce mot-'
nj 3- E CX 13 M -r-( M
^,^5-PC C-HXcOO
^o rHtocooa.-H
I-H i-^ o, m
X - jp tO o <^
U D D x-s ^-|
C ^x vH -H
O « * -H -H *-*
B 3. js ^ ^ ^
CO
c?
CO
0) U
DO-H « PC
n) fl) C 3 O
MO U-< -H
0) 4J 4J 3 ^H
CO fl3 CO CO CO
(M l-l O
X-3 " >, D. CO
-< 3 C -( CD u
£ to a) j: t3 e
*j o AJ 3
C -O C C 4J o
§P o o a> B
mo e * co
U 2
T
a, M
«C M
IH±
Z i-l P
u £ a:
rH
CO
4J
01
"S
ss
ty CT>
AJ r-
CO^ '
00
(S|
OO
ON
c-T T
>^CM 00 00
_i | r«- r«-
3 ^o cr> ON
j:
4J
u a)
to o CO)
D as u 4J
*J d) f-4
CD CO -H ON U O.
to a) M CO CO
-2 T3 CO
O C 4-1
CO 0
-? c? ^
^o u a.
00
CO
e u
O (0
CU -»-4 0) T)
4-« u >s e co
4) to tfl r tO C
bo M-I u m o
tO *H 4J OJ »-< -H
U< 3 C! T3 O -H *-»
CD CO Q) C CO CO tO
> O (Q U
« T3 tf 0) 4) 4J
CO ^ bO 4J C
>,
f-r 3 i-i M-I o
i-t CM M C 0) rH C
tB O ^ C > 3 O
o co to to to co o
x-\
CO
II
M :*
i i
H en
ai a
co
-H O
CO 1H
DO
4-1 d o eu
0) CO r* 4J
-rf 0 3
4J ^~* DO kl 4J ^^
kl « CU O -H ?
co 2 3 oi 4j oo
3 00 kl 4J CO <7i
0) O* O OJ d I
4J Z Z I-l ^
to
o
-H to
Cd Cd 4-1 »W
x7
" T)
>, d ki
kl * CO 01
CO 1-4 XI
3 i-l >, O OO
C >-i H 4J r-*
CO Q. 3 O C7N
i-> 4J
01 kl
4J O
rH ^
d ro
y k. o
2 0) -H
3 4J kl
to oo o)
co g
tn 01 <
ki
»
u
to
XI
CJ kl
O 1-4 00
to cd B
o
01 01 i-l
DO 4-1 4J
cd cd cd
kt »4^4 kl
OJ t-J 4J
> 3 B
cd co cu
o
x: -o d
I d o
v£} CO U
Y
Oj »-* Q
-d, h- * M
3 i ^
25 H £
W ce5 5
,
r-t
tO
4J
a» *-N
m
"4-( 00
M-t CTv
«3C
A-4
-------
(Q CO
3 oe
T3 0)
C +J «
X) -i- O>
in +>
C ID
1- C <4-
QJ (T3 i
4-» O) 3
(/I CX. I/I
,
X L. -O ~
3 C flj
ID UJ (D >
CO -r-
CVJ+J
O na O o
C t_ c/1 O)
ra -4-> a.
c <- in
^O 0) O 4J
CM o *«- t-
M rg
C
r- y>
C 4J
u n
_
0) »- t/i
OT rtj C
m o
O
Q. ul
c
4->
O
o .
o -
CO CO
CO
> 3
nj in
c
4->
e
4-1
C
u
c
o
o
ily /annual
c
o
E
1
o
CO
t_
0.
u
OJ
>
4->
J=
CO
c
o
4->
Q.
0
ex
4->
CO
c_
it-
's
in
l/>
4J
1
1
O)
o
o
O
E
0)
1- O
O C
*-> 0)
cnoo
3 O.
OJ
A-5
-------
APPENDIX B
STANDARDIZED MODEL INPUT
DATA (FIGURES AND TABLES)
B- 1
-------
V1' 1 ;
A &r'
,\ i " -
T"' '"' '' "^ ^
'<' '!
; i t : / ; : . j . .
''!"'! r'~
1 . LJ~\^
\ - -
- - - 1
- ^ - - - , -
{''"'}' ".
j
1 '
'.- u
-/:;-':--
... ..
- .. 7- '
i
^i-iiKUi^ii
. .
\\-n!^rnt^: '"ij-r<'
,,f| : v.TiutjVi
t. ir
B-2
-------
a
Figure B.2 Annual 1980 patterns of precipitation (cm) using a) CMC gridded data
and b) data from precipitation chemistry monitoring sites.
B- 3
-------
a
Figure B.3 Winter 1980 patterns of precipitation (cm) using a) CMC gridded data
and b) data from precipitation chemistry monitoring sites.
B- 4
-------
a
Figure B.4 Spring 1980 patterns of precipitation (cm) using a) CMC gridded data
and b) data from precij itation chemistry monitoring sites.
B- 5
-------
a
Figure B.5 Summer 1980 patterns of precipitation (cm) using a) CMC gridded data
and b) data from precipitation chemistry monitoring sites.
B-
-------
a
Figure B.6 Autumn 1980 patterns of precipitation (cm) using a) CMC gridded data
and b) data from precipitation chemistry monitoring sites.
B- 7
-------
UJ
o:
u_
COMPARISON OF PRECIPITATION AMOUNTS
50
45
40 -
35 -
30 -
25 -
20 -
15 -
ANNUAL 1960
'
48
64 80
96
112
126
144
160
176
192
1771 CMC GRID CELL
MIDPOINT AMOUNT (CM)
MONITORING SITE
Figure B.I Histograms of 1980 annual precipitation amounts a) for 127-km
grid cells containing precipitation chemistry monitoring sites and b) at the 72
precipitation chemistry monitoring sites across eastern North America.
B- 8
-------
B-9
-------
O CO CO
""" 'NOIlVJLIdl33Hd
B-10
-------
TABLE B.I THE 1980 ANNUAL SULFUR DIOXIDE EMISSION RATES
(KILOTONNES PER YEAR) AND PRECISION ESTIMATES FOR CANADA #.
Province
Alberta
Nova Scotia
British Columbia
Ontario
Manitoba
Prince Edward Is.
New Brunswick
Quebec
Newfoundland
Saskatchewan
N.W. Territories
Yukon Territory
Rate
543.2
218.0
193.3
1826.5
490.9
5.7
215.0
1157.0
60.7
56.8
2.2
0.5
4769.8
Precision
±29.3
±14.5
±17.3
±9.3
±10.3
±44.1
±14.4
±14.5
±25.7
±42.3
*
*
±6.1
# (MOI, 1982b)
* No estimate available.
B-ll
-------
TABLE B.2 THE 1980 ANNUAL SULFUR DIOXIDE EMISSION RATES
(KILOTONNES PER YEAR) FOR THE UNITED STATES f.
State
Rate
State
Rate
Alabama
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
D.C.
Florida
Georgia
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisianna
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
762.8
762.5
78.7
493.2
109.7
68.0
103.1
13.9
1111.1
807.6
50.4
1392.9
1744.0
434.5
209.7
1065.9
468.2
145.9
296.7
254.4
813.7
248.6
274.6
1172.5
151.7
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
82.2
132.3
100.7
291.7
214.2
787.0
605.0
137.2
2612.4
92.3
50.5
1724.7
13.0
325.1
68.7
1049.6
1204.3
119.3
8.5
357.0
295.2
1041.1
641.2
228.1
Total
25154.4
# Version 2.0 of the National Acid Precipitation Assessment Program (NAPAP)
emission inventory (unpublished).
B-12
-------
TABLE B.3 UNITED STATES AND CANADIAN STACK HEIGHT
FREQUENCY DISTRIBUTION.
Physical Stack
Height Interval (m)
0- 50
51 - 100
101 - 150
151-200
201 - 250
251 - 300
301 - 350
351 - 400
Missing
Heights
Total
Canadian Point Sources
Number SO, Emissions
(ktonnes/yr)
450
208
45
21
1
3
0
2
0
730
543.7
636.3
752.7
1011.5
16.3
249.3
0.0
746.3
0.0
3956.1
United States Point Sources
Number SO2 Emissions
(ktonnes/yr)
100367
5728
374
281
111
46
55
6
69591
176559
3481.2
4972.5
2754.0
4520.7
2294.1
1089.7
1818.3
237.8
2383.4
23551.7
B-13
-------
APPENDIX C
SCREENING AND CALCULATION PROCEDURES
FOR THE MODEL EVALUATION DATA SET
Excerpts from the Final Report of Unified Deposition Data Base Committee(1985)
reprinted with permission of the primary author.
C- 1
-------
C.I Concentration and Deposition Calculation Procedures
All seasonal and annual concentrations are precipitation-weighted average values
defined as
C = i=l
fp,
where
C = the seasonal/annual weighted concentration (u.moles/1);
Cj = the daily/weekly/monthly concentration (jimoles/1) measured during collection
period i;
PJ = the precipitation depth (cm) recorded during collection period i; and
N= number of simultaneous valid measurements of both precipitation and
concentration during the season/year.
Valid sample criteria applicable to the annual and seasonal summaries are:
1) All sampling periods for which it is known that no precipitation occurred are
considered valid sample periods. This applies mainly to weekly and monthly
sampling protocols. For event and daily sampling protocols the absence of a
sample record for a day implies that no precipitation occurred (this
assumption is not always valid for the earlier data).
2) The wet deposition sample must be a wet-only sample. All samples identified
as bulk, partially bulk, or undefined are invalid.
3) Wet deposition samples that have insufficient precipitation to complete a
chemical analysis for a specific ion species are invalid for that specific ion
species. Event and daily samples are most likely to have this occur.
4) The concentration of an ion species accompanied by a comment code
designating the measurement to be "suspect" or "invalid" is declared an invalid
sample. Deletion of the ion species concentration by network personnel for
the same reason has the same effect.
C- 2
-------
5) The actual sampling period for a wet deposition sample must he close to the
network's protocol sampling period.
All seasonal/annual deposition values are calculated by multiplying the
precipitation-weighted average concentrations by the total precipitation amounts for
the season/year (Pt) defined as
D = CPt
where
Pf =
and N is the number of collection periods with precipitation amounts for the
season/year. Valid sample results designated as below the detection limit are included
in the average value as one-half of the detection limit.
The precipitation depth is generally obtained from a colocated standard gauge
measurement. At some locations, however, there may be a systematic overcatch
problem (i.e. the collector depth is greater than the standard gauge depth. This type of
situation is relatively rare but, when it does occur, a malfunctioning standard gauge is
normally the source of the problem. For these cases, the gauge and collector depth
data are carefully examined and the deposition is calculated using both the gauge
depth as well as the maximum of gauge or collector depths (whichever is greater during
each collection period). If the deposition calculated using the maximum of gauge or
collector depths is more than 10% greater than the deposition calculated using the
gauge depth alone and if there is no obvious problem with either the gauge or
collector, then the concentration or deposition is calculated using the greater of the
two values, i.e.,
Cj (Pj or Pjc, whichever is greater)
C = i=l
Pj or Pjc, whichever is greater)
and
D = C 2-
Jj or Pjc, whichever is greater)
C- 3
-------
where
PJ = the standard gauge depth (cm) and
Pjc = the collector depth (cm).
Precipitation chemistry data are often missing from one or more collection
periods. In these cases the annual/seasonal deposition is handled by multiplying the
precipitation-weighted concentration (using the available data) during the year/season
by the measured total precipitation depth (Pt) during the year/season.
The precipitation depth data are collected at the precipitation chemistry
monitoring site. If depth data are missing from the standard gauge for a particular
sampling period, then different procedures have historically been used in Canada and
the United States. In Canada, interpolated gauge amounts from surrounding weather
service stations are generally used, while in the United States collector depths are
used.
C.2 Definition of Annual and Climatological Seasons
Preparation of annual and seasonal data summaries requires specific definitions
for annual and climatological seasons. Annual periods are here selected to coincide as
closely as possible to the calendar year. Seasonal periods are defined to coincide as
closely as possible to the climatological seasons, defined here as: winter - January,
February and March; spring - April, May and June; summer - July, August and
September; autumn - October, November and December.
The fact that some cumulative networks use collection periods that do not
necessarily coincide with the preferred annual and climatological seasonal periods
based on calendar months presents a problem. For example, NADP follows a weekly
collection period starting on Tuesday. Entire collection periods must be allocated to
an annual or seasonal period either on a fractional basis or a fractional precipitation
occurrence basis. The latter requires knowledge of when precipitation occurred within
a single collection period. Since this information is not available for all networks, the
allocation of collection periods is computed here on a fractional time basis.
The procedure used here for the fractional time allocation of samples to an annual
or seasonal period requires that at least fifty percent of the time covered by a
collection period be contained in the annual or seasonal period. This applies to both
the beginning and end of an annual or seasonal period. For each summary the first and
the last date of the period summarized is given in the data tables.
C- 4
-------
C.3 Screening Criteria
Data quality is an important consideration in any wet deposition summary process.
Aspects of data quality include site representativeness, network operation and sample
analysis procedures, screening of individual samples on the basis of available sample
comment codes, and data completeness measures for the period being summarized.
Network procedures and the screening process applied to individual samples are
discussed in Appendix A of the UDDBC Report. Data quality levels discussed below
are based on only those samples that meet the screening criteria described in this
Appendix.
C.3.1 Site Representation
Each valid mean concentration and deposition amount is assigned an overall data
quality level ranging from "1" to "3", where level "1" is the highest level. This overall
data quality level is based on a site representativeness rating and a data completeness
rating. Regionally unrepresentative sites, those sites meeting one or more of the
criteria in Table C.I, are considered to be unrepresentative for monitoring regional
scale acid deposition. These sites receive a rating of "3" unless an examination of the
historical data from the site, or the judgement of those familiar with the site, indicated
that the local interferences were small or insignificant.
TABLE C.I CRITERIA FOR UNREPRESENTATIVE MONITORING SITES.
1. There is a continuous industrial source, a town or suburban area located within 10
km or
2. There is a major point source (greater than 10,000 tons sulfur dioxide or nitric
oxide per year) located within 50 km or the sum of point source emissions within
50 km is greater than 10,000 tons per year or
3. There is a surface pollutant storage facility (e.g. salt pile) located within 100 m or
4. There are transportation sources, furnaces or incinerators located within 100 m or
5. There is cultivation or other agriculture activity within 500 m or
6. Buildings, trees, etc., impinge on the cone defined by a 45 degree angle above the
horizontal plane and centered at the site (30 degrees is considered optimal but 45
degrees is the highest acceptable angle or
7. The local area is known to be dusty due to poor ground cover.
C- 5
-------
On the other hand, sites with a rating of "1" are not exposed to any of the
interferences listed in Table C.I and are considered to be regionally representative.
However, there is a large number of sites which fall into neither the "3" nor "1"
categories, and for which evaluation of site representativeness is difficult. The
comments of individual network operators, analyses of the historical data, or site audits
were used to distinguish further between type "2(a)" and "2(b)" sites. In the former
case, those familiar with the site or with historical data from the site, have judged that
the local interferences are probably small or insignificant and the sites are potentially
representative. On the other hand, "2(b)" sites have more significant problems
associated with them, but those familiar with their surroundings or historical data did
not feel that they should clearly go into the "3" category. They were nevertheless
judged to be potentially unrepresentative and data from them should be regarded with
somewhat less confidence than data from "2(a)" sites.
It should be noted that detailed site evaluations (based on both the nature of the
surroundings and historical data characteristics) are presently underway in a number of
networks. Therefore, the ratings used in the ISDME, although based on the best
information available at the time, should be considered as tentative and subject to
possible revision.
C.3.2 Data Completeness
Each annual and seasonal average value is assigned one of three data
completeness ratings. A value not meeting the lowest rating requirements is excluded
from the data tables. Five data completeness measures are used to determine the data
completeness rating. A sixth measure, percent sea salt correction, is used for sulfate
values for sites within 100 km or the coast. All six criteria must be met before an
individual sulfate or nitrate value is assigned a given data completeness rating. How
these measures are used to determine the data completeness rating is given in Tables
C.2 and C.3. The abbreviations of the data completeness measures in the these tables
are defined as follows:
% PCL - Percent Precipitation Coverage Length is defined as the percent of the
summary period for which information on whether or not precipitation is available:
% PCL = 100 (SPL - NDDPP)/SPL
where
SPL = the number of days in the summary period and
NDPP = the number of days when it is not known if precipitation occurred.
C- 6
-------
If precipitation is known to have occurred during a particular sampling period but
no measurement of the amount is available then no knowledge of precipitation is
assumed. Total precipitation depth Pt is the amount of precipitation occurring during
the period of precipitation coverage. This data completeness measure does not
include any consideration of the availability of a valid precipitation chemistry sample.
% VSL - Percent Valid Sample Length is the percent of the days in the
summary period that are associated with valid sample periods:
% VSL = 100 (NDNP + NDVCMP)/SPL
where
NDNP = the number of days in the summary period associated with sample periods
during which no precipitation occurs.
NDVCMP = the number of days in the summary period with valid sample
component measurement on measured precipitation sample.
% COL EFF - Percent Collection Efficiency is the ratio (converted to a percent)
of the total sample volume (converted to a gauge depth) to the total precipitation
depth:
% COL EFF = 100 (EPDVC/ERGVC)
where
EPDVC = the sum of depths calculated from the sample volume for qualifying
samples and
ERGVC = the sum of the standard gauge depths for qualifying samples.
Qualifying samples are those that have both (a) a collocated standard gauge and
sample volume measurement available and (b) a valid sample component
measurement. The collection efficiency requirement for Canadian winter average
values has been relaxed somewhat compared to other seasons due to the generally
poorer collector performance for snow sampling. If the 80% requirement for other
seasons (level 1) is applied to the winter months when a very large percentage of the
precipitation in Canada is in the form of snow, then the data loss is quite large. It is
felt that a lower percentage could be accepted for winter because the problems are
primarily due to undercatch of snow. The sample collected is expected, therefore, to
be representative. The Canadian annual collection efficiency requirement is also
relaxed somewhat as a consequence.
C- 7
-------
% SEASALT - Percent Sea Salt Correction is the percent of the average
sulfate concentration that is estimated to be due to sea salt:
% SEASALT = 100 (XBAR - ABARC)/XBAR
where
XBAR = the arithmetic mean of sulfate concentrations from uncorrected valid
sulfate samples and
XBARC = the arithmetic mean of sulfate concentrations from sea salt corrected
valid sulfate samples.
The sea salt correction uses sodium or magnesium as tracers of sea salt, as
described in Appendix D of the UDDBC Report.
TABLE C.2 DATA COMPLETENESS RATING CRITERIA FOR SEASONAL
SULFATE VALUES.
Data Completeness
Measure
Level 1
Level 2
Level3
%PCL
%VSL
% COL EFF
Winter
Spring
Summer
Fall
% SEASALT
(sulfate)
^95%
^75%
<25%
^90%
.>70%(50%)
>.5Q%
^90%
2i60%(30%)'
<75%
* The values within the parentheses apply to Canadian data only.
C- 8
-------
TABLE C.3 DATA COMPLETENESS RATING CRITERIA FOR ANNUAL
SULFATE VALUES.
Data Completeness
Measure
Level 1
Level 2
Level 3
%PCL
Annual
Each Quarter^
%VSL
Annual
Each Quarter/1
% COL EFF
Annual
Winter
_>90%
>80%
>6Q%
2i75%(65%)
_>90%
>50%
^60%
^50%
>:40%(35%)*
>40%(20%)*
Spring
Summer
Autumn
% SEASALT
(sulfate)
^75%
<25%
>_50% _>40%
_<50% j<75%
^ An annual summary must not only meet the annual data completeness criteria, but
also the quarterly criteria for each quarter. The addition of quarterly criteria for %
TP, VSL and VSMP to the annual criteria is to insure that adequate data from each
quarter is present in the annual summary. The quarterly criteria for % TP, VSL and
VSMP in Table C.3 are relaxed from the seasonal criteria in Table C.2 since the
emphasis is on insuring adequate data for an annual summary, and not reporting a
quarterly summary.
* The values within the parentheses apply to Canadian data only.
C.4 Assessment of Uncertainties in Deposition Amounts
Errors in deposition calculated from precipitation chemistry measurements can
arise from several sources such as actual sampler design, errors due to laboratory
procedures, field related errors caused by sampling periods, poor catch, contamination
and missing samples. These errors are difficult to quantify and may vary from network
C- 9
-------
to network. The magnitude of these errors has been investigated here by
1) Reviewing studies aimed at quantifying network bias as well as accuracy and
precision of measurements;
2) Statistically analyzing the ramifications of missing sampling periods; and
3) Comparing observed deposition amounts for 1980-1983 at colocated or closely
located samplers.
Vet et al. (1981) intercompared the performance of CANSAP and APIOS
collectors at five sites in Ontario. Samples were collected on a monthly basis from
spring 1979 to spring 1980. They found that the CANSAP data averaged 20 percent
higher than APIOS sulphate concentrations. De Pina et al. (1980) intercompared
samplers used in the CANSAP, MAP3S, NADP and EPRI networks during the period
July 4, 1978 to April 30, 1979. Since NADP and EPRI used the same sampler type it is
referred to here as EPRI.
Detailed analysis of these data by Vet and Onloch 1981 (see also Vet and Reid,
1982) revealed that the mean values for deposition ratios of sulfate calculated by event
was 1.24 +. 0.47 for CANSAP to MAP3S and 1.16 +. 0.38 for CANSAP to EPRI. Over
the entire period CANSAP overestimated sulfur deposition compared to the other
samplers by 20 and 15 percent, respectively. The differences may be somewhat lower in
1980, since the CANSAP operational protocol changed. Bias and precision in the
EPRI daily network for 1979-1980 were estimated by Topal et al. (1984). They
detected a consistent anion-cation inbalance, larger than observed in the NADP
network (Stensland and Bowersox, 1984), but were unable to trace its origin. The
relative annual precision in concentration and precipitation based on pooled data from
colocated samplers at nine sites were 6% for sulfate concentration and 5% for
precipitation. To discount outliers, the annual precision was defined as the median
value of the colocated sample differences (ACD) and the relative precision was ACD
per median value of concentration or precipitation.
A recent study of the APIOS networks (Lusis, personal communication) found the
reproducibility of sulfur concentration in precipitation with wet-only Sangamo sampler
(used in the cumulative network) over a 28 day period was at least 88%. They
experienced less than 10% evaporative losses of precipitation and estimated accuracy
of deposition to better than 10%. The annual wet deposition measured by the 28d
cumulative Sangamo sampler was about 10% higher than thatdetermined from the
daily Aerochem Metric samplers. Thus, network differences can easily account for
more than 10% differences in estimated deposition amounts.
Uncertainties due to missing data in 1980 were estimated for the NADP, APN and
CANSAP networks. Weekly concentration amounts were removed randomly
throughout winter and summer seasons. In the Canadian data, for which precipitation
is always available, the seasonal deposition was recalculated from the remaining
concentration data by scaling with seasonal precipitation. For the NADP data, where
C-10
-------
precipitation coverage may deviate from 100%, the new seasonal deposition was
computed in two ways, i.e. as for the APN and from periods, when both concentration
and precipitation were available. The latter approach creates a consistent
underestimation. When 1, 2, 3 and 4 weeks of data were missing, the coefficient of
variation was about +_ 4, +. 6, _+. 7, and +_ 9% in for the APN data, while a coefficient
of variation and an underestimation of 8, 16, 24 and 30% were observed in the United
States data. Absolute maximum errors at the individual sites for any reason were 10,
20, 30, 40%, respectively, for the APN network.
Missing periods - The seasonal/annual precipitation weighted mean concentrations
are determined from simultaneous valid measurements (n) of precipitation and
concentrations. This concentration is multiplied by the total precipitation depth
measured for the season/year to yield the seasonal/annual deposition. For the
Canadian networks and frequently for the NADP network, estimates of precipitation
are available for each collection period (N) within the season/year, while in the
MAP3S and EPRI networks precipitation is not always recorded (i.e. number of
collection periods
-------
TABLE C.4. DIFFERENCES IN THE SEASONAL AND ANNUAL 1980
AMOUNTS OF SULFUR WET DEPOSITION (EXPRESSED AS A PERCENT)
AT COLOCATED OR CLOSELY LOCATED SAMPLERS.
PERIOD
Annual
Winter
Spring
Summer
Autumn
OVERALL
RATING
1
1
land 2
land 2
1
1
land 2
land 2
1
1
land 2
land 2
1
1
land 2
land 2
1
1
land 2
1 and 2
DISTANCE
(KM)
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
< 1
< 90
NUMBER
OF PAIRS
1
2
3
7
3
7
6
19
2
7
4
15
3
10
7
20
2
8
5
42
MAXIMUM %
DIFFERENCE
8
26
11
27
49
70
49
70
13
54
19
54
32
59
32
70
13
44
35
99
MEAN%
DIFFERENCE
8
17
9
13
30
37
24
29
11
17
15
24
26
28
19
21
7
24
19
33
C-12
-------
TABLE C.5. DIFFERENCES IN THE ANNUAL AMOUNTS OF SULFUR
WET DEPOSITION (EXPRESSED AS A PERCENT) AT COLOCATED OR
CLOSELY LOCATED SAMPLERS FOR 1981 -1982.
PERIOD OVERALL DISTANCE NUMBER MAXIMUM % MEAN %
RATING (KM) OF PAIRS DIFFERENCE DIFFERENCE
1981 1 < 1.1 5 12 9
1 < 26 11 44 19
1 and 2 < 1.1 9 13 8
1 and 2 < 26 27 44 26
1982 1 < 1.1 7 15 7
1 < 26 15 37 10
1 and 2 < 1.3 13 39 16
1 and 2 < 26 34 65 18
C-13
-------
APPENDIX D
COMPARISON OF PREDICTIONS AND OBSERVATIONS
OF SULFUR WET DEPOSITION AT THE ISDME SITES
D-l
-------
D
UH
P
co
o
00
ON
P
Z
z
o
CO
z
o
oo
Opq
wffi
i
Uh
>i
co
QPH
3§
H-l ,
-
(0
UJ
o
in i
CO CO t°* Ift
co ui o o-
O oioKti/ioovD~rCOUl-
-------
Q
W
D
Z
P
z
8
Q
3
I O.
g
(O
Ul
s
(J
IU
o
h-rjcsicjcj itvjtococgascj-o
ocj-ofj*r>»<»-«\jeo-
-------
D
oo
ODL
< Ki-.ncr-.-t in in LA co o o o CM -* o* o^ific^cjeocou1)
ZZ O -»«-« O r-««-**«t>-i O OO r-* »H t-H KJ O <\J *-lr^OOOOOO
D.
fc 2
O "
^/ (/) O -H K> »-
c^
z
O 2
II M OOOOOOOOOOOOOOOOOO OOOO.-«OIOOOOOOOOOOOOO~<.-tO
b .
^K C^ M i^O^O^ CsJKlOO"-*^ -«
Wb^ :c «
f-H H- OOOO OC3OOOO O OO
£S
^ ITT WO
Q/H -I <
O UJ H
t^- (J o o:
<-. t( O Ul
»rooo-o
^H/^v Ot-
O^J MM OON-4 O ' Kl CM tM O O O >H ^ O CJ -< O O -H >*r*i
(1 -*(O
UP 5
00
-so
Qfe §
Wn
n u y
II, g
CQ n w
^ QJ M
D-4
-------
S
D
2
O
U
Q
3
o:
o _ __ _ __
- ^ (\j
CM CM
O.
(-
<
lOOOOOOrtl-<.rtOOOOOOI^HO-HO
CK
M
M
E
a
a
ct
LU
(O
ut
o
a
o
t-
o
o
a.
'
>
-------
o
00
O
*
CO
on
Z
O
r ^ \xj
«P
<*>.
2^
W'
OTJI
UP
Q'fc
C. 1
*c
5 "*'
UJ
a
10
o
o K
M M
-J (A
5
o f~ ^0 ^pjiftocj-oa}-yo>oa3e\jof^o-
orjroiooocMK>fjcjo^«-<-<-<-< -ncjMcjro.CMit\jc\jcji-Hcjcj^cjrjcjrtt^'t>\t\tO'f*OflO't4-CI*&
OMIOCJ M oMKifor*iror-Mf>r«.o
eor^r^ K No o
O^*HO o r^oj c\j
O O O O i
ojcjrjtoroooooooooo
31
H:
D-6
-------
<
r:
uj
w
o
ii/(wa'Kitorocgto
CJOC\JOrt.-l.-lKICvlKlevli-
-------
(*
D
5
D
C/3
O
oo
ON
i
04
W
S
s
t/5
fc
O
C/3
z
o
S C/3
Q W
W E
PS! en
Sz
QO
IP
^p
o^
p>
5«
>w
Pl^
Un
^x^ ^^
oo
oo
CQX4
OW
Up
ffiP
^
§1
§00
29
<^
lz
o§
^E
oo
^P
Q&
3S
^
tt
W
O
<
i
(E
D:
V)
_i
uj
(0
in
o to
> eo *
OOM^H ** *? o o -< Kl O I O O -«
r^rotoN r- & J>
UIOKI
orOsO'*O'Oo<-JOM-}'
o r^ocsjr- rooo K>K» r^ r^ r^of^ro
*« s±co«oco OK>CO r^
-------
Q
tq
D
Z
P
z
o
U
Q
S
co
o
O
-J
8
U
a
to
(Or~O-O
O>
I/I
'-* I I IOOO~H--H-o
-------
OS
D
s
D
00
o
&
i-H
Z
s
&
<
L.
O
oo
Z
O
s^
ss
£K
^£
«l
oo>-s
^3
o^
p5
W
^S
WQ
^1
ou
Wffi
ffiH
H^
UH<
O^
Z5
oo oo"
>I K
SSI
5^
og
^E
00
w-jO
Q'S
S°
I 0.
S
ui r-. co ro
- 03 < ^ -tOO CU O*-«*-4O*-<
OOOOOOOOOOOOOOOOOO OOOOOOOOOOOOOOOOOOOOOOOO
j'tfcoro ^r-r^Kf^M -< cvir~cj-«r'>-«o-<^i^Ki^«o--N«oiAr^Okr^inir»>oo^i^ui«j'»-tNocors-CT>i/iNO'^^o
O OO
OOO>-> O O O O O O
OOOOOOOOOOOOOOOOOOOOOOOO
OOKIO
KI-OOOJ
r^ r-*K>rooo
-H in ^- co o- o
o
o-
o -< c\j CJ o o cj cj ro Ki
o i o ^ o
HO
r^ KI KICM
r^ ^ ^ruir
o o t i
D-10
-------
Q
UJ
D
Z
P
Z
O
u
Q
w
hJ
V)
o t-
M M
-I (O
UJ
g
u
o
in i
M I
ooooooooooooooooooooooo
ooooooooooooooooooooooo
-H O O O - 1~>O~
-------
w
ffi
H
H
O
H
oo
O
(X,
W
Q
H
W
D
fa
D
CO
o
00
p
Z
S
$
UJ
a
o
i
o
a
M
_j
a
in
UJ
K
O
UJ
H
O
a:
coD
V)
UJ
o
i
1
1
«
1
»
4
a
<
5
UJ
a:
o
M
u
§
a.
a
V)
*
Q
H
a
M
l-l
1
p
a:
a
K-
UJ
V)
UJ
§
o
g
5
H
in
g
LJ
£
z:
M
CO
(0
UJ
<
^
Z
D
5 UJ
-< M
J 10
4
>
UJ
O
o
u
UJ
Q
V)
M
fx o o in
CO O CM O
o o o o
1 1 1
0 - O CM
ix ^ -« m
o o o o
CM vO CO Kl
ro ^ ro ->
o o o o
1 1 1
O Kl CM Kl
CM m in in
o o o o
rx -H co o
rn in in KI
O 0 0 0
o ~* o o
o o o o
1 1 1 1
~> CM -. 0~
O CM O in
O O 0 O
! xo in rx
rx m KI m
O O 0 O
in CM o CM
o o o o
r-l t-4 O rx
sD -H CJ 1
o o o o
Kl 0- 0 -«
<» CM O *
o o o o
1
m o fx KI
0 0 >O CO
CM o- -^ in
H KI sj- m
U LJ LJ O
CM o rx -a
CM Kl CM Kl
o o o o
CM * CM Kl
o o o o
H in Kl CM
o o "< o
o o o o
1 1
CM fx 0 in
£00-0
0000
K1 ix ix co
o *o >o O -O
o o o o o
§£«££
o o o o o
m » rx >o in
o o o o o
-< CM in co m
r~ * co -o in
o o o o o
r- co o i in
in
o
O Kl * M
U (J U 3
ix rx in in
CM CO O CO
0 -H 0 .1
1 1 1 1
O 9> O CO
Kl M Kl O
0 -» 0 0
CM O CM -O
Kl O* ^ CM
o o o o
1
CM O Kl K>
O i O O
1 1 1 1
O 1 Kl CM
O Kl CM «
o o o o
1
in ix i ,!
i m m o
O 0 O O
1
o KI o rx
o -« o o
1 1 1
e> N o -<
O CO
o-
rx co c
33:
_4
tf
o
1
cO
o
o
o
'
^
Kl
0
CO
CM
o
CM
o
o
Kl
o
a
O ^ CM Kl
33333
Kl
CO
o
1
in
CM
o
in
o
1
CO
o
in
CM
o
CM
o
N
in
o
IHt
o
CM
Kl
o
r-.
o
1
a-
o
1
Kl
CM
o
-------
Q
W
j~~)
Z
P
O
u
Q
W
J
Ul
*
3
2
O
M
!"
<
a
M
_1
> _J
N. UJ
** O
-1 O
ui x:
a
o
£
i
O
M
Hi
mat
< <
O 3
M Z
_J Z
^ ^
>
II
_l
<
3
O
M
V)
Ul
CK
O
Ul
NJ
HI
O
o:
Q
Z
>-
10
c
)
h
<
C
h
4
5
a.
2
UJ
a:
Q
M
o
I
a
o
*-* d
O O
1
O O-
o in
0 0
"< ^
Kl CM
O O
eo r-
O CM
0 0
1
in *o
O Kl
o o
1 1
rt 0
-I 1-4
0 0
1
r^ KI
O Cvl
O £>
~4 CVJ
CM CVJ
3 3
r^ o^
O CO
0 O
1
O- 1
o
o
^
o
o
in
Kl
O
-O
CM
O
1
Kl
CM
O
1
rH
CM
o
1
Kl
Kl
CO
O -H CM
333
Kl
0
t-l
1
-4
Kl
o
!_4
-"
M
1
N
Kl
O
^*
CM
O
<~t
Kl
O
1
p*
"
O
ft*
*
o
1
o
0-
o
1
in
o
1
to
3-
o
1
Kl
CVJ
-O
Kl
=
Kl O-
in j-
-< o
1 1
-« o
1 CVJ
0 0
-i CO
CM
o
o
o
o*
o
o
f^«
o
o
t-l
o
o
-o
o
1
H
*
O
1
a
3-
o
i
^
*
Kl
M) r^
-3- J-
3 3
CM
Kl
O
1
CO
CM
0
CM
Kl
O
1
I/I
O
l_l
*
o
~o
o
o
1
r^
^
o
m
o
1
o
Kl
0
1
CM
*-4
O
1
in
in
0
1
(^
sO
CO
CO^'O'HCMKI^m
33333333
sfi r* CO O*
in in in in
3333
D-13
-------
w
ffi
H
^
O
H
c/5
O
p ,
w
Q
H
ETJ
^
-x
p
UH
P
00
o
as
^^
f^
H
ITT
p +
N*N
^^
!^
tu
O
OO
>
p
g
55
PJ
PH*
Q
M
Noo
Q H
5 t/3
"T-
Q £?
Z
§
M
.<
a
M
f
< to
N LU
-1 0
UJ TL.
O
O
E
1
O
r- a
< UJ
Q r-
w Z
-J M
> 3
II
_j
^
3
s
UJ
cx
LU
NJ
n
a
cc
«£
|
H-
V)
e
»-
h
4
1
ft-
a
i:
i
UJ
D:
a
M
o
i
a
a
in
^
o
<
M
a
>-(
M
1
W-
P
a:
D
r-
UJ
IU
o
o
U
LU
0
M
i-H O LA -3*
O O CM Kl
O O O O
1
o -H ro KI
LA CM CM sO
0 O O O
1
CM O «-« O
- CM ro I-H
O i-H O O
1 1
O> >3" IA fx
in KI o ix
O O 0 O
Kl O CM in
CM CM £! in
o o o o
rx j- o- 4-
CM in ~* *o
O O 0 O
1 1
CO CO CM LA
in KI KI co
0 O 0 O
1 1
in NO in
O O CM LA
o o o o
LA Kl O CM
O Kl LA 0
O ~ 0 O
1 1 1
*3- -O CM Kl
O O O O
1
-i in m ix
ro CM o LA
0 O 0 O
1
CM IO CO Kl
Kl co ro CM
O 0 CM -<
-! Kl * LA -<
O O O O *<
U CJ CJ U U
-« LA cn
CM ro in
o o o
in KI
CM vj-
o o
Kl CJ N*-
o o o
1 1
-0 -« Kl
Kl |x tx
o o o
CM O- 0
3 -o r^
O 0 0
O CM sO
CM CM s3"
O 0 O
«O Kl sj-
O vfl
O 0
\3* (7s
1 O
o o
1
Kl Kl
O 0
LA -3-
m in
o o
ro ix
CM m
CM CM
sO |x CO
CJ CJ U
CM 0 CJ -H s3-
o in co ix <3- .
o o o o o
CM 0 <0
O O O
co ro .3- ro co
o o o o o
1 1 1
LA O O- <-« -!
in -3- co co rx
o o o o o
in in vo
« -o ±
* 0
1 1
-O 0
CO Kl
O O
1
O O
tT- LA
O CM
Kl J- -« CJ
CJ U 3 3
o
o
I
CO
o
o
1
CM
O~
O
1
CJ
CM
O
tr-
et
o
o
Kl
O
1
0*
CM
O
1
CM
O
1
fx
-31
^
1
in
o
1
Kl
O
'
O
CM
»H
Kl
§
KI r» in
-3- IA -t
CM O *
1 1 1
-3- ix rx
o o in
ro o o
i i i
-H -. CO
CO CM sO
«-* CM O
1 1 1
o KI rx
LA -1 0
O O O
1
3- Kl LA
0
MOO
1 1 1
f> Kl O
O t-< LA
1-4 t-^ O
1 1 1
-t O O-
rx CM -H
o o o
1 1
tx 0s «<
CM CM O
CM O O
1 1 1
Ix ,3- Ix
>o CM ro
-3- cj cj
i i i
sD LA fx
LA - CM
1 1 1
-H O Kl
O- LA LA
0 O
1 1 1
o rx KI
CO -O Kl
O O 1
3- in -o rx
3333
O CO CM
-< Ul -H
o o o
1
in ~o KI
O CM ^H
O O O
Kl CO
LA * Kl
o o -^
1 1 1
in * -«
ro IA CJ
Kl CJ -O
O O O
1 1
it CJ st
CM -1 Kl
O O O
1
i-< CJ Kl
-H ro o
o o o
1 1 1
rx ix ro
Ix CO -3-
o o -<
1 1 1
s3- Kl CO
0 O 0
1 1 1
O O i
CM -H
O O O
i : i
|x o N
KI co co
p-l -* O
co o o
333
3- -
o o
1
IA -
o o
o o
1
CM O
-< O
o o
1
O Kl
in in
o o
CM in
- Kl
O 0
1A CO
ro ro
o o
-» o
o o
rx r*
ro CM
0 0
1 1
o o^
rx i
o o
1 1
CJ sO
ro so
0
1 1
st LA
CM ro
0 0
ro KI
|x Kl
-< CM
-HCMKIsJ-lAsOrxCOOO
3333333333
D-14
-------
o
t-
1
a
a
i
Q
M
to
UJ
o:
a
LU
N
M
a
a:
Q
D
P
O
U
r-;
Q
(O
Ij
LU
O
O
r
i
1
«
i
t-
a
5
UJ
a:
o
M
u
£
a.
Q:
in
Q
M
a:
M
M
1
JE
tt
o
t-
Qf
UJ
in
UJ
o
o
o
i:
1^
^*
in
o
o
£
lL
M
in
l/>
LU
<
^
z
D
4 UJ
H M
J 10
I
>
UJ
a
o
u
Ul
Q
in
M
-4
O O O 7s
O O O O
1 1
fO ± o
o o o o
1 1 1 1
r>- o -^ CM
in KI CM o
O O -< Kl
1 1 1 1
^ o «-4 in
0 O O -H
1 1 1 1
c- in in ,0
-< Kl Kl O-
O O O O
1 1
0 0 Kl Kl
O O *0 CM
CM CM *-l -*
«* CM Kl s±
CM CM CM CM
p«.
Is-
O
1
O
CM
O
1
CO
f
o
1
Kl
*
O
-o
o
0
o
o
1
0-
in
o
o
-c
0
1
o
CO
rH
1
CO
^
1
c -
C'
o
1
CJ
Ul
_,
in -o o
CM CJ s±
333
^ <^ o ^ r^ c^ r^
vD i O CJ Kl CT- O
O 0 rt O O O
1 1 1 1 1 1 1
r^ *o o co KI -o co
Kl in -O Kl CJ -s
O O 0 O O f-4 0
1 1 1 1 1 1
*Q 9s ^4 Kl f^ >£ *~4
Kl Kl CO CO CO in CM
H O O O < CM -H
1 1 1 1 1 1 1
-. -0 «O -< Kl o Kl t
O O O O O O O
1 1
* Mn * Kl -. *
Kl >J O Kl 4 ^ O
O O O 0 O O O
1 1
Kl \O * CM CO CO O
r^ CM in -<
O O O O 0 0 O
1 1 1 1
«O CO O Kl %^ O <
-H CM CM Kl Kl >O »-H
O 0 -H O -H -< -<
1 1 1 1 1 1 1
CM CO ^ ^ C> CO >O
co CM in in co o h-
-H »-» CM i-l CM Kl «~l
1 1 1 1 1 1 1
CM ^ in CM *-H Kl ^*
o -< -» & in *o ^ co c^
in in in in in sn
333333
D-15
-------
w
ffi
o
h<
H
(K
GO
O
OH
W
Q
H
W
J
D
00
o
00
o
2
OH
O
co
D
Q
>i
00
W
Q
W
g°
<2< M
UJ
o
UJ
Q
O
r- O
O M
M o:
-i a.
< to
3
a
M
m
UJ
a:
a
UJ
M
Sw
>I >*H
CQ^
UJ
§
U
1
M
r*» & r4* ro
r^ o *-* >o
o o o o
i i i
in s-
o o o o
1
o o o o
ro -! co co
O vj- ft CM
o o o o
co f"*- CO co
r^ cj sj- ro
o o o o
a- i co in
in -a- o ro
o o o o
ro ro co cj
r« co ro ro
o CM ro -H
^< ro J- in
(J U U U
0 o o o
o
o
CJ O O CJ
o o o o
1 1 1
0 O O O
0 O O O
o o o o
o o o o
O 0 O 0
ro ro r~ ro
U (J U U
in o o *;f
f-l O 0 CM 1 -<
1 1 1 1 1 1
i CM in r- -f ro
o o o o o o
1
*j* co *o t** in cy*
co >H o * ro -*
r-l O O -H -! O
1 1 1 1 1
in *^ ^ ^it* *o in
o in ro CM o -3-
o o o o o o
1 1 1
* -i CO ro ro in
o <* *j- ro o **
o o o o o o
1 1
in -< -H tr- o ro o
0 O 0 1 O O
1 1 1 1 1
in c* >o m o^ r*»
in ro o o ft o
O O O ~4 O O
1 1 1
o o CM in ro ro
O O O O 0 O
1 1
r*» o »-« in -H -H
O O O -H O O
1 III
r*» *3* CM o* co o
co -H CM in in o
000*^00
i iii
* cj ro & o -o
*o o o r^ CM ^-4
o o o > o o
II III
r>. r~ ro ro o o
-< CM CJ «-l ^^ CJ
*>-icjro.*in\or
0333333:
>~« sO O
ro ro CO
000
o r~ in
o o o
in -o ro
ro ro in
o o o
1
ro cj cj
o in ro
O O 0
ro co ro
>o in ro
o o o
O* it i
CM CM O
O 0 0
1 CO f»
** ro CM
o o o
in o
0 0
CO »
in NO
o o
~< -a-
-< m
O 0
CO -1
-o
o o
CM ro
O 0
x^ CO
m r^
o o
ro r^
O 0
o r-.
O 0
r- o
t^ CM
CJ CM
ro - ro
o >*
cj ro
co o- o
333
D-16
-------
Q
W
£
Z
O
u
oq
Q
s
o
a
M
_l
<
I-
_i o.
< (/)
>
(O
a
UJ
KJ
M
O
5
o
0.
5
UJ
a
o
M
u
*t
a
<
i-~
**
o
>
M
OC.
M
1
C
a
0
t-
CC:
UJ
in
UJ
O
O
o
<
Q
o
£
LL
M
IAJ
£
z
o
M
h-
-J in
<
UJ
o
O
u
UJ
E
a
tn
M
o in -H ui
OOOO
1 1 1
O I/I It -3-
(O O
cj * o o>
OOOO
1 '
-* Lft O -t
r«- ro in *
OOOO
^o «~l If) CsJCVJCJ«-«i-f
OOOOOOOOOOO
1 '
UlCjrtMKIKIIOOIOKt^
OOOOOOOOOOO
1
?£?3«3o'2£2?J
OOOOOOOOOOO
1
«*ift Kooo^ Kt -f in h*tfio»
^jc5v±*tf'^«tf'55'5^?^'ifiu^iriiiAt/iSSir»Sir
^I3IDI33Z)O3IDI31333I3OOOZDI3OID2
D-17
-------
u
DC
H
Z
O
OO
O
OH
UJ
Q
H
W
UH
J
oo
o
00
OS
W
UH
O
00
D
Q
»i
00
W
Q
Wvi
Q£
3
§
a
o
z:
i
o
M
5
a
o
m
UJ
on
o
UJ
Nl
a
tx
a
r
(0
J
o
z:
c
t-
<
»-
<
o.
£,
UJ
a
0
M
U
a.
o:
§
M
a:
M
M
i
a:
a
h-
UJ
in
Ul
i;
a
o
i:
<
h-
V)
a
g
3;
M
10
to
<
1-
£
3
H
-
( UJ
1 M
J in
t
>
UJ
o
o
u
x:
0
M
in to f-*
O- 0 CO
o o o
i i
C\J *O C^*
h- in o
o o o
1
~i CM in
I/I 0 *
O O O
1
m *-H O
O 0 O
r- r- -.
o o o
m o to
cy* ("» co
o to to
1 tO s±
U (J U
<»
0
o
1
-« i-H
ooo
1 1 1
to -< o-
to in to
ooo
-HO*
CM Ul fO
ooo
CO to O
»
o r- >j
000
m r>» CM
o in o
ooo
1 1
in co h>
NO O CM
ooo
1 1 1
co m CM
CM vT 1
ooo
1 1
O CM f~
CO *O CO
CM tO CM
* in ^
333
in * -
to to to
ooo
1 1 1
CM 0 CO *
. , o
ooo
1 1 1
in r- -o
o -« to
ooo
O - -H
CM Ul -O
ooo
1
in o *o
CM IO *
ooo
1
to <
H CM fO
ooo
1 1
O <7- ^
-< in to
ooo
1 1
to o o
o to co
IO CM CM
h* CO O o
3333
r-> o-
r CM
o o
1
CO CO
o -o
o o
o to
O CM
o o
1
CO ~4
-« in
0 0
to r-
10 m
o o
o\ o
o o
1 1
tO CM
0 0
1
O f)
in in
0 0
CO CM
-± m
0 0
to *
to to
o o
r- in
-* to
0 O
IO M
* *
-* CM IO
333
CO
o
CO
o
o
o-
CM
o
£
o
1
o
o
o
1
o
CM
CM
0
o
1
CO
in
o
o
in
o
in
o
m
o
M
CO
o
333
CM
CM
o
to
o
in
CM
CM
-4
CM
o
1
CO
o
o
0
in
1
CM
r-
0
1
o
to
o
1
CO
o
1
to
a-
o
CO
3
10 0
o *-*
CM CM
1 1
in in
i tO
0 0
1
o
0 0
1
in -o
CM tO
0 0
O f-
-H CM
0 O
-H O
in in
o o
1 t
M) in
-± m
o o
o to
to to
0 O
!» 0
1 O
0 O
1 1
CO CM
o o
1 1
to o
sj- CM
o o
1 1
r«- o
"-I (M
co o o
333
D-18
-------
UJ
t-
a
M
1
<
UJ
o
i
o
a:
UJ
3
O
M
CO
UJ
a:
o
UJ
M
a
Q:
to
_J
UJ
o
x:
c
i-
c
t-
«
OL
z:
i
UJ
a
Q
M
U
£
OL
D£
CO
**
a
M
a.
M
M
1
(E
D£
§
UJ
CO
Ul
n
o
o
t-
<
1-
co
o
o
z:
z:
M
CO
CO
^
<
K
z
3
< UJ
3 H*
H M
J CO
i
>
UJ
O
o
U
Ul
o
CO
W
O
CJ
o
1
m
3-
o
1
to
o
^
o
o
rv.
in
o
^
o
o
p~l
f-
o
m
o
-t
3-
O
o
o
^
CJ
o
r»
in
ro
-4
CJ
3
O* O
r*» cj
o o
1
~i m
o o
cj r-
-H CJ
o o
to o
O i-«
O 0
cj tn
ro tn
o o
cj -a
o -a
o o
1
t-i ro
-«
31
O
f-4
->
r-t
1
to
o
o
ro
o
o
CO
o
1
_
o
o
CO
o
1
o*
0
_
1
ro
~«
rt
i
y,
o-
o
1
IX
31
^H
*o o
CJ ^3
3 3
O £
O
o ro
o o
1 1
0- 0
to in
o o
CO ->
3- -0
O 0
~i r*>
o ro
o o
i
ro o-
CJ CJ
o o
3- 0
O 0
0- f-
o ro
o o
r- in
O CJ
o o
1
CJ O-
o -<
0 O
1
to f*
r- o-
cj ro
~4 CJ
-r
0- CJ
0 O
1
Kl tO
~o to
0 O
in o
i O
1
-3- -H
-< o
1
-« CO
O -H
1 O
1 1
CO -3"
in o
~4 O
1
ro ro
-< -3-
-« in
to
H *
0 O
1 1
>3" CO
in in
o o
10 .0
in in
o o
cj CO
O i-l
o o
1
in -«
-« to
o o
in -o
o o
3- in
o o
o in
to cj
0 0
to
o
o-
o
1
CO
o
r-1
O
o
-*
N
o
0-
3-
o
IT-
in
o
f^
o
l-l
O 1 CJ K
in in in IT.
3333
CJ
t>
rt
1
fx
CJ
o
CO
o
^
(O
o
3-
ro
o
o
CJ
o
1
CJ
o
CO
o
o
4
CJ
0
CO
l-l
o
to
ro
o
1
(S^
-4
CJ
-3" in so r*« co o
in in in in in in
333333
D-19
-------
UJ
3
O
O
r-
O -I
HI -I
-J <
[In
O
2
^
D
Q
r**H
GO
w
&
Q
W
N .
rf
II
3
O
in
UJ
a
a
UJ
hj
HI
a
a
2
<
in
in
LLI
O
c
r-
t-
<
r-
4
0.
-t
JC
UJ
o:
a
M
U
a
ce
10
<
o
<
M
a-
M
M
1
t-
o:
a
r-
UJ
in
UJ
r
0
o
s:
r-
a
g
s:
H4
in
in
"*
<
r
5
*
( UJ
-1 HI
J V)
t
»
UJ
0
o
(J
!C
0
10
M
ro cr- fx ro
v- i -. ro -^
o o o o o
< o- in
in in in
0 00
CJ N? CJ o O
*H O -4 O O
III 1
-H rx cj ro NJ-
o o o o o
o o o o o
-4 Kl *-* -! -«
1/1 in fx «o in
o o o o o
co co in KI co
in KI rx m N^-
o o o o o
NO r- co rx rx
0 O O O 0
rx KI ro o o
in -3- co ON o
0 -1 0 rt CJ
u u u u u u u
0-
cr-
o
i
o
O-
0
1
CO
CJ
CJ
1
to
o
CJ
o
£
CJ
o
1
o-
o-
o
1
CJ
NO
O-
1
CO
o
1
o
0-
0
ro NJ- -^ «-
0 u 3 :
"* CJ O fx
Nj* ^ OJ CJ
O CJ O CJ
1 1 1
? *H CJ O
O r* O O
1 1 1
rx o ro rx
O CJ -H -H
i I i i
in cj NO -H
in o cj -«
0 O 0 O
cj rx rx o-
in cj i o
o o o o
1 1
N» .-" ro m
O CJ 1 -H
i i i i
O NO ^* O
o o ** in
o o o o
1 1 1
-i eo <» NJ-
-< co o ro
O " O i-l
i i i
rx 1-4 NO cj
NO o -3- ro
O Kl O Kl
1 1 1 1
eo c> * cj
Nj)
NO NO
o o
in t>
NO in
0 0
rx o
o o
i i
* 0
CJ "HI
O 0
o- o
Kl -O
0 0
NO Nt
O CJ
0 0
1
CJ CJ
0 CJ
o o
1
CJ CJ
0 O
in -H
cj KI Nt in
3333
Kl
o
r-4
1
1
o
Kl
1
o
o
1
CO
o
o
1
CJ
in
CJ
o
CJ
1
CO
CO
o
o
Kl
CJ
1
o
Kl
CJ
1
CJ
1
o
3 3
*
1
**H
o
NO
o
in
in
o
-4
o
o
ro
CO
o
1
Kl
in
o
Kl
NO
o
CJ
0
f-t
1
CJ
in
«4
1
Kl
o
1
Kl
0>
0
<0 o o
333
D-20
-------
Q
UJ
D
Z
»-H
H
Z
O
u
o
r<
Q
tq
UJ
lu
Q
Q
to
UJ
a
Q
IU
N
l-(
Q
a
a.
OtOrO «-H^3K1or»i-iO
O 0 0 O O 1 OO-HOOOOO
II 1 1 1 1 1 1 1 1 1 1
co cj m *o & *o *^ ro co ^ *^ co in co
-H-3-cjo-mcj ttocjrocgroiocj
oooooo oooooooo
II 1 1 1 1 1 1 1
-* m -ji^roro
oooooo oooooooo
1
cjocOcj CM iO *« «H OOi'-i^Q'twa
oooooo oooooooo
III II III
o-*
-^cjcjcr-f^-j- cjoo^cjfxr^o
OOOOO^H OOCJOOOO'H
II 1 1 1 1 1 1 1 1 1
^MCOOOCO ^o3'x;
3OOZD!un)IDZ)I3I3^3IDr3Z3Z3II
ft
*-4
O
-------
Q
W
^
OH
Q
o
p
w
V5
PQ
O
.in
O
2
O
CO
OH
feg
§°
Hg
K>
oS
CQ O
01
§
V)
uj in
3 UJ
o
UJ U)
K- UJ
U H
M M
O (O
UJ
»H (/I (- O
«/) _; < Z
3 -i or <
(O -J UJ
«t (O
t- o
UJ u. O
2 O
cr en
D- M
r <
o o
o ^
(J
»H
Z Q
O UJ
w o:
h- a
UJ
a: o
a. uj
o >
o or
UJ
a
Vt
D:
a.
x
u
§
in -i
ce uj
UJ I- X
p M u
|cOM
ooooooooooo
ooooooooooo
ooooooooooo
ODOOKIltM-3-lfliriKlO
a. a. a o !
< < o < JC :
a- r < I- E i
^
ad
c
H
»-
<<
C
D
C
(.
§
h-
<
O
M
_J
|
3
V)
UJ
D£
II
_l
3
O
M
CO
a
o
UJ
N
M
D:
<
g
<
»-
to
_i
UJ
o
o
1
M
1-
<
o
M
_J
<
>
II
_J
3
a
en
a:
a
K
0
4 UJ
a:
C a.
i
-J
) <
) 3
a
(O
UJ
a;
>
UJ
a
e
i/>
z
<
UJ
£
>
UJ
Q
Q
H*
V)
5
UJ
3Z
|
X
ID
X
1
_l
<
g
-J
X
o
M
X
_J
«J
^f
§
_i
X
(9
M
X
_l
_l
<
§
g
M
X
_J
_I
<
§
X
u
M
X
1
_l
<
_l
UJ
ooooooooooo
I I I I I I I I I I I
ooooooooooo
I I I I I I I I I
ooooooooooo
I I I I I I I I
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
II I II
ooooooooooo
II II
ooooooooooo
II I II I
*-4i-ICVJC\J
-------
fe
o
C/3
Z
o
p
u
h-H
Q
W
C4
(X
Q
<* on
uo
_J co
Mo
a
UJ
u
Z D
U UJ
M CK
I- O.
J &
Ul
a: o
tt ui
O >
U tt
UJ
tn
to
o
§
C/5
ATION
^^*
05
W
02
O
W
g
UH
O
Z
o
t/J
n
Cl
Z
o
M
M
to
g
Ul
Q
§
U.
«n
Ul
2
t-
0
in
ui in
33
> «<
EDICTEO
SITES V
o: z
a. o
M
in >-
-i <
UJ O
0 M
0 -1
i: <
Ul
x a
H Ul
-i or
1 Ul
< in
tn
u. O
O
Ul
Z X
O H
in
M 1-
n i_i
O
CO
0-
o
o:
Ul
3
ID
U <
o
M
M
Ul
O
g
o
in -i
CH Ul
UJ >- X
CD M ID
t tn M
ooooooooooo
ooooo CJCMOOOCM
ooooooooooo
II III III
oocooroo**^«o^eoiA
a-inKi.jj'Ki^^i/i^s*
OOOOOOOOOOO
OOOOOOOOOOO
ooooooooooo
1OOOOOO -uj-i>:ca:z:<<<
UJtnOUJ>-HKUJ*HH>E
<ioi-5
a
to
UJ
on
ooooooooooo
I I I I I I I I I I I
ooooooooooo
I I I I I I I I I I I
ooooooooooo
I I I I I I I I I I I
1000000^000
ooooooooooo
<-4OO
ooooooooooo
II I I I I I I
OOOOOOOOOOO
II I I I I I I
OOOOOOOOOOO
II I I I I I I
OOOOOOOOOOO
OOOOOOOOOOO
OOOOOOOOO
O'-^OOOOO'HOOO
II I I I I I I
OOOOOOOOOOO
II I III
a. o. o o o
< to < i: z:
D: s: f i- a. t-
D-23
-------
O
g
5
CU
c«
b
O
C/5
Z
O
Q
W
«
DH
Q
00
z
o
p
^
«
W
(>0
CQ
O
w
ffi
H
u-
O
£
O
oo
HH
?
OH
S^'
8§
dfc
< 00
UO
C OH
|s
k
00^
^§
T-l P
a" El
WD
Joo
CQ o
£S
H t-H
I °
cn
u i
M I
O I
§1U
a
UJ
o
o
5
o
z
H-
m
3
JC
O
UJ
h-
u
: o
) Ul
^ a
- a.
:
j
'. a
* UJ
3 >
> a
UJ
H
_l
<
>
s.
<
3
O
M
UJ
a.
n
3
0
in
Ul
o:
o
UJ
M
M
O
fv
^
K
(O
_I
Ul
o
i
§
M
H"
<
o
w
_l
<
>
II
_l
<
o
o
in
Ul
ct
0
Ul
h-
U
) 0
H Ul
J
J 40
i <(
J 3
O
10
Ul
o:
>
Ul
O
a
H-
in
z
<
UJ
s
>
UJ
0
a
t-
tn
Z
<
UJ
r
i
X
ID
X
_j
_J
<
§
X
u
X
_J
.J
<
g
_l
X
o
M
X
_l
_l
<
3
o
_l
g
M
X
_l
_J
<
3
O
^J
X
o
M
X
J
_l
<
_l
Ul
ooooooooooo
I I I I I I I I I
< o 1/1 -o ^o -o
ooooooooooo
I I I I I I I I I I I
ooooooooooo
I I I I I I I I I
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
>IO-4OOOOOO
OOOCJOOOOOOO
O^^O'HOOOOOOO
o o o o
I I
o o o -> <
I I
8
< < Q < £
a i: < i- Q. t
D-24
-------
as
u
§
S
u>
1/5
UH
O
C/2
z
o
Q
W
tf
(X
Q
C/5
Z
O
w
C/3
PQ
O
w
o
on
HH
§
T-H I-J
Q'S
to
UJ 10
^3
>5
o
ui in
a «j i-
MM
o in
Z uj
O a: Z
M Q. O
-------
CO
o
p
05
u
CO
ca
o
w
UH
o
ft*
8§
feu
li
co^
ci5
WD
CQo
A
Dl
o
a
Ul
o
3
Ul
in
ui in
3 UJ
-I 3
< -I
in
U)
a in
E5
M
in H
uj a
o M
o -J
o
to
O1
IE °
_J
Ul <
> u.
a
UJ
o
a
a
<
i
»-
to
z
«t
jc
o
u
: o
- CL
C
j *a
j
i O
: LU
i >
i tt
UJ
lO
§
2:
ID
X
_l
-J
<
3
i
u>
X
_J
_l
«i
i<
a u
Ul 1-
CO *
E t
3
C
g
""*
X
0
M
X
_l
_l
<
o
fv
Q.
in
CO
0
Q
Ul
CK
Q.
in
co
o
a
ct
a.
in
CO
o
o
E
in
ca
0
a
Ul
£
in
ca
o
o
UJ
a:
Q.
in
ca
O
g
» -I
J
- X
H O
0 M
X
' j
<
_l
Ul
o
i
ooooooooooo
I I I
ooooooooooo
I I I
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
ooooooooooo
OOOOOOO-HOOO
ooooooooooo
o o
a a. o o o a
< < o < j: r: M
cr z: < K a. f- o
Kuj-J>i:a:i:<<<
inoui»HKujM»->z:
II
_J
<
3
O
M
in
UJ
a:
a
t-
i 0
| |^j
CK
; a.
j «
c
5<
J 3
O
in
Ul
a.
UJ
a
o
l-
in
2
^
UJ
E
UJ
o
a
(O
r
UJ
r
3
O
g
M
X
J
_i
4^
i
_j
X
o
X
_J
_l
3
O
_l
X
U)
M
X
_J
^
<
2
1
X
19
|^
X
_l
3
O
_l
X
(9
M
X
_J
_J
-1
Ul
§
ooooooooooo
I I I I I I I I I I
ooooooooooo
I I I I I I I I
ooooooooooo
I I I I I I I I I
ooooooooooo
OOOOOOO-HOOO
ooooooooooo
II I I I I I
ooooooooooo
III I I I I I
ooooooooooo
II I I I I I I
OOOOOOO«-(OO
cocor^o*>o>£
-------
APPENDIX E
ANNUAL 1980 MODEL PERFORMANCES
E-l
-------
30
TOTAL SULFUPI WET DEPOSITION kq S h»
ANNUAL 1980
48
MODEL AES ASTRAP MOE RELMAP RIVAD RTM StHIAD SIMPMOD STATMOD TVA UMACID VALIDATION
Figure E.I Annual 1980 box plots of the predictions and observations of sulfur wet
deposition (kg S/ha).
*
0
8 .
1
1 1
--- 1 1 1
1 1 1
1 1 1
- -- - » t - t - t - * *
III 1 .....
1 1 III
1 lit ---. 1 1
1 1 1 .....
-- II i-.-i 1 6 1
301 331 -- . .--- 131 1
1 125137 1
1 .1... 1 8
SULFATE CONCENTRATION ng m3
ANNUAL 1980
1
1 1
1 1
1 1
.- -. i 1 1
1 III
I . . i i
... 1 i 1 II i
1 1 1 .---
1 -- 1 1
- - 2 6 1 II -
26 1 II
1 1 1
I ---
1 271
1 1 ---!
1 291
1
SERTAO SIMPMOD STATMOD TVA
Figure E.2 Annual 1980 box plots of the predictions of sulfate air concentrations
(Ug/m3).
E-2
-------
1
1 ,
,.
'"
10 9 ..
1 10 5
3
3
0
I
I
, I
I
I
I
I I
I I
I |
I I 20 E
* «
8 5 I
I
0
0
I
I
I
I
I
1
1
1
1 >-
691 81
I
I
...I
SO2 CONCENTRATION ,,g mj
ANNUAL 1980
0
00 0
0
0
4 - »
, , , ,
_,
1
1 --- -4-
1 .---.
II II
1 . 1 . . .
1 1 .--- 1 1 981
1 1851 851 --- 1
5 4 1
ASTRAP MOE
RELMAP RIVAO
SERTAO SIMPMDO STATMOD TVA
Figure E.3 Annual 1980 box plots of the predictions of sulfur dioxide air
concentrations (|ig/m3).
TOTAL SULFUR DRY DEPOSITION, kg S ha,
0
0
4
0
_.
+ --
4--
.-- .
4 -*-
9 9 1
1
651 59 40
1 1
1 1
ANNUAL 1980
I
0
0
I
.... 0
1
«
1
0
0
1
1
1 0
1
1 0
1
1 1
. 1 . . . 1
---. 1 1 1 ..--.
1 1
4---. 1 |
112 II
I I
4 I 4 4--. 4 4
I 8 5 I
I I
I I -.-
-. -»- I
I I 4---4
I 4 4 4 4 6 8
I 6866
4 I 50 I
4 3 I I I
1 1
AES ASTRAP MOE RELMAP RIVAD RTM SERTAG SIMPMOD STATMOO TVA UMACID
Figure E.4 Annual 1980 box plots of the predictions of sulfur dry deposition (kg
S/ha).
E-3
-------
E-4
-------
tf>
z
o
I
cc
UJ
W
to
o
D
Z
E-5
-------
"Sf
z
z
<
Q
O
5
0.
S
V)
E-6
-------
C/J
oc
UJ
03
CO
O
I
E-7
-------
D
Z
Z
<
o.
<
tr
M
E-8
-------
'X
Z
5
,
//- 'i\
:!f vr
-------
<
D
Z
Z
a
o
1
C/5
fcfc
E-10
-------
MUM
Figure E.7 Annual 1980 locations of centroids.
MAXIMUM ANNUAL SULFUR WET DEPOSITION, 1980 (kg S/ha)
12.5/6 U24
23.1/9
/O= OBSERVED
MODELS
TVA
/2= AES
/3= SIMPMOD
/4= STATMOD
/5= MOE
/6= SERTAD
/7= RTM II
RIVAD
/9= ASTRAP
/A= UMACID
/B= RELMAP
19.6/1 «U46
18.9/2 »U44
12.6/5 13.8/4 /^
19"5/3
Figure E.8 Annual 1980 locations and magnitudes (kg S/ha) of the maximum amount
of sulfur wet deposition at the ISDME sites.
E-ll
-------
IS) Q. UJ Q. Q 5
UJ < O < «
£E S S > oc
oc
O
CO
O
O
18
16
14
12
10
8
6
4
2
0
rnn
SULFATE
SO2
Figure E.9 Comparison of the annual 1980 observations and predictions of the air
concentrations (|ig/m3) of sulfur dioxide and sulfate at the APN sites.
E-12
-------
o
00
2
^
Z
^
W
H
O
v^
UH
1
D
^2
^T
u
^<
>25
UJ
U
1
p^
o
UH
U
OH
J
PL)
Q
O
S
o
ks^
(2j
1
n
C/5
T~H
^"^ Q
30
> 1 1 i
i
§**
>-
£
X
£
w
X
af
>
QC
si
oe.
X
i
>-
*
<
00
CO
. * < o
f- a- o o f- *o
o o 10 o« P-- »n
in O tn O O> O
?\l >» TV
CO * ** >O K> (M
O O O CO Kl O
* F*-
O in o to (*- r-
O O O rO O O*»
CM o o> >* - co
OJ O O O CO vt
rsj in K> oj i*%
CO N- K) «~ O* F«-
(X - eo oj r\j o
r- S «- *-
P*- O O K) ^O O
KI ry o - N- r*-
f + + + O*
o f» in -o oo ro
° r ^ s s :
- + + + O>
in co rn *» *» oo
o - rg s. o o>
O C
Q i^ 1_ O ^0
C (AtDrtcAOJM
Q. o) w nj o OJ ra i-
O 0> 0> 3 O> D>
C **-££u3D V
i~ oi_oooo -c
0)0) o)zz(/i «- Tp
ffl «j u .- 5
c Q °
X - (\J
m
w <
-o -
° 3
OfM--jOOfMOOO T- IM CO
OOOKIOlMOrO WM
iKloSSoSr, fc^"
OOOOOOOs» +
oK538S5o o-ajo
o-oroo-oh- ^^M
o-*fs.»
o--*ooo» o«
rj
*
en *
S. -s - r 8
*N. x: to o 01
w» *-« *. en
^ CB *** ^ TJ
^^ «-^ jrf C v v
5»S.?S2^g- 33i
'L, 'Z _l^(/S O
Q.>(rt>«>c C*-»O««-
v)> ra< «) nj o c i
Q. O. Q. - Q. 01 C* <9 .C
to -^' in ' 0
CD
fc
in
S fc 0 sr rvj
00 2 ' '
5 S 5 t
fM -O CO O O
-S cor.ro
00 ^ * +
4 co ro o in
O O (SJ
o K- in -o -£
d d . . ' '
K> S O OQ K
O O "f^ .
in -
O -O CO O in -*
0 0
ro 5i (C "* J\J
O O IM V *
o -X r\j o o
O O fM
r g e
' "e « * S 2
^ "g *5 8- £ !
* .o» V oc u t »
. i ._(_,_ C C
i- U (/> i- O
W > Ol LJ
4> t C 'w -t- -t^ ^ g
a. a c c -> vi .
0) O O) CT (0 (0
x: ro o
- 01
1 "
a. o
a u
I 'f
I J
g !
U Jj
fe. »
«; ».
° S
0 S
a> i-
H &
a> «-
A)
a> £
2 !
O O
C H-
= *J
r -
." s
+ a
s >
« *
*
H(X
E-13
------- |