&EPA
United States
Environmental Protection
Agency
Office of Research and
Development
Washington, D.C. 20460
EPA/600/R-00/097
May 2000
An Accuracy Assessment of
1997 Land sat Thematic
Mapper Derived Land Cover
for the Upper San Pedro
Watershed (U.S./Mexico)
037LEB01.RPT * 11/30/00
-------
EPA/600/R-00/097
May 2000
An Accuracy Assessment of
1997 Landsat Thematic Mapper
Derived Land Cover for the
Upper San Pedro Watershed
(U.S./Mexico)
Susan M. Skirvin
Samuel E. Drake
John K. Maingi
Stuart E. Marsh
Arizona Remote Sensing Center
Office of Arid Lands Studies
University of Arizona
Tucson, Arizona
William G. Kepner
U.S. Environmental Protection Agency
Office of Research and Development
National Exposure Research Laboratory
Las Vegas, Nevada
May 30, 2000
-------
Notice
The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development
(ORD), funded and managed the research described here. This report has been peer reviewed by the
EPA and approved for publication. Mention of trade names or commercial products does not constitute
endorsement or recommendation by EPA for use.
Acknowledgements
We gratefully acknowledge Dr. Thomas M. Lillesand, Environmental Remote Sensing Center, University
of Wisconsin, Madison, WI and Dr. Christopher Elvidge, National Geophysical Data Center, National
Oceanic and Atmospheric Administration, Boulder, CO for their helpful criticism and suggestions as
reviewers for this report.
-------
Table of Contents
Notice ii
Acknowledgements ii
Executive Summary vi
Chapter 1 Introduction 1
Chapter 2 Methodology 6
2.1 Video and GIS Data Preparation 6
2.2 Video Sample Point Selection 6
2.2.1 Initial Homogeneity Screening 6
2.2.2 Statistical Requirements 8
2.2.3 Random Frame Selection and Evaluation 9
Chapter 3 Results and Discussion 11
Chapter 4 Conclusions 13
References Cited 14
-------
List of Figures
Figure Description Page
j_ Location of the Upper San Pedro River basin. Arizona/Sonora 2
2 Map of 1997 land cover for the Upper San Pedro watershed 3
3_ Video acquisition flight lines overlaid on the 1997 San Pedro land cover map 5
4 Video sample points overlaid on homogeneous areas on the 1997 land cover map 7
5 Locations of video sample points and misclassifications on 1997 land cover map 12
IV
-------
List of Tables
Table Description Page
! Land cover class descriptions for the Upper San Pedro Watershed (Maingi et a/.. 1999) .... 4
2 Availability of video sample points by land cover class in the 1997 map 8
3. Number of sample points per land cover class stratified by area 9
4 Results of video-based accuracy assessment of the 1997 land cover map: classification
error matrix, summary, and statistics 10
-------
Executive Summary
High-resolution airborne color video data were used to evaluate the accuracy of a land cover map of the
upper San Pedro River watershed, derived from June 1997 Landsat Thematic Mapper data. The land
cover map was interpreted and generated by Institute del Medio Ambiente y el Desarrollo Sustentable del
Estado de Sonora (IMADES), Hermosillo, Sonora and supplied to the Arizona Remote Sensing Center at
the University of Arizona for evaluation. Map pixel size had been increased from 30 to 60 meters to
match the 1973, 1986, and 1992 North American Landscape Characterization (NALC) land cover maps
produced from Landsat MSS data.
The airborne color video data included six flight lines acquired 2-5 May 1997 over the San Pedro
watershed in the U.S. GPS time and coordinate information encoded on the video tapes were used to
generate GIS point coverages of video frames covering the upper San Pedro. A total of 527 video sample
points were drawn randomly from a subset of 4567 frames falling on areas of uniform cover classes at
least 180 meters square. Sample points were stratified by cover class area, with a minimum sample of 24
points for classes of small areal extent. The Water class was extremely rare (covering less than 0.1% of
the study area) and was excluded from video data analysis for lack of data. Video sample points were
reviewed by an experienced interpreter who assigned land cover class labels based on available
descriptions. Map and video labels were compared to generate a classification error matrix, which
produced an overall map accuracy of about 72%.
VI
-------
Chapter 1
Introduction
There is keen interest among Federal agencies, States, and the public to evaluate environmental
conditions at community, watershed, regional, and national scales. Advances in computer technology,
geographic information systems (GIS) and the use of remotely sensed imagery have provided the first
opportunity to assess ecological resource conditions at a number of scales and to determine cross-scale
relationships between landscape composition and pattern, fundamental ecological processes, and
ecological goods and services. Providing quantifiable information on the thematic and spatial accuracy
of land use and land cover data derived from remotely sensed sources is a fundamental step in achieving
goals related to performing large spatial assessments using space-based technologies.
The U.S. Environmental Protection Agency has established a priority research area for the development
and implementation of methods to document the accuracy of classified land cover and land
characteristics databases (Jones et al., 2000). The research provided in this report assesses the accuracy
of a classification product, derived from a Landsat satellite platform, for a watershed in southeast
Arizona and northeast Sonora, Mexico. Secondly, it provides a methodology using georeferenced high-
resolution airborne videography as a substitute for actual ground sampling for contemporary imagery.
A land cover map for the Upper San Pedro Watershed (Figure 1) was provided to the Arizona Remote
Sensing Center, University of Arizona, by the Instituto del Medio Ambiente y el Desarrollo Sustentable
del Estado de Sonora (IMADES) for classification accuracy assessment. The map (Figure 2) was based
on a digital classification of a Landsat Thematic Mapper (TM) image acquired on 8 June 1997. The same
10-class classification scheme used for 1973, 1986 and 1992 North American Landscape
Characterization (NALC) land cover maps was used in the preparation of the 1997 map (Table 1). The
selected cover classes represent very broad biome-level categories of biological organization and are
similar to the ecological formation levels as provided in the classification system for biotic communities
of North America (Brown et al., 1979). The ten classes included Forest, Oak Woodland, Mesquite
Woodland, Grassland, Desertscrub, J^iparian, Agriculture, Urban, Water, and Barren. The classes were
selected prior to digitally classifying the imagery, after direct consultation with the major land managers
and stakeholder groups within the San Pedro watershed in Arizona and Mexico. The resolution of the
1997 land cover map had been degraded from 30 meter to 60 meter pixel size for consistency with NALC
land cover maps derived from Landsat MSS data for the project area under a previous accuracy
assessment study (see Maingi et al., 1999).
Accuracy assessment of the 1997 land cover map was performed using airborne color video data encoded
with GPS time and latitude and longitude coordinates. The video data were acquired between 2 and 5
May 1997 and were therefore nearly coincident with the June Landsat TM scene. There were 11 hours of
continuously recorded videography of the San Pedro Watershed for the area north of the US-Mexico
border. The video was acquired at a flying height of 600 m above ground level (agl). The camera used a
motorized 15X-zoom lens that was computer controlled to cycle every 12 seconds during acquisition,
with a full-zoom view held for three seconds. The swath width at wide angle was about 750 m, and
-------
approximately 50 m at full zoom. At full zoom, the ground pixel size was about 7 cm, and the scale of a
frame, when displayed on a 13 inch monitor, was about 1:200. The video footage was acquired by flying
north-south transects spaced 5 km apart and the total flight coverage encompassed a distance of nearly
2000 km (Figure 3V
A similar video data set was acquired over the same transects between 17 and 19 November 1995. This
video was available for comparison with the interpretation of the 1997 video. For another research
project (McClaran et al., 1999), over 550 full zoom video frames from the 1995 video were selected and
interpreted and estimates of canopy cover and plant density of species or species groups determined.
Identification of species was accomplished by using cues such as size, shape, color, texture, shadow and
context, along with ancillary knowledge derived from fieldwork. However, resolution of the video data
did not permit species identification of grasses and forbs or small shrubs under 1 meter tall. Although
the nominal accuracy of the encoded GPS coordinates acquired using a Trimble Basic Receiver was only
100 m, ground sampling revealed the positional accuracy was closer to 40 m.
Upper San Pedro Watershed
Galiuro Mts.
inchester Mts.
10 0 10 20 30 Kilometers
Sierra Mariquita •
Little Dragoon Mts.
Sierra San Jose
Sierra Los Ajos
Figure 1. Location of the Upper San Pedro River basin, Arizona/Sonora.
-------
1997 Land Cover Classes
| | Forest
| | Woodland Oak
^| Woodland Mesquite
| | Grassland
| | Desertscrub
| | Riparian
| | Agriculture
| | Urban
| | Water
^B Barren
I Cloud
20
IN
A
40 Kilometers
Figure 2. Map of 1997 land cover for the Upper San Pedro watershed.
-------
Table 1. Land cover class descriptions for the Upper San Pedro Watershed (Maingi et al, 1999).
Forest
Vegetative communities comprised principally of trees potentially over 10m in height
and frequently characterized by closed or multi-layered canopies. Species in this
category are evergreen (with the exception of aspen), largely coniferous (e.g.
ponderosa pine, pinyon pine), and restricted to the upper elevations of mountains that
arise off the desert floor.
Oak Woodland
Vegetative communities dominated by evergreen trees (Quercus spp.) with a mean
height usually between 6 and 15m. Tree canopy is usually open or interrupted and
singularly layered. This cover type often grades into forests at its upper boundary and
into semi-arid grassland below.
Mesquite Woodland
Vegetative communities dominated by leguminous trees whose crowns cover 15% or
more of the ground often resulting in dense thickets. Historically maintained maximum
development on alluvium of old dissected flood plains; now present without proximity
to major watercourses. Winter deciduous and generally found at elevations below
1,200m.
Grassland
Vegetative communities dominated by perennial and annual grasses with occasional
herbaceous species present. Generally grass height is under 1 m and they occur at
elevations between 1,100 and 1,700m; sometimes as high as 1,900m. This is a
landscape largely dominated by perennial bunch grasses separated by intervening
bare ground or low-growing sod grasses and annual grasses with a less-interrupted
canopy. Semi-arid grasslands are mostly positioned in elevation between evergreen
woodland above and desertscrub below.
Desertscrub
Vegetative communities comprised of short shrubs with sparse foliage and small cacti
that occur between 700 and 1,500m in elevation. Within the San Pedro river basin this
community is often dominated by one of at least three species, i.e. creosotebush,
tarbush, and whitethorn acacia. Individual plants are often separated by significant
areas of barren ground devoid of perennial vegetation. Many desertscrub species are
drought-deciduous.
Riparian
Vegetative communities adjacent to perennial and intermittent stream reaches. Trees
can potentially exceed an overstory height of 10m and are frequently characterized by
closed or multi-layered canopies depending on regeneration. Species within the San
Pedro basin are largely dominated by two species, i.e. cottonwood and Goodding
willow. Riparian species are largely winter deciduous.
Agriculture
Crops actively cultivated (and irrigated). In the San Pedro River basin these are
primarily found along the upper terraces of the riparian corridor and are dominated by
hay and alfalfa. They are minimally represented in overall extent (less than 3%) within
the basin and are irrigated by ground and pivot-sprinkler systems.
Urban
(Low and High Density)
This is a land cover dominated by small ejidos (farming villages or communes),
retirement homes, or residential neighborhoods (Sierra Vista). Heavy industry is
represented by a single open-pit copper mining district near the headwaters of the San
Pedro River near Cananea, Sonora (Mexico).
Water
Sparse free-standing water is available in the watershed. This category would be
mostly represented by perennial reaches of the San Pedro and Babocomari rivers with
some attached pools or represses (earthen reservoirs), tailings ponds near Cananea,
ponds near recreational sites such as parks and golf courses, and sewage treatment
ponds east of the city of Sierra Vista, Arizona.
Barren
A cover class represented by large rock outcropping or active and abandoned mines
(including tailings) that are largely absent of above-ground vegetation.
-------
Flight Line ID
. May59711
• May49722
• May49712
May39722
• May39712
• May29711
1997 Land Cover Classes
| | Forest
| | Woodland Oak
_ | Woodland Mesquite
| | Grassland
| | Desertscrub
| | Riparian
| | Agriculture
| | Urban
| | Water
^B Barren
I Cloud
20
v>
Figure 3. Video acquisition flight lines overlaid on the 1997 San Pedro land cover map. Flight line colors
represent the different acquisition dates during 1997.
-------
Chapter 2
Methodology
2.1 Video and GIS Data Preparation
Information for the 1997 land cover map indicated a projection in UTM Zone 12, based on the Clarke
1866 spheroid and NAD27 datum. The map was clipped to the revised upper San Pedro watershed
boundary (Maingi et a/., 1999) which encompassed the 1973, 1986, and 1992 NALC land cover maps
using the same projection. The encoded airborne video GPS time and geographic coordinate data were
extracted from the video into a spreadsheet for each flight line. Coordinate data were exported from each
spreadsheet as a text file in Arc "generate" format (ESRI, 1998), then transformed into an Arc/Info point
coverage of the video flight line. Flight line coverages were reprojected from geographic to UTM
coordinates with the same projection as the 1997 land cover map (Figure 3).
Individual frames of the video data were identified during viewing by a time display showing
hours:minutes:seconds, in addition to a counter which numbered the 30 frames recorded per second.
Encoded time data (available only as hh:mm:ss), along with flight line tape number, were exported from
the spreadsheets as delimited text, then imported into Arc View 3.1 (ESRI, 1998) and re-exported as a
dBase table which was joined to the point attribute table of the flight line coverage. This roundabout
procedure was necessary to preserve the time information as character data in the desired format and to
link times and tape numbers with the flight line points, each of which represented the first of 30 frames
per second. The coverages were inspected for erroneous coordinate or time data, indicated by points
which fell off the flight lines or which were out of time sequence; such points were deleted.
2.2 Video Sample Point Selection
2.2.1 Initial Homogeneity Screening
To minimize the likelihood of video sample points falling on boundaries between land cover classes,
selection of random sample points along the video flight lines was restricted to relatively homogeneous
areas within classes. This was accomplished by applying a 3 x 3 diversity filter to the 1997 map
(ERDAS, 1998). The diversity filter replaced the center pixel in a moving window by the number of
different data file values (land cover classes) present among the pixels in the window. Pixels assigned the
value of one therefore represented centers of 180-meter square homogeneous areas on the map.
Background areas (class = 0) were excluded from the process, while the diversity function was not
applied at clouds or cloud-shadowed pixels (class = 11). These restrictions prevented the selection of
pixels that fell at the edge of the map or within openings in clouds and cloud-shadowed areas, where the
adjacent land cover classes were not known.
The diversity-filtered image was converted to an Arc/Info grid and cells with value = 1 were selected to
create a homogeneity mask, with other cell values set to NOD ATA. The 1997 land cover map was also
-------
converted to an Arc/Info grid and the homogeneity mask was applied in order to select only land cover
cells falling in homogeneous areas (Figure 4). The masked homogeneous land cover grid was then
converted to a polygon coverage.
JP,
&
s- . . *
^f ;1^
'X-** &.-t*^&t
a l
1997 Land Cover Classes
| | Forest
^ Woodland Oak
| Woodland Mesquite
^] Grassland
^\ Desertscrub
^ Riparian
| Agriculture
| Urban
| Water
| Barren
H Cloud
Figure 4. Video sample points overlaid on homogeneous areas on the 1997 land cover map. The
homogeneous areas were determined by using a diversity filter.
-------
In order to determine which points along the video flight lines fell within homogeneous areas, video
point coverages were intersected with the homogeneous land cover polygon coverage in Arc/Info
(Figure 4). The resulting intersected coverages were merged into a single point coverage, representing the
total number of sample points that were available from the video for classification accuracy assessment
(N = 4567). The range of 1997 land cover classes represented in the video sample points is shown in
Table 2. as well as classes for all flight line points falling within the upper San Pedro watershed (N =
18,104). Land cover class information was added to the video sample point coverage by summarizing
classes of the land cover grid on zones of the sample point coverage in Arc View, then exporting the
resulting .dbf table to Info and joining to the sample point attribute table in Arc/Info.
Table 2. Availability of video sample points by land cover class in
the 1997 map.
1997 Land Cover Class
Forest
Woodland Oak
Woodland Mesquite
Grassland
Desertscrub
Riparian
Agriculture
Urban
Water
Barren
Total
Video Points
177
1721
2312
5245
6876
327
399
459
6
75
Video Points in
Homogeneous Areas
122
1109
123
968
1576
148
228
243
1
51
2.2.2 Statistical Requirements
An equation based on binomial probability theory, relating classification accuracy assessment sample
size to overall classification accuracy and allowable error, can be used to calculate the allowable error on
the accuracy of a land cover map (van Genderen and Lock, 1977; Fitzpatrick-Lins, 1981, Marsh etal,
1994). The equation is:
where:
N = number of samples required
Z = Standard normal deviate for the 95% two-tail confidence level (1.96)
p = expected or calculated accuracy (in percentage)
q = 100-p
E = Allowable error
-------
We assumed a ± 5% allowable error and an overall map accuracy of at least 60%. Using the formula
shown above, we needed a total of 369 sample points for the accuracy assessment. Apportionment of
sample points to the different land cover classes was stratified according to area. However, because the
area covered by some of the smaller land cover classes was negligible compared to the rest of the classes,
these classes were not apportioned a sufficient number of sample points. If sample size within a stratum
is too small, chances are that no classification errors would be sampled even if the classification is poor
(Miguel-Ayanz and Biging, 1997). In such situations, van Genderen and Lock (1977) suggested that the
smallest sample in this class should be 20 or 30 for maps in which the admissible percentage errors are
15% and 10%, respectively. For this reason, the minimum number of sample points for any class was 20,
which increased the minimum total number of sample points from 371 to 464 (Table 3).
Table 3. Number of sample points per land cover class stratified by area.
Land cover
Forest
Woodland Oak
Woodland Mesquite
Grassland
Desertscrub
Riparian
Agriculture
Urban
Water
Barren
Total
Area (Ha)
7193
90540
101559
263475
229571
9217
14530
16562
417
6814
739878
Proportion
of area (%)
1.0
12.2
13.7
35.6
31.0
1.3
2.0
2.2
<0.1
0.9
100.00
Calculated
samples
4
46
51
132
115
5
7
8
0
3
371
Minimum
number of
samples
20
46
51
132
115
20
20
20
20
20
464
Final number
of samples
24
55
56
159
137
24
24
24
0
24
527
2.2.3 Random Frame Selection and Evaluation
From the set of 4567 candidate frames, a random sample of 527 was chosen for analysis, stratified by
map class as shown in Table 3. Class 9, Water, was excluded from analysis for lack of data (only 6
possible frames; Table 2) and does not appear in the final error matrix. Within each remaining map
class, frames were assigned a simple, unique sequential identification number, and random selection of
these numbers was performed with Microsoft Excel 97's RANDBETWEEN function. A surplus of about
15% over the calculated minimum number of frames needed in each class was selected, giving the set of
527 frames instead of the minimum 464.
The spreadsheet records for frames selected for analysis were edited and pasted into a new worksheet
which contained only information on the frames' latitude, longitude, videotape library identifier, and
GPS time for frame location on the tape. This information was supplied to the videography interpreter,
along with the 10 map class descriptions shown in Table 1. and the instruction that each video frame in
the selected sample be located, visually interpreted and classified into one of the 10 map classes.
Since the videography varied in scale between wide angle and 15X full zoom, and no selection could be
made for frames at a particular scale, the operator visually estimated a 60x60m window around each
frame center for interpretation. The contents of this window in each case were used to classify a
"frame," regardless of the frame's actual scale. The local context for the interpretation of each frame
-------
was provided through analysis of the continuous videography. Although the accuracy of video frame
interpretation was not assessed explicitly in this study, it is expected to be very high. Drake (1996)
reported that land cover identification of similar airborne videography at the more detailed biotic
community level averaged 80% accuracy after only 3 hours of interpreter training. The interpreter for
this study had substantial prior experience in both video frame interpretation and ground sampling for
videography accuracy assessment in this region.
After interpretation and classification of the sampled 527 video frames, the assigned video classifications
were entered into a spreadsheet containing the assigned map classifications for each sample location, and
used as reference data to assess the accuracy of the map classifications. The facilities of SYSTAT 9
(SPSS, Inc., 1999) were used to create an error matrix for the data and to generate Cohen's Kappa and
Kendall's Tau-b statistics for quantification of overall classification accuracy. Raw percent correct for
the matrix, and user's and producer's accuracies for each class were also calculated. These results are
summarized in Table 4.
Table 4. Results of video-based accuracy assessment of the 1997 land cover map:
classification error matrix, summary, and statistics.
Land Cover Classes
1997
1
2
3
4
5
6
7
8
10
Grand Total
Reference (Video Frame Data)
1
20
2
0
0
0
0
0
0
0
22
2
4
50
1
8
4
0
0
0
0
67
Land Cover Class
1. Forest
2. Woodland Oak
3. Woodland Mesquite
4. Grassland
5. Desertscrub
6. Riparian
7. Agriculture
8. Urban
9. Water
10. Barren
| Total:
3
0
0
27
16
4
0
1
0
2
50
4
0
3
13
113
12
0
0
0
0
141
97
Map Total
24
55
56
159
137
24
24
24
N/A
24
527
5
0
0
12
21
115
0
15
0
19
182
6
0
0
2
0
0
21
2
0
0
25
7
0
0
0
0
0
2
5
0
0
7
8
0
0
1
1
2
1
1
24
0
30
10
0
0
0
0
0
0
0
0
3
3
Grand Total
24
55
56
159
137
24
24
24
24
527
Video
Total
22
67
50
141
182
25
7
30
N/A
3
527
Number
Correct
20
50
27
113
115
21
5
24
N/A
3
378
Producer's
Accuracy
(%)
91
75
54
80
63
84
71
80
N/A
100
User's
Accuracy
(%)
83
91
48
71
84
88
21
100
N/A
13
Overall Accuracy (%): 71.73
Coefficient
Kendall's Tau-B
Cohen's Kappa
Value
0.741
0.646
Standard
Error
0.024
0.024
10
-------
Chapter 3
Results and Discussion
Examination of the results in Table 4 by map class indicates that there is substantial variability in
classification accuracy. Producer's accuracy varies from 100% to 54%, and user's accuracy from 100%
to 13%. For most classes the two measures are roughly comparable and fall within the range of 60-90%.
The Barren, Agriculture, and Mesquite Woodland classes are those presenting some difficulty. Three
general explanations may be offered for discrepancies between map classifications and video
classifications, and the resulting low assessed accuracy of a class. First, locational errors or
misregistration of sampled points may be to blame; second, vagueness or inconsistency of the class
definitions used (leading to some unmeasurable "error" in the reference data) may be responsible; and
third, true thematic errors or misclassifications of map polygons may have been detected.
Of the above explanations, there is no evidence that locational error of the sampled video frames
contributed significantly to a reduction in the assessed accuracy of any class; such reduction can be
attributed to vagueness in class definitions and to true thematic map errors. The selection of candidate
frames of 60x60m only from within homogeneous map areas of 180x180m helped ensure that sampled
frames fell within the intended class, given the 40m mean locational error of frame center coordinates
established in an earlier validation study. Observation of the spatial context of some video frame
classifications differing from map classifications did not indicate that a coordinate offset on the order of
100 meters or less in any direction would have brought the two classifications into agreement.
Vagueness and overlap in the class definitions provided to the video interpreter were certainly an issue in
this accuracy assessment, and it is likely that doubt and inconsistency between video interpretation
criteria and satellite image classification criteria were the cause of some discrepancies between the video
reference data and the map data being assessed. All nine non-water classes as originally defined by the
satellite image classification group contained a certain degree of vagueness. This forced the interpreter
of the video frames to make subjective judgements that may have been contrary to those subjective
decisions made by the satellite image classification group. Of particular note are the difficulties in
separating Forest from Oak Woodland, and Mesquite Woodland from Grassland from Desertscrub based
on the initial idealized descriptions of these classes. In reality, these classes intergrade subtly more often
than not, and some viable criteria must be used to separate intermediate cases. No concrete criteria were
available at the time of analysis that would be appropriate at both satellite and video scales. Moreover,
the criteria of potential tree height (overlapping, in the critical case of Forest and Oak Woodland) and
general elevation ranges for classes, provided in the descriptions, could not be used practically during
video interpretation.
While some apparent classification error can be attributed to a lack of clear, applicable class descriptions,
it is probably much less than that attributable to true thematic error in the map being assessed. This is
certainly true for the low-accuracy classes labeled Barren and Agriculture. Review of the discordant
sample locations in these classes clearly showed, for example, characteristic Desertscrub in video frames
mapped as Agriculture, and Mesquite Woodland (>15% canopy cover) in frames mapped as Barren (see
11
-------
examples in Figure 5). While the Barren class had a producer's accuracy of 100%, its user's accuracy
was only 12.5% because so many mapped polygons were labeled Barren which the videotape revealed as
supporting a substantial vegetative cover of 30-50% or more. While no definite criterion was given in
the class description, a video frame was not classified as Barren unless its total cover was less than 10%.
Desertscrub mislabelled Agricultu
Desertscrub
mislabelled
Barren
Mesquite Woodland mislabelled Barren ?
Video sample points used
in accuracy assessment
• Incorrect land cover
class
• Correct land cover
class
1997 Land Cover Classes
Forest
Woodland Oak
Woodland Mesquite
Grassland
Desertscrub
Riparian
Agriculture
Urban
Water
Barren
Cloud
20
40 Kilometers
Figure 5. Locations of video sample points and misclassifications on 1997 land cover
map. Examples shown for land cover classes with low user's accuracy.
12
-------
Chapter 4
Conclusions
The results of this accuracy assessment show that six of nine classes were mapped with good to excellent
accuracy. Surprisingly, the contrasting classes labeled Barren and Agriculture were mapped least
accurately from the user's standpoint. Not surprisingly, the more ill-defined and heterogeneous class
Mesquite Woodland had both low producer's accuracy and user's accuracy.
The use of georeferenced high-resolution airborne videography as a proxy for actual ground sampling for
satellite image classification accuracy assessment has merit. A statistically meaningful number of sample
points can be gathered at practical expense. Coordinate locational error can be controlled and
compensated for in the methodology. The choice of scale in videography acquisition can allow for the
identification of plant species or communities, and the clear depiction of cultural features and land cover
characteristics.
The key to using aerial videography for a meaningful accuracy assessment is providing the video
interpreter with applicable classification criteria. Class definitions must be concrete, mutually exclusive,
and capable of making the distinctions necessary for classifying the inevitable video frames that seem
intermediate between two or more classes. They must also be congruent with those definitions used by
the developers of the image-map to be assessed, although exactly how to achieve this congruence may be
a larger question. Without the application of clear and appropriate criteria, the measured accuracy of a
map is liable to be biased negatively.
13
-------
References Cited
Brown, D.E., C.H. Lowe, and C.P. Pase. 1979. A digitized classification system for the biotic
communities of North America, with community (series) and association examples for the
Southwest. Journal of the Arizona-Nevada Academy of Science, 14 (Suppl. 1):1-16.
Drake, S. 1996. Visual interpretation of vegetation classes from airborne videography: an evaluation of
observer proficiency with minimal training. Photogrammetric Engineering and Remote Sensing,
62(8):969-978.
ERDAS. 1998. ERDAS IMAGINE. Version 8.3. Atlanta, Georgia: ERDAS Inc.
ESRI. 1998. ARC/INFO. Version. 7.1.2. Redlands, California: Environmental Systems Research Institute.
ESRI. 1998. ArcView. Version. 3.1. Redlands, California: Environmental Systems Research Institute.
Fitzpatrick-Lins, K. 1981. Comparison of sampling procedures and data analysis for land use and land
cover maps. Photogrammetric Engineering and Remote Sensing 47(3): 343-351.
Jones, K.B., L.R Williams, A.M. Pitchford, E.T. Slonecker, J.D. Wickham, R.V. O'Neill, D. Garofalo,
and W.G. Kepner. 2000. A national assessment of landscape change and impacts to aquatic
resources: a 10-year strategic plan for the Landscape Sciences Program. U.S. Environmental
Protection Agency, Office of Research and Development (EPA/600/R-00/001).
Kepner, W.G., C.J. Watts, C.M. Edmonds, J.K. Maingi, S.E. Marsh, and G. Luna. (In press). A landscape
approach for detecting and evaluating change in a semi-arid environment. Journal of
Environmental Monitoring and Assessment, 64(1).
Maingi, J.K., S.E. Marsh, and W.G. Kepner. 1999. An accuracy assessment of 1992 Landsat-MSS
derived land cover for the Upper San Pedro Watershed (U.S./Mexico). Web page [accessed 15
May 2000]. May be accessed at http://www.epa.gov/crdlvweb/pdf/3531eb99.pdf
Marsh, S.E., J.L. Walsh, and C. Sobrevila. 1994. Evaluation of airborne video data for land cover
classification accuracy assessment in an Isolated Brazilian Forest. Remote Sensing of Environment,
48: 61-69.
McClaran, M.P., D.M. Meko, S.E. Marsh, S.E. Drake, and S.M. Skirvin. 1999. Evaluation of the Effects
of Global Climate Change on the San Pedro Watershed: Final Report. The University of Arizona
and U.S.G.S. Biological Resource Division Cooperative Agreement A950-A1-0012.
Miguel-Ayanz, J.S. and G.S. Biging. 1997. Comparison of single-stage and multi-stage classification
approaches for cover type mapping with TM and SPOT data. Remote Sensing of Environment, 59:
92-104.
14
-------
SPSS Inc. 1999. SYSTAT. Version 9.0, Chicago, Illinois: SPSS Inc.
van Genderen, J.L. and B.F. Lock. 1977. Testing land use map accuracy. Photogrammetric Engineering
and Remote Sensing, 43:1135-37.
15
------- |