United States
Environmental Protection
Agency
Office Of TransP°rtation EPA420-P-05-002
and Air Quality February 2005
MOVES2004 Validation Results
Draft Report
-------
EPA420-P-05-002
February 2005
MOVES2004 Validation Results
Draft Report
John Koupal
Sujan Srivastava
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
NOTICE
This Technical Report does not necessarily represent final EPA decisions or positions.
It is intended to present technical analysis of issues using data that are currently available.
The purpose in the release of such reports is to facilitate an exchange of
technical information and to inform the public of technical developments.
-------
Table of Contents
1. Introduction 2
2. Total Fuel Consumption Validation 3
2.1 FHWA Fuel Consumption Estimates 3
2.2 MOVES2004 Estimates 5
2.2.1 Aggregation Level 5
2.2.2 Calculation of Fuel Consumption 6
2.3 Results 7
2.3.1 National Results 7
2.3.3 State-by-State Results 8
3. CH4 and N2O Inventory Comparison 12
3.1 U.S. Inventory Estimates 12
3.2 MOVES Estimates 12
3.3 Results 12
4. Fuel Economy Comparison 13
4.1 Fleet MPG vs. FHWA Estimates 14
4.2 Model Year MPG vs. EPA Fuel Economy Trends 16
5. Conclusions 19
Appendix A: Peer Review Comments 20
References 26
-------
1. Introduction
MOVES2004 is the first iteration of EPA's new generation mobile source
modeling framework. The model estimates energy consumption (total, petroleum-based,
fossil-based) and emissions of methane (CH/t) and nitrous oxide (N2O) for all on-road
sources, for the U.S (by state or county if desired), for calendar years 1999 through 2050.
Ultimately, the model will include emissions of "criteria" pollutants including ozone
precursors (HC and NOx), CO, PM and air toxics from all on- and off-road mobile sources,
and will replace the current EPA models MOBILE6 and NONROAD. Additional
background information and detail on the MOVES design and technical inputs are contained
in several other reports, which are best navigated using the overview document "A
Roadmap to MOVES2004".
The primary reason the first version of MOVES estimates energy consumption is
to validate model performance against top-down estimates of fuel consumption, compiled
from fuel sales tax records. In the report "Modeling Mobile Source Emissions" published
in 2000, the National Research Council stressed the need for EPA's models to undergo
more systematic validation and sensitivity analysis,1 and from the beginning validation
has been a top priority in the design and development of MOVES. Validation of
MOVES2004 results is important not only to gauge the accuracy of MOVES energy
consumption estimates, but also because many aspects of energy and emission estimation
methodology used in MOVES2004 will form the basis for criteria pollutant emission
estimation in later versions. Positive validation results provide assurance that the
underlying MOVES methodology is fundamentally sound.
Validation efforts have been conducted in the past on the MOBILE series of
models - however, as there is no true "top-down" measure of criteria pollutant emissions,
validation efforts are limited to a variety of methods meant to give a snapshot of overall
model performance. These validation methods include: tunnel studies, comparison to
independent emission data (e.g. chassis dynamometer or remote sensing studies), or using
ambient monitoring data to construct pollutant ratios (e.g. VOC:NOx) for comparison to
model predictions of these ratios. The results of such validations are often difficult to
draw conclusions from - for example the results of a recent MOBILE6 validation effort
sponsored by the Coordinating Research Council differed greatly depending on the
method used.2
Validation of fuel consumption is considered more reliable than criteria pollutants
because an estimate of top-down fuel consumption is available through fuel tax records.
Recent validation studies of EPA's NONROAD and the California Air Resources
Board's EMFAC models have employed top-down fuel sales, in the latter case as a step
in generating criteria pollutants validation based on fuel-specific emission rates.3'4 Top-
down fuel sales and energy consumption estimates are compiled by the federal
government - the Department of Transportation's Federal Highway Administration
(FHWA) and Department of Energy's Energy Information Agency (EIA) - and reported
annually, after some adjustment to attempt to account for uncertainties in end use and
fuel losses. Fuel tax receipts collected by individual states are used as the basis for
-------
compiling these estimates. The validation performed for this analysis compares the
bottom-up estimates of fuel consumption (via energy consumption) generated by
MOVES to these independent top-down consumption estimates for the entire U.S., and
by state.
In addition to this validation comparison, this report presents a comparison of
MOVES2004 methane (CH4) and nitrous oxide (N2O) estimates to the "official" U.S.
Inventory prepared by EPA's Office of Air and Radiation, and a comparison of MOVES
fuel economy estimates to those produced by FHWA and EPA's Fuel Economy Trends
Report. These comparisons are not considered to be independent validations of MOVES
because either they are derivative of total fuel consumption estimates, or they employ
methods of estimation which overlap with the "bottom-up" methods employed in
MOVES; however, we are including them in this report to give a sense for how MOVES
compares to current state-of-the-practice estimates of CH4 and N2O emissions and fuel
economy, and why differences occur.
The comparisons between MOVES and fuel consumption, CH4 and N2O
emissions are only made in the calendar year range from 1999 through 2002, since 1999
is the "base" year for MOVES2004 and the earliest for which the model produces
estimates, and 2002 is the latest year for which top-down fuel sales and CH4 and N2O
emission estimates are available. The comparisons can therefore provide a sense for
model performance during this short span of years, but it does not allow a check on the
validity of longer-term growth assumptions.
A pre-publication version of this report underwent formal peer review by
Professor Robert Harley of UC Berkeley; the resulting comments and our responses to
these comments are contained in Appendix A.
2. Total Fuel Consumption Validation
2.1 FHWA Fuel Consumption Estimates
Fuel consumption records are compiled by each state and submitted to FHWA,
who publishes the estimates in the Highway Statistics annual report series.5 Both raw
and adjusted results are presented in Highway Statistics, broken out by gasoline and
"special fuel". According to the Highway Statistics website, special fuels "include diesel
fuel and, to the extent they can be quantified, liquefied petroleum gases [LPG] such as
propane". According to MOVES, LPG accounts for less than 0.05 percent of special
fuel volume, so the "special fuel" category is essentially all diesel. For this analysis, we
used results reported in Table MF-21 from Highway Statistics 1999 through 2002. For
gasoline, we used total gasoline for highway use. This includes gasohol (E10) and
reformulated gasoline (RFG) sold as motor fuel. For diesel, we used special fuel for
private and commercial highway use.
-------
The following information on the FHWA fuel consumption estimates is for the
most part copied from the Highway Statistics website. The source of the motor fuel data
are state tax records, which are submitted to FHWA as raw consumption values. FHWA
adjusts the results by subtracting estimated non-highway use from the total use reported
by the states. According to the Highway Statistics website, "Over the last several years,
there have been numerous changes in State fuel tax laws and procedures that have
resulted in improved fuel tax compliance, especially for diesel fuel. The improved
compliance has resulted in increased fuel volumes being reported by the States to
FHWA."
The FHWA reported values do not include data on fuel purchased by the federal
government for military use or fuel exported from the United States. The gasoline
consumption levels include estimates of public use (separated into federal civilian and
state, county and municipal government), but the special fuel levels do not. This is of
note because this would mean that relatively large publicly-owned fleets of diesel
vehicles - e.g. garbage trucks, transit buses, school buses - would not be included in the
FHWA estimates. The sources included in the FHWA and MOVES fuel consumption
estimates are therefore not completely aligned, which contributes to some uncertainty in the
comparison between the two.
FHWA made additional adjustments to allow for losses from destruction,
evaporation, spillage, etc, and reports some variability among states in how this is
quantified: "Some States make a flat percentage allowance for losses in storage and
handling, and others allow for actual losses not to exceed a specified percentage. Still
others permit distributors to claim stock losses in reconciliations of inventories, thus
exempting the lost volume from taxation. Losses by destruction, where reported
separately, are also included in this column. The maximum allowance used in the
analysis to cover losses in storage and handling was one percent. Because of accounting
methods, losses can be reported as a net gain." Adjustments are made in the annual data
to exclude percentage losses in excess of 1 percent and to reflect usage rather than tax
collections.
The FHWA estimates used for this analysis are shown in Table 2-1.
-------
Table 2-1: U.S. Annual Highway Fuel Consumption Estimates from FHWA
(billion gallons)
Year
1999
2000
2001
2002
Gasoline
128.7
128.9
129.7
133.0
Special Fuel
31.9
33.4
33.4
34.8
50 State plus District of Columbia
2.2 MOVES2004 Estimates
2.2.1 Aggregation Level
MOVES2004 calculates energy consumption rather than fuel consumption. To
generate fuel consumption estimates, total energy consumption estimates were first
generated and converted to fuel consumption as a post-processing step.
There are several ways to produce national, annual totals of energy consumption
in MOVES2004 based on the choices the user has for pre-aggregation of geographic and
temporal resolution. At the most disaggregate level, the model could be run for every
county in the nation, by hour of the day, across all days of the week and months of the
year. At the most aggregate level, specifying a pre-aggregation level of "nation" and
"year" in the model will result in a pre-aggregation routine creating average inputs for the
entire nation as a single "county" and the entire year as a single "hour". There are
several options in between these two bounds: for example, geography could be pre-
aggregated up to the state level from county or time span could be pre-aggregated up to
the day or month level.
The key point with regard to aggregation is that results will be different depending
on the level of aggregation chosen. This is because some effects in the model (including
temperature, air conditioning, and the distribution of operating modes) are not linear,
and a single model run using an arithmetic average of several input data points will not
yield the same results as separate model runs at each data point. For example, a run at 40
degrees will not equal the average of the results from runs at 30 and 50 degrees, because
the equation for temperature effects (documented in the report "Energy and Emission
Inputs for MOVES2004", or Energy and Emission Report) is not linear. In MOVES,
temperature inputs vary by each county in the nation, by month and by hour.
Aggregating these inputs to the nation / year level produces a single average temperature
of 61 degrees. At this temperature, air conditioning effects are suppressed, but start
energy consumption is increased by about 37 percent due to temperature effect based on
Equation 9-5 in the Energy and Emission Report.
-------
Operating mode distributions are derived from the mapping of driving schedules
to average speed ranges, as discussed in the report "MOVES2004 Software Design
Reference Manual". In the pre-aggregation routine, average speeds are aggregated before
mapping to driving schedules, so the mix of drive schedules for the aggregated case
would not be the same as the mix for the disaggregate case. Average speed varies only
by hour of the day in MOVES2004 (not geographic location), so aggregation to the day
level (and higher) results in a mix of speeds that differ from the hourly levels.
The benefit of using the aggregation options in MOVES is model runtime
performance. At this stage the time required to do a run at the most disaggregate level
would be prohibitively long in single-computer configuration. The most aggregate case
(nation / year) is the quickest method for producing national / annual results, and would
be a likely choice for doing national-level runs. To assess the magnitude of difference
among pre-aggregation options, we performed a sensitivity analysis on total energy
results by generating MOVES national results for 2002 at four different levels of pre-
aggregation: nation/year, state/year, nation/month, and state/month. The results of each
run are shown in Table 2-2.
Table 2-2: Sensitivity Analysis of 2002 U.S. Annual Energy Consumption Estimates
at Different Levels of Pre-Aggregation (Petajoules)
Pre-Aggregation Level
Nation / Year
State / Year
Nation / Month
State / Month
Gasoline
15,751
15,853
16,089
16,142
Diesel
4,521
4,546
4,609
4,621
The trend shown here is that MOVES energy consumption results increase
slightly as geographic and temporal resolution is increased. When geography is varied
between nation and state and the time span is varied from year to month, results vary by
about 2-3 percent. Additional work is necessary to determine the sensitivity of MOVES
results to fine levels of dissaggregation, i.e. to the county level for geography, and day or
hour for time.
2.2.2 Calculation of Fuel Consumption
Total energy results from MOVES2004 for the state/month aggregation level
were generated for all MOVES source types (i.e. vehicle classes) for gasoline (including
E10 and RFG) and diesel, for each year from 1999 through 2002. Although MOVES can
generate estimate of petroleum-based energy, total energy is the appropriate metric of
comparison for this analysis since the FHWA estimates include E10 and RFG, and total
energy consumption estimates in MOVES accounts for the oxygenate used in these fuels.
-------
The next step was to convert the MOVES total energy results to fuel
consumption, which required estimates of heating value for each fuel, shown in Table 2-
3. There are many estimates of heating value in the literature, as they depend on fuel
density and quality; for this analysis we used Lower Heating Values (LHVs) from
Heywood6 to stay consistent with the methodology used in the development of
MOVES2004 energy rates (as documented in the Energy and Emission Input report). For
the validation analysis we developed an average gasoline energy content by weighting
together conventional gasoline, RFG and E10 energy contents according to the
national/annual volume shares of these fuels estimated by MOVES (67 percent
Conventional Gasoline, 21 percent RFG and 12 percent E10).
Table 2-3: Energy Content by Fuel Type
Fuel Subtype
Conventional Gasoline
Reformulated Gasoline
E10a
National Average Gasolineb
Conventional Diesel
LPG
E100C
Lower Heating
Value (KJ/gram)
44.0
42.9
-
-
43.2
46.4
26.9
Density
(Kg/gallon)
2.8
2.8
-
-
3.2
1.9
3.0
Energy
Content
(MJ/gallon)
124
121
120
123
137
89
80
a Volume-based weighted average of Conventional Gas (90%) and E100 (10%)
b VMT-based weighted average of Conventional Gas (67%), RFG (21%) and E10 (12%)
c Reference only
2.3 Results
2.3.1 National Results
The fuel consumption calculated from the MOVES total energy results are shown
in Table 2-4. The "special fuel" category for MOVES reflects diesel fuel only; LPG is
estimated to represent only about 0.05 percent of special fuel volume.
-------
Table 2-4: U.S. Annual Highway Fuel Consumption Estimates
from FHWA and MOVES2004 (billion gallons)
Year
1999
2000
2001
2002
Gasoline
FHWA
128.7
128.9
129.7
133.0
MOVES
126.6
127.9
129.0
131.5
% Diff
-2%
-1%
-1%
-1%
Special Fuel
FHWA
31.9
33.4
33.4
34.8
MOVES
30.8
32.0
32.7
33.8
% Diff
-3%
-4%
-2%
-3%
As shown in Table 2-4, the MOVES results compare well with the top-down
estimates. Gasoline results from MOVES are 1-2 percent lower than FHWA, depending
on the year. Special fuel results are 2-4 percent lower. For both fuels, MOVES tracks
the increase in consumption reported by FHWA across the years analyzed. We would
expect off-road use of motor gasoline and differences between states in reporting
methods and accounting for spillage and losses would contribute to the overall
uncertainty of the FHWA estimates. FHWA doesn't quantify this uncertainty, so it is
not known whether MOVES estimates would fall within the uncertainty bounds of the
top-down estimates.
One source of uncertainty in this comparison is the inclusion of some vehicles in
the MOVES estimates which are not included in the FHWA estimates. The FHWA
gasoline totals exclude military vehicles and the special fuel totals exclude all publicly-
owned vehicles. Military vehicle travel is not accounted for explicitly in MOVES, but
would be accounted for to the extent the FHWA VMT estimates (the basis of MOVES
activity estimates) include their travel. The MOVES source types do include some
vehicle categories likely dominated by publicly-owned vehicles, such as refuse trucks,
transit buses and school buses. In 2002, these three source types comprise about 1.3
percent of MOVES special fuel estimates. Perhaps a more equitable comparison would
be to remove these vehicles from the special fuel totals shown in Table 2-4, although
removing these vehicles wouldn't fully address this issue since a) some of these
categories do include privately-owned vehicles (e.g. commercial refuse haulers), and b)
there are publicly-owned vehicles in other MOVES categories (e.g. passenger fleet
vehicles) that cannot be removed from the MOVES estimates.
2.3.3 State-by-State Results
In response to peer review comments (Appendix A), we also conducted a
comparison of calendar year 2002 fuel consumption results between MOVES and FHWA
on a state-by-state basis. State level estimates are shown in Table 2-5, and maps of the
contiguous 48 states color-coded by the absolute difference in fuel consumption between
MOVES and FHWA are shown in Figures 2-1 and 2-2. Differences between MOVES
estimates and FHWA estimates vary quite a bit state-to-state, although overall agreement
-------
is good in many states, particularly for gasoline; the difference is 5 percent or better for
19 states, and 10 percent or better for 35 states. As illustrated in Figures 2-1 and 2-2, the
spread and magnitude of difference between MOVES and FHWA is larger for special
fuel than for gasoline. The maximum difference among states is 23 percent for gasoline
(Oklahoma), and 146 percent for special fuel (Hawaii).
Two factors contributing to larger differences in state-by-state comparisons
relative to the comparison of national totals are a) cross-border travel and b) differences
between fleet composition and activity patterns state-to-state, relative to the national
defaults used in MOVES. It is difficult to discern which factor contributes more in a
given state, although the cross-border travel issue is more relevant for smaller states,
particularly those with a lot of travel (e.g. commuter traffic) to nearby states. For
example, the MOVES gasoline results for New Jersey and New Hampshire are 16 and 19
percent lower than the FHWA estimates, respectively. Many vehicles based in these
states commute to nearby states; for example, 2000 U.S. Census data estimates that 13
percent of New Hampshire residents work in Massachusetts.7 The difference between
MOVES and FHWA fuel consumption estimates for states with high commuter outflow
likely reflects that a substantial amount of the fuel purchased in these states is used in
other states. The cross-border travel issue is accounted for to some degree with freight
trucks in the FHWA estimates through the International Fuel Tax Agreement (IFTA),
whose intent is to reallocate fuel taxes to states where fuel is used rather than sold.
Cross-border travel for light-duty vehicles are not accounted for explicitly by FHWA.
The second factor driving larger variability in state-by-state comparisons is how
well the national default assumptions characterize a particular state. The less
representative the national defaults are for a given state, the larger deviation from top-
down fuel sales we would expect. There are numerous inputs this would apply to,
including: the age of the vehicle fleet, congestion levels, meteorology, or terrain. For
example, MOVES would tend to overpredict fuel consumption for a state with a vehicle
fleet that is younger than the national average. For best results in state-level analysis, we
would recommend customizing MOVES to include state-specific inputs such as VMT,
age distribution and average speed distribution where available.
-------
Table 2-5: Comparison of Year 2002 Fuel Consumption Estimates by State
(x 100,000 gallons)
ALABAMA
ALASKA
ARIZONA
ARKANSAS
CALIFORNIA
COLORADO
CONNECTICUT
DELAWARE
DISTRICT OF COLUMBIA
FLORIDA
GEORGIA
HAWAII
IDAHO
ILLINOIS
INDIANA
IOWA
KANSAS
KENTUCKY
LOUISIANA
MAINE
MARYLAND
MASSACHUSETTS
MICHIGAN
MINNESOTA
MISSISSIPPI
MISSOURI
MONTANA
NEBRASKA
NEVADA
NEW HAMPSHIRE
NEW JERSEY
NEW MEXICO
NEW YORK
NORTH CAROLINA
NORTH DAKOTA
OHIO
OKLAHOMA
OREGON
PENNSYLVANIA
RHODE ISLAND
SOUTH CAROLINA
SOUTH DAKOTA
TENNESSEE
TEXAS
UTAH
VERMONT
VIRGINIA
WASHINGTON
WEST VIRGINIA
WISCONSIN
WYOMING
FHWA
2,541
237
2,541
1,383
15,386
2,023
1,520
409
141
7,720
4,827
433
625
5,061
3,077
1,519
1,161
2,063
2,219
699
2,505
2,786
4,997
2,587
1,537
3,008
476
830
974
688
4,003
905
5,617
4,140
336
5,094
1,708
1,515
5,114
390
2,270
419
2,989
11,116
993
336
3,792
2,660
799
2,474
313
Gasoline
MOVES
2,609
225
2,325
1,329
13,633
1,950
1,470
428
188
7,262
4,812
409
674
5,068
3,403
1,373
1,329
2,266
1,896
635
2,349
2,641
4,695
2,513
1,648
3,342
452
841
848
554
3,360
1,046
6,449
4,407
342
5,279
2,105
1,642
4,933
415
1,981
370
3,076
10,470
1,079
336
3,523
2,556
832
2,847
357
%Diff
3%
-5%
-8%
-4%
-11%
-4%
-3%
4%
34%
-6%
0%
-6%
8%
0%
11%
-10%
14%
10%
-15%
-9%
-6%
-5%
-6%
-3%
7%
11%
-5%
1%
-13%
-19%
-16%
16%
15%
6%
2%
4%
23%
8%
-4%
6%
-13%
-12%
3%
-6%
9%
0%
-7%
-4%
4%
15%
14%
FHWA
673
110
709
573
2,733
550
229
62
27
1,355
1,459
38
224
1,373
1,360
507
407
914
588
168
504
399
944
653
540
951
202
375
288
105
834
420
1,168
981
150
1,505
807
460
1,375
55
623
159
905
3,125
355
64
936
572
270
684
305
Special Fuel
MOVES
715
62
633
426
3,014
549
355
99
30
1,687
1,302
94
199
1,274
944
412
375
664
576
196
598
578
1,138
653
496
897
154
251
218
159
689
341
1,394
1,114
106
1,347
557
452
1,290
89
654
130
873
2,626
297
98
977
630
282
734
130
%Diff
6%
-44%
-11%
-26%
10%
0%
55%
60%
12%
24%
-11%
146%
-11%
-7%
-31%
-19%
-8%
-27%
-2%
16%
19%
45%
21%
0%
-8%
-6%
-24%
-33%
-24%
51%
-17%
-19%
19%
14%
-29%
-10%
-31%
-2%
-6%
61%
5%
-18%
-4%
-16%
-16%
52%
4%
10%
4%
7%
-57%
10
-------
Figure 2-1: Absolute Percent Difference between MOVES and FHWA Gasoline
Consumption Estimates by State (Continental U.S.)
Figure 2-2: Absolute Percent Difference between MOVES and FHWA
Special Fuel Consumption Estimates by State (Continental U.S.)
11
-------
3. CH4 and N2O Inventory Comparison
3.1 U.S. Inventory Estimates
In accordance with International Panel on Climate Change (IPCC) guidelines, the
U.S. EPA Office of Air and Radiation compiles and reports retrospective greenhouse gas
emission inventories for all sectors each year in the "Inventory of U.S. Greenhouse Gas
Emissions and Sinks". The latest version of this was published in April 2004, reporting
results for 1990 though 2002.8 The methods for generating CH4 and N2O in the
Emissions & Sinks report is a standard "bottom-up" methodology, combining gram per
mile emission rates for these pollutants with estimates of vehicle miles traveled (VMT)
and fleet mix to generate total mass emission estimates. CH4 and N2O estimates from
the Emissions & Sinks report were derived from Tables 3-22 and 3-23, "CH4 [N2O]
Emissions from Mobile Combustion". We also summed these pollutants by calendar
year, for gasoline and diesel. The summed results are shown in Table 3-1 and 3-2.
3.2 MOVES Estimates
For CFLt and N2O, MOVES is essentially an update of the process used for
Emissions & Sinks, with similar estimates for VMT and fleet mix but more recent data
for emission rates. CH4 and N2O emission rates were updated in MOVES2004 to
incorporate recent vehicle testing conducted primarily by EPA (for CFLt) and the
California Air Resources Board (for N2O). The details of this analysis are documented in
a separate report prepared by ICF Consulting.9 Since the emission rates have changed
significantly from those used in the Emissions & Sinks report, we do not expect MOVES
CFLt and N2O inventory results to track those from the Emissions & Sinks report.
3.3 Results
We compared aggregate on-road totals of CFLt and N2O generated by MOVES to
the 2004 Emissions & Sinks report. MOVES was run in national/annual pre-aggregation
mode to generate these results (unlike energy consumption, the level of pre-aggregation
will not affect CFLt and N2O results because temperature, A/C or speed effects are not
applied to these pollutants). CFLt and N2O results were summed across source use types
for gasoline and diesel. The results are shown in Tables 3-1 and 3-2.
12
-------
Table 3-1: U.S. Annual CH4 emissions from
Emissions & Sinks Report and MOVES2004 (gigagrams = 109 grams)
Year
1999
2000
2001
2002
Gasoline
E&S
174
169
164
159
MOVES
94
85
77
72
% Diff
-46%
-50%
-53%
-55%
Diesel
E&S
14
14
14
14
MOVES
0.4
0.5
0.5
0.5
% Diff
-96%
-96%
-96%
-96%
Table 3-2: U.S. Annual N2O emissions from
Emissions & Sinks Report and MOVES2004 (gigagrams = 109 grams)
Year
1999
2000
2001
2002
Gasoline
E&S
169
164
157
150
MOVES
115
111
106
102
% Diff
-32%
-32%
-33%
-32%
Diesel
E&S
10
10
10
10
MOVES
0.5
0.6
0.6
0.6
% Diff
-95%
-94%
-94%
-94%
The large differences between MOVES and Emissions & Sinks for CH^ and N2O
can be attributed directly to the updated emission rates. The new rates are significantly
lower than the rates used in Emissions & Sinks, particularly for diesel.
4. Fuel Economy Comparison
A useful comparison to make is between fuel economy (MPG) results derived
from MOVES2004 output and alternative estimates - on a fleetwide basis as estimated by
FHWA, and by model year as estimated from EPA label values. However, these
comparisons are not considered to be independent validations of MOVES results because
either they share some of the same data used in MOVES (in the case of FHWA), or
because they are simply alternate methods of generating fuel economy estimates (in the
case of the EPA label values).
13
-------
4.1 Fleet MPG vs. FHWA Estimates
MPG estimates for the entire on-road fleet are published by FHWA in the annual
Highway Statistics series, broken down by vehicle categories: passenger car, other 2-axle
4-tire vehicle (i.e. light trucks), bus, single-unit truck, combination truck and motorcycles.
The source of the "gallon" estimates are the top-down fuel consumption estimates compiled
from fuel tax records presented in Section 2. According to the Highway Statistics
Table VM-1, FHWA allocates total fuel consumption into each vehicle category based on
"miles per gallon for both diesel and gasoline powered vehicles using state-supplied data,
the 1997 VIUS, and other sources as a baseline"; hence, the FHWA MPG estimates by
vehicle category are based on an estimated allocation of total fuel consumption, and are not
a true top-down measure. The "miles" estimates are based on the estimates of total vehicle
miles traveled (VMT) compiled through FHWA's Highway Performance Monitoring System
(HPMS) and reported in the Highway Statistics reports (Tables VM-1 & 2); these VMT
estimates are also used in MOVES.
MOVES MPG estimates were calculated from VMT estimates produced by the
model divided by fuel consumption as derived from total energy results presented in
Section 2. To match the vehicle categories reported by FHWA, a post-processing step
was necessary to combine MOVES source types (which were subsets of the HPMS
vehicle categories) into the HPMS categories according to the breakdown shown in Table
4-1.
14
-------
Table 4-1: HPMS Vehicle Classes & MOVES Source Use Type
HPMS Class
Passenger Cars
Other 2-axle / 4-tire Vehicles
Single Unit Trucks
Buses
Combination Trucks
Motorcycles
MOVES Use Type
Passenger Car
Passenger Truck
Light Commercial Truck
Refuse Truck
Single-Unit Short-Haul Truck
Single-Unit Long-Haul Truck
Motorhome
Intercity Bus
Transit Bus
School Bus
Combination Short-Haul Truck
Combination Long-Haul Truck
Motorcycle
Because MOVES2004 uses the VMT data compiled by FHWA directly in the
model, differences between MOVES MPG and FHWA MPG on a vehicle category basis
would be traced to the differences in total fuel consumption estimates discussed in
Section 2, and different methods for deriving fuel consumption by vehicle category. The
fleet fuel economy estimates for calendar year 2002 by HPMS category are shown in
Table 4-2. The passenger car, light truck and combination truck categories (which
dominate both gasoline and diesel consumption) differ by five percent or lower, while the
relative differences for buses, single unit trucks and motorcycles differ on the order of 30
to 40 percent.
15
-------
Table 4-2: 2002 U.S. Fleetwide Fuel Economy from
FHWA and MOVES2004 (MPG)
Vehicle Class
Passenger Car
Light Truck
Bus
Single Unit Truck
Combination Truck
Motorcycles
FHWA
22.0
17.4
6.8
7.4
5.2
50.0
MOVES
22.8
16.6
9.7
9.8
5.3
30.5
% diff
4%
-5%
43%
33%
1%
-39%
4.2 Model Year MPG vs. EPA Fuel Economy Trends
Another way to gauge MOVES fuel economy results is to compare by-model year
MPG from MOVES with the sales-weighted model year averages reported in the Fuel
Economy Trends report published yearly by EPA.10 The Trends report analyzes official
fuel economy data generated for the process of determining compliance with Corporate
Average Fuel Economy (CAFE) requirements and the EPA fuel economy labeling
program. The report includes an estimate of harmonically averaged fuel economy for
each model year from 1975 through 2004, weighted by sales of each vehicle line. For
this analysis, we used the combined city/highway values reported in Table 1 of the
Trends report. We used the "adjusted" values from this table, which reflect the
downward adjustment (roughly 15 percent) applied to the raw measured values in order
to better estimate "real world" fuel economy.
To generate by-model year MPG estimates for MOVES, we did a national / annual
run for calendar year 2004 in which we specified output reporting at the model year level
(as noted in Section 2, more disaggregate runs will result in fuel economy results which
are a few percent lower). This reports total energy and distance (VMT) for each model
year in the 30 year window prior to and including the analysis year. MOVES does not
account for deterioration in energy consumption due to vehicle age, hence it is assumed
that MPG does not change with age. We then converted energy consumption to fuel
consumption using the methods discussed in Section 2, and calculated MPG as VMT
divided by fuel consumption for each model year.
Since the Trends report only addresses light-duty cars and trucks, we limited the
MOVES estimates to the passenger car and passenger truck use types. The passenger car
use types should map directly to the light-duty vehicle class used in the Trends report.
This is not the case for trucks, however. Trucks included in the Trends report have an
upper weight cutoff of 8,500 Ibs gross vehicle weight (GVW), which is the cutoff for the
CAFE regulations, whereas MOVES includes trucks heavier than 8,500 Ibs GVW in the
16
-------
passenger truck category. Vehicle weight classes in MOVES are based on loaded weight,
not GVW, and the distribution of loaded vehicle weight is based on information from the
U.S. Census Bureau's Vehicle Inventory and Use Survey (VIUS 1997).11 Based on
VIUS, MOVES estimates that about 6 percent of passenger trucks have a loaded weight
above 6000 Ibs (up to 14,000 Ibs), many of which would have a GVW rating higher than
8,500 Ibs. This means that passenger trucks exceeding 8500 Ibs GVW, such as the Ford
Expedition or the GM Hummer, are included in the MOVES estimates but not the Trends
estimates.
The by-model year comparisons for model years 1975 through 2004 are shown for
cars and trucks in Figures 4-1 and 4-2.
Figure 4-1: Passenger Car Fuel Economy by Model Year for
Fuel Economy Trends Report and MOVES2004 (MPG)
30.0
25.0
20.0
. ™ ••
7~
Si
'7Z -i R n
&
E •
0
° -in n
1 U.U
. ' /
£^
— MOVES
--•-•FE Trends
0.0
1975
1980
1985
1990
Model Year
1995
2000
2005
17
-------
Figure 4-2: Light Truck Fuel Economy by Model Year for
Fuel Economy Trends Report and MOVES2004 (MPG)
25.0
20.0
5.0
0.0
MOVES
FE Trends
1975
1980
1985
1990
Model Year
1995
2000
2005
For passenger cars, the MOVES estimates track changes in the Trends results well
but are generally lower, with larger differences in earlier model years and closer
agreement in later years. The lower MOVES estimates highlight possible differences
between the real-world methodologies employed in MOVES (e.g. driving patterns
derived from in-use driving surveys, cold temperature effects for starts) and the
methodology used to estimate real-world fuel economy in the Trends estimate, i.e.
applying a downward adjustment to raw results from the Fuel Economy Test Procedure.
Changes to this procedure are currently being considered by EPA in order to better reflect
real world results.
The MOVES passenger truck results are generally also lower than the Trends light-
truck estimates, except for a stretch of years in the early 1990's where the estimates are
higher by a few percent. Large swings in MOVES results in the early 1980's can be
traced to anomalies in truck weight data as derived from VIUS and Oak Ridge National
Lab datasets. The large drop in MOVES fuel economy from 1996 to 1997 can be traced
to a jump in average truck weight, which appears to be brought on by the introduction of
heavier trucks on the market in the 1997 model year - most notably the Ford Expedition
(the reader should consult the report "MOVES2004 Highway Population and Activity
Data" for more detail on the default weight distributions used in the model). The cause
of the higher MOVES estimates relative to Trends in the early 1990's merits further
investigation.
18
-------
5. Conclusions
Our conclusions from this analysis are as follows:
• National fuel consumption estimates derived from MOVES2004 total energy
results show good agreement with top-down estimates from fuel tax records
compiled by FHWA. In 2002, MOVES estimates for gasoline consumption for a
state/month aggregation case were 1 percent lower than FHWA estimates, and
diesel consumption estimates were 3-4 percent lower depending on the treatment
of publicly-owned vehicles.
• State-by-state comparisons of MOVES and FHWA fuel consumption are
generally favorable, particularly for gasoline, with larger variability in diesel
results. Increased variability in state-by-state results is likely a function of a)
cross-border travel, and b) how applicable national defaults are to a particular
state.
• A sensitivity analysis of MOVES energy consumption results versus the level of
geographic and temporal resolution showed a 2-3 percent difference between the
highest level of pre-aggregation (nation/year) and the state/month level, with
intermediate levels (state/year and nation/month) falling in between.
• MOVES CH4 and N2O emission inventory estimates are significantly lower than
inventory estimates compiled by EPA's Office of Air and Radiation: roughly 30
to 60 percent lower for gasoline, over 90 percent for diesel. This difference is
almost entirely due to new emission factors developed for MOVES which
incorporate recent test data.
• A comparison was made to fuel economy estimates for the entire on-road fleet
generated by FHWA, and by-model year estimates from EPA's Fuel Economy
Trends Report. MOVES MPG estimates agree well with the FHWA estimates for
the vehicle categories which dominate fuel consumption (passenger car, light
truck, and combination truck), which is expected based on the total fuel
consumption results. The MOVES by-model year results are generally lower than
the Fuel Economy Trends Report estimates, although agreement is closest in the
most recent model years.
Overall, the comparisons presented in this report are encouraging, particularly the good
agreement between fuel consumption estimates derived from MOVES and the top-down
fuel sales data compiled by FHWA. We believe the analyses presented here are a
responsive first step towards the charge given to EPA by the National Research Council
and other parties to employ more systematic model validation. At the same time, we
recognize that model validation and evaluation is an ongoing process which must expand
into a fuller assessment of sensitivities and uncertainties, and incorporate new data as it
becomes available.
19
-------
Appendix A: Peer Review Comments
Professor Robert Harley of the Department of Civil and Environmental
Engineering of the University of California at Berkeley was contracted to provide formal
peer review on a pre-publication version of this document. His comments are included
verbatim in this Appendix. Responses to substantive (i.e. non-editorial) comments have
been added following each comment, in italics to differentiate it from the original
comments. It is important to keep in mind when reading these comments that
MOVES2004 was revised in the time between the pre-publication and published versions
of this report, to correct errors and update default inputs; as the peer review comments
were made based on draft results, some of the specific comments apply to results which
are no longer in this report.
20
-------
Review of "MOVES2004 Validation Results'
by
Robert Harley
Department of Civil and Environmental Engineering
University of California at Berkeley
October 2004
MAJOR COMMENTS
Ml. The title of the report should be changed to give a more detailed picture of the issues
that are being addressed. For example, based on the title alone, I would have assumed
that the report provided an assessment of CO, NOX, VOC, and PM emission estimates. I
recommend the following revised report title: "Assessment of MOVES Model Estimates
of Fuel Consumption, Fuel Economy, and Greenhouse Gas Emissions".
M2a. Rates of growth for gasoline and diesel differ: on-road diesel fuel use is growing
much faster than gasoline. Therefore, I recommend comparisons of MOVES with
FFEWA national on-road fuel consumption estimates for earlier years such as 1990 as
well as circa 2000 comparisons that are already included in the report. While useful and
informative, the comparisons around the year 2000 don't give a clear picture of whether
the differing long-term rates of growth of gasoline and diesel use are accurately
represented in the MOVES model. Capturing differences in activity growth rates by
sector will be very important for making future year emission projections. Some older
transportation models focus on total traffic and incorrectly assume diesel is a small fixed
fraction of the total, growing at the same rate as the total vehicle miles of travel (VMT),
which is dominated by gasoline vehicles.
M2b. The differences in national fuel consumption shown on page 5 (see Table 2-3) for
1999-2003 are small and all about the same for gasoline, with larger and more variable
differences for diesel (special) fuel. This narrow window of years does not help to assess
whether long-term growth rates are represented accurately within MOVES.
EPA Response to 2a & 2b: The earliest calendar year MOVES2004 can currently
provide estimates for is 1999, so the suggested comparison isn 't currently possible.
M2c. The text on page 2 notes improved fuel tax compliance for diesel fuel resulting in
states reporting higher diesel fuel sales to FFEWA over time. This may contribute to a
faster growth rate for diesel fuel compared to gasoline. Other relevant factors could
include differences in fuel economy trends between light-duty gasoline vehicles and
heavy-duty diesel trucks, different rates of growth in the numbers of light-duty gasoline
vs. heavy-duty diesel vehicles on the road, and different rates of growth in the amount of
21
-------
driving per vehicle per year.
EPA Response: Differences in fuel economy trends between light-duty gasoline vehicles
and heavy-duty diesel truck are addressed in MOVES to the extend they are reflected in
the data used to generate total energy rates. Different rates of growth in the numbers of
light-duty gasoline vs. heavy-duty diesel vehicles on the road are derived from El A 's
National Energy Modeling System (NEMS). Different rates of growth in the amount of
driving per vehicle per year are accounted for in MOVES only indirectly; total VMT and
sales grows at different rates, which implies differences in per-vehicle VMT growth.
M3. Another evaluation of MOVES that should be done is to compare state-level annual
on-road fuel consumption between FHWA and MOVES for the year 2000 (aggregrates
by EPA Region, PAD District, etc. could also be examined). A common misconception
is that state-level fuel comparisons cannot be done for diesel fuel as heavy-duty trucks
can drive -1000 miles between refuelings and therefore often cross state lines. Note
however that current law requires inter-state truckers to file quarterly international fuel
tax agreement (IFTA) returns in their home state only, with fuel excise taxes then
remitted by the home state to other jurisdictions in proportion to where fuel is used rather
than where it is purchased. According to FHWA, all Canadian provinces and all states
except Alaska and Hawaii were participating in the IFTA program as of October 1996.
EPA Response: In response to this comment we 've added a state-by-state comparison of
fuel consumption levels estimated by FHWA and MOVES, in Section 2.3.3.
M4. For diesel fuel, where off-road fuel use is a significant fraction of the total, how will
EPA check for balance between refinery distillate fuel production supplied to the U.S.
market and total on-road + off-road engine activity? Though an accurate separation of
on-road from off-road fuel use may be difficult to achieve, the combined total may still
be well-defined. The validation efforts reported here encompass only on-road fuel use,
but I thought that MOVES includes off-road engine activity and emission estimates as
well as on-road.
EPA Response: MOVES will eventually include off-road, but not at this time. When off-
road has been integrated into MOVES we can go a broader comparison to refinery
distillate production. For now, we are relying on FHWA 's process for separating
highway and non-highway fuel use.
M5. There is a pervasive issue of tabular data being presented with unjustified precision
in the draft report. For example, in the comparison of greenhouse gas emission estimates
(section 3), the cited EPA GHG report presented CH4 and N2O to the nearest Gg, whereas
Tables 3-2 and 3-3 append ".0" to each of those numbers, implying a sudden order of
magnitude improvement in the precision of the estimates. The stated uncertainties in the
EPA greenhouse gas emission inventory report are 7, 9, and 18% respectively for CO2,
CH4, and N2O. The uncertainty corresponds to -80 Tg of CO2 from gasoline engines, so
these numbers should not be reported in Table 3-1 as 1096.3 etc. For CO2 emissions from
gasoline engines, all that can be concluded is that the two emission estimates agree within
22
-------
their associated ranges of uncertainty. I couldn't reproduce the diesel CC>2 numbers from
E&S report due to use in the MOVES2004 report of unpublished data from a contractor,
but uncertainties in excluding off-road diesel fuel use imply at minimum the diesel CC>2
numbers should be rounded to the nearest 1 Tg, not 0.1 Tg. As the comparison of GHG
emissions is central to the present report, more supporting information should be included
on the reasons and data sources for revisions to CH4 and N2O emission factors.
M6. The tone of the concluding paragraph is upbeat in promoting the MOVES model and
its validity. More balanced wording should be used here. For example, the first sentence
of this paragraph could be dropped without detracting from the conclusions. Agreement
between FHWA and MOVES fuel consumption should be characterized as "close", not
"very close". The diesel fuel estimates disagree by more than 10%. I recommend
additional rewording as follows: "... NRC's review of mobile source emissions models
recommended increased attention by EPA to model validation efforts, and the results
reported here are responsive to that charge for overall levels of on-road engine activity
and greenhouse gas emissions."
EPA Response: We have modified these sections in response to this comment, although
the magnitude of difference in the comparison changed since these comments were made
as a results of changes to the model.
23
-------
DETAILED COMMENTS
Dl. On page 1, an introductory paragraph should be added that describes the MOVES
model and its intended uses, to provide context for the reader.
D2. On page 1, citations to recent validation studies of NONROAD and EMFAC models
are missing (note 3).
D3. At the bottom of page 4, there is a reference to Heywood. This needs to be footnoted
and added to the reference list.
D4. Table 2.2 needs numerous revisions. Precision in the final column of the table
(energy content) is to 6 figures, which is excessive. Restate in MJ/L and/or MJ/gallon
units. Units for the second column should be kJ not KJ. I recommend stating fuel density
in specific gravity or kg/L units. Excessive precision of 4 figures is currently specified
for fuel densities. The density difference between conventional and reformulated
gasoline indicated in the table may be too small: reducing aromatic content is expected to
reduce fuel density. In California, we reported that the density of gasoline fell from 0.76
to 0.74 kg/L with the 1996 introduction of Phase 2 RFG (Kean et al., SAE technical
paper no 2002-01-1713, see Table 1), though note this fuel reformulation was more
severe than required in Federal RFG areas outside of California.
Response: the RFG fuel density is based on GREET estimates for Federal RFG. GREET
does account for California RFG separately, but uses the same density.
D5. On page 5 (section 2.3), it would be helpful to state what fraction of total special fuel
is attributed to LPG by the MOVES model. I expect it to be a negligible fraction on the
national scale. This information could be derived from the EPA GHG report, which
includes inventories of mobile source CO2 emissions by fuel type including LPG.
Response: the LPG contribution has been added to the report (approximately 0.05
percent of "specialfuel" volume).
D6. On page 6, the meaning of the last sentence of the first full paragraph is unclear, and
the text is garbled: "but we would the uncertainty in off-road use..."
D7. The stated units of measure in Tables 3-2 and 3-3 are incorrect. Emission estimates
for CFL; and N2O are in Gigagrams (Gg), not Teragrams (Tg). It may help to include
notes that Tera = 1012 and Giga = 109. Percentage differences should be rounded to the
nearest whole value in all tables appearing on this page.
D8. On page 10, Table 4-1, spaces are missing in some of the entries in the 2nd column.
In Table 4-2, the percentage differences should be rounded to the nearest whole number.
The text at the top of page 11 should not given reasons for a 4% difference as those
numbers agree within the range of uncertainty in the estimates.
24
-------
D9. On page 11, a citation for the Vehicle Inventory and Use Survey (VIUS) is missing
from the reference list. The text on page 11 should read "Vehicle Inventory and Use"
rather than "Vehicle In Use".
D10. On page 13, the Figure is incorrectly numbered 4-1; it should be 4-2. For both
figures, the y-axis tick mark labels should be rounded to the nearest 1 mpg rather than 0.1
mpg.
DILI recommend caution in making statements such as that appearing at the bottom of
page 13 ("MOVES results suggest that this adjustment is no longer adequate to reflect
true on-road fuel economy"). MOVES provides estimates of in-use vehicle fuel
economy, not ground-truth data. The 15% downward adjustment applied to fuel economy
measured in the FTP may indeed be inadequate. However, I would want to see direct
measurements of in-use vehicle fuel economy and comparisons to FTP values for the
same vehicles before making strong statements about the need for revisions to in-use fuel
economy adjustments.
Response: We have modified the report in light of these comments.
D12. The first bullet in the conclusions section traces the higher fuel consumption
estimated by MOVES in part to the exclusion of publicly-owned vehicles from the
FHWA special fuel consumption estimates. But when MOVES estimates were adjusted
to exclude refuse trucks and buses (see Table 2-3), the comparison with FHWA still
indicated an 8-14% offset vs. a 10-15% offset before adjustment. Publicly-owned
vehicles therefore do not appear to be a major contributor to the offset, at least assuming
that most of the relevant fuel use is by buses. Other factors may be at work here, and/or
the differences may not be significant relative to uncertainties in the estimates.
Response: The magnitude and direction of this difference has changed since these
comments were made. In general the contribution of publicly owned vehicles can not be
quantified at this time, and further work will be required to understand this.
D13. In the reference list, web links to cited reports should be included where available.
Many of the cited references will otherwise be difficult to locate.
25
-------
References
1 National Research Council. 2000. Modeling Mobile Source Emissions. National Academy Press;
Washington, D.C. URL with link to this book: http://books.nap.edu/catalog/9857.html
2 Environ International Corp., Evaluation of the U.S. EPA MOBILE6 Highway Vehicle Emission Factor
Model, Coordinating Research Council Project E-64 Final Report, March 2004. URL with link to report:
http://www.crcao.com/reports/recentstudies2004/CRC_E-64_Final_032004.pdf
3 Kean, A.J.; Sawyer, R.F.; Harley, R.A. (2000) A Fuel-Based Assessment of Off-Road Engine Diesel
Engine Emissions Journal of the Air & Waste Management Association 50, 1929-1939. URL with a link to
article: http://www.ce.berkeley.edu/~harley/research_3.php
4 Singer, B.C.; Harley, R.A. (2000) A Fuel-Based Inventory of Motor Vehicle Exhaust Emissions in the
Los Angeles Area During Summer 1991 Atmospheric Environment 34, 1783-1795. URL with a link to
article: http://www.ce.berkeley.edu/~harley/research_3.php
5 FHWA Office of Highway Policy Information, Highway Statistics 2002, 2004. URL with link to report:
http://www.fhwa.dot.gov/policy/ohim/hs02/index.htm
6 Heywood, J., Internal Combustion Engine Fundamentals (p 915), McGraw-Hill, 1988
7 U.S. Census 2000 County-To-County Worker Flow Files,
http://www.census.gov/population/www/cen2000/mcdworkerflow.html
8 U.S. EPA Office of Atmospheric Programs, Inventory of U.S. Greenhouse Gas Emissions and Sinks: 1990
- 2002, EPA Report No. EPA430-R-04-003, April 2004. URL with link to report:
http://yosemite.epa.gov/oar/globalwarming.nsf/content/ResourceCenterPublicationsGHGEmissionsUSEmis
sionslnventory2004.html
9 ICF, Update of Methane and Nitrous Oxide Emission Factors for On-Highway Vehicles, 2004
10 U.S. EPA Office of Transportation & Air Quality, Light-Duty Automotive Technology and Fuel
Economy Trends: 1975 through 2004, EPA420-R-04-001, April 2004 . URL with link to report:
http://www.epa.gov/otaq/fetrends.htm
11 U.S. Census Bureau, 1997 Economic Census: Vehicle Inventory and Use Survey, October 1999 URL
Link to report: http://www.census.gov/svsd/www/97vehinv.html
26
------- |