The Effects of Gasoline Sulfur Level
on Emissions from Tier 2 Vehicles
in the In-Use Fleet
Draft
&EPA
United States
Environmental Protection
Agency
-------
The Effects of Gasoline Sulfur Level
on Emissions from Tier 2 Vehicles
in the In-Use Fleet
Draft
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
United States
Environmental Protection
Agency
EPA-420-D-13-003
April 2013
-------
DRAFT DOCUMENT
of
Acronyms 2
1. Executive Summary 3
2. Introduction 7
2.1. Background 7
2.2 Motivation for this Study 10
3. Study Design 10
3.1. Measurement of Reversible In-Use Loading (Clean-Out Effect) 11
3.2 Effect of Sulfur Level 12
4. Test Vehicle Selection, Recruitment, and Delivery 12
4.1 Choice of Makes and Models 12
4.2 Vehicle Recruitment Criteria 13
4.3 Initial Checks and Test Vehicle Delivery 14
5. TestFuel Specs and Procurement 15
6. Test Procedures 16
6.1 Initial fuel exchange and vehicle prep 16
6.2 Test procedure description 17
7. Data Analysis and Results 22
7.1. Data Preparation 23
7.1.1. Imputation of measurements with low concentration 23
7.1.2. Detection of outliers 26
7.2. Modeling Methodology 27
7.3. Statistical Analysis and Results 29
7.3.1. Effect of clean-out at 28 ppm 29
7.3.2. Effect of clean-out at 5 ppm 34
7.3.3. Effect of Sulfur level 38
7.3.4. Sensitivity Analysis 53
8. Summary and Conclusions 56
9. References 59
Appendix A. Details of On-Road Mileage Accumulation Route 62
Appendix B. Emission Concentration Plots 64
Appendix C. Additional Information on Emission Measurements 67
Appendix D. Discussion of Univariate and Multivariate Analysis of Variance 68
Appendix E. Plots of All Pollutants and Bags 69
-------
DRAFT DOCUMENT
A/F Ratio
BIC
CH4
CO
CRC
CS
EPA
F
FID
FTP
HC
LOQ
MIL
ML
MSAT
NMHC
NMOG
NOX
NVFEL
PGM
PM
PPB
PPM
PSI
PZEV
REML
RLD
RVP
SAS
THC
UN
Air-to-Fuel Ratio
Schwarz Bayesian Criterion
Methane
Carbon Monoxide
Coordinating Research Council
Compound Symmetry Covariance
Environmental Protection Agency
Fahrenheit
Flame lonization Detector
Federal Test Procedure
Hydrocarbons
Limit of Quantification
Malfunction Indicator Lamp
Maximum Likelihood
Mobile Source Air Toxics
Non-Methane Hydrocarbons
Non-Methane Organic Gases
Oxides of Nitrogen
National Vehicle and Fuel Emission Laboratory
Platinum Group Metals
Particulate Matter
Parts per Billion
Parts per Million
Pounds per Square Inch
Partial Zero-Emissions Vehicle
Restricted Maximum Likelihood
Restricted Likelihood Distance
Reid Vapor Pressure
Statistical Analysis Systems
Total Hydrocarbons
Unstructured Covariance
-------
DRAFT DOCUMENT
1.
Fuel sulfur content has long been understood to affect the performance of emission
aftertreatment catalysts in light duty vehicles, where the sulfur and its oxides occupy active
precious metal sites and oxygen storage materials, reducing the catalyst's efficiency in
destroying pollutants. Numerous studies have shown the direct impact of fuel sulfur levels
above 30 ppm on emissions, data that formed the basis of the sulfur controls in EPA's Tier 2
rulemaking for light duty vehicle emissions, published in 2000.1
Following the successful implementation of the Tier 2 sulfur standards, new research has
focused on the emission reduction potential of lowering sulfur levels below 30 ppm, particularly
on Tier 2 and newer technology vehicles, under the hypothesis that increased reliance on the
catalytic converter will result in a higher sensitivity to fuel sulfur content. A 2005 study
conducted jointly by EPA and several automakers found large decreases in NOx and HC
emissions from vehicles meeting Tier 2 Bin 5 emission levels when operating on 6 ppm versus
r\
32 ppm sulfur test fuel. In order to gain further understanding of how these emission reductions
would translate into the in-use fleet, EPA conducted the study described here to assess the state
of sulfur loading (poisoning) in typical in-use vehicles, as well as the effect of fuel sulfur level
on these vehicles during subsequent mileage accumulation. It was designed to take into
consideration what was known from prior studies on sulfur build-up in catalysts over time and
the effect of periodic regeneration events that can occur during higher speed and load operation
in day-to-day driving.
The study sample described in this analysis consisted of 81 cars and light trucks recruited
from owners in southeast Michigan, covering model years 2007-9 with approximately 20,000-
40,000 odometer miles. The makes and models targeted for recruitment were chosen to be
representative of high sales vehicles covering a range of types and sizes. Test fuels were two
non-ethanol gasolines with properties typical of certification fuel, one at a sulfur level of 5 ppm
and the other at 28 ppm, the higher level chosen to be representative of retail fuel available to the
public in the vehicle recruiting area. All emissions data was collected using the FTP cycle at a
nominal ambient temperature of 75°F.
-------
DRAFT DOCUMENT
Using the 28 ppm test fuel, emissions data were collected from vehicles in their as-
received state, and then following a high-speed/load "clean-out" procedure consisting of two
back-to-back US06 cycles intended to reduce sulfur loading in the catalyst. A statistical analysis
of this data showed highly significant reductions in several pollutants including NOX and
hydrocarbons (Table ES-1), suggesting that reversible sulfur loading exists in the in-use fleet and
has a measurable effect on aftertreatment performance. For example, Bag 2 NOX emissions
dropped 32 percent between the pre- and post-cleanout tests on 28 ppm fuel.
Table ES-1 Average clean-out effect on in-use emissions using 28 ppm test fuel
Bagl
Bag 2
Bag3
FTP Composite
Bag 1 - Bag 3
NOX
(p-value)
-
31.9%
(0.0009)
38.3%
(O.OOOl)
11.4%
(O.OOOl)
-
THC
(p-value)
-
16.5%
(0.0024)
21.4%
(O.OOOl)
4.1%
(0.0187)
-
CO
(p-value)
4.7%
(0.0737)
-
19.5%
(0.0011)
7.6%
(0.0008)
4.2%
(0.0714)
NMHC
(p-value)
-
17.8%
(0.0181)
27.8%
(O.OOOl)
3.0%
(0.0751)
-
CH4
(p-value)
-
15.3%
(0.0015)
12.0%
(O.OOOl)
6.9%
(0.0003)
-
PM
(p-value)
15.4%
(< 0.0001)
-
24.5%
(O.OOOl)
13.7%
(O.OOOl)
-
The clean-out effect is not significant at a = 0.10 when no reduction estimate is provided.
To assess the impact of lower sulfur fuel on in-use emissions, a representative subset of
vehicles was kept to conduct testing on 28 ppm and 5 ppm fuels with accumulated mileage. A
first step in this portion of the study was to assess differences in the effectiveness of the clean-
out procedure when done using different fuel sulfur levels. Table ES-2 presents a comparison of
emissions immediately following (<50 miles) the clean-out procedures at the low vs. high sulfur
level. These results show significant emission reductions for the 5 ppm fuel relative to the 28
ppm fuel immediately after this clean-out; for example, Bag 2 NOX emissions were 47 percent
lower after a clean-out on the 5 ppm fuel vs. following the clean-out on the 28 ppm fuel. This
indicates that either the catalyst is not fully desulfurized after a clean out procedure as long as
there is sulfur in the fuel, or that there is an instantaneous effect of sulfur concentration in the
exhaust gas on the catalyst's activity.
-------
DRAFT DOCUMENT
Table ES-2 Percent reduction in emissions from 28 ppm to 5 ppm
for the first three repeat FTP tests immediately following clean-out
Bagl
Bag 2
Bag3
FTP
Composite
Bag 1 - Bag 3
NOX
(p-value)
5.9%
(0.0896)
47.3%
(0.0010)
51.2%
(O.OOOl)
17.7%
(0.0001)
-
THC
(p-value)
5.4%
(0.0118)
40.2%
(O.OOOl)
35.0%
(O.OOOl)
11.2%
(O.OOOl)
-
CO
(p-value)
7.3%
(0.0023)
-
10.1%
(0.0988)
8.3%
(0.0003)
5.8%
(0.0412)
NMHC
(p-value)
4.6%
(0.0465)
34.4%
(0.0041)
45.0%
(O.OOOl)
8.8%
(0.0003)
-
CH4
(p-value)
11.1%
(O.OOOl)
53.6%
(O.OOOl)
25.4%
(O.OOOl)
21.4%
(O.OOOl)
-
PM
-
-
-
-
-
The effectiveness of clean-out cycle is not significant at a = 0.10 when no reduction estimate is provided.
To assess the overall in-use reduction between high and low sulfur fuel, a mixed model
analysis of all data as a function of fuel sulfur level and miles driven after cleanout was
performed. This analysis found highly significant reductions for several pollutants, as shown in
Table ES-3; the reductions for Bag 2 NOx were particularly high, estimated at 59 percent
between 28 ppm and 5 ppm. For most pollutants, the analysis suggested the effect of sulfur level
didn't depend on miles driven after the fuel change, and therefore the emission benefit of lower
fuel sulfur occurred immediately and continued as miles were accumulated. Some results, such
as Bag 1 hydrocarbons, did show a significant miles-by-sulfur interaction. In this case,
determination of a sulfur level effect for the in-use fleet required estimation of a miles-equivalent
level of sulfur loading, which can be gleaned from the cleanout results obtained from the
baseline testing on the vehicles as-received.
-------
DRAFT DOCUMENT
Table ES-3 Summary of mixed model results for emission reductions from 28 to 5 ppm
sulfur, adjusted for in-use sulfur loading (mileage accumulation) where appropriate
Bagl
Bag 2
Bag3
FTP
Composite
Bag 1 - Bag 3
NOX
(p-value)
10.7%
(0.0033)
59.2%
(< 0.0001)
62.1%
(< 0.0001)
23.0%f
(0.0180)
-
THC
(p-value)
8.5%T
(0.0382)
48.8%
(< 0.0001)
40.2%
(< 0.0001)
13.0%f
(0.0027)
5.2%
(0.0063)
CO
(p-value)
7.5%T
(0.0552)
-
20.1%
(< 0.0001)
11.9%f
(0.0378)
4.3%
(0.0689)
NMHC
(p-value)
7.5%
(< 0.0001)
44.8%T
(0.0260)
49.9%
(< 0.0001)
10.6%T
(0.0032)
5.1%
(0.0107)
CH4
(p-value)
13.9%T
(< 0.0001)
49.9%
(< 0.0001)
29.2%
(< 0.0001)
25.8%f
(< 0.0001)
4.6%
(0.0514)
NOX+NMOG
(p-value)
N/A
N/A
N/A
17.3%
(0.0140)
N/A
PM
-
-
-
-
-
^Model with significant sulfur and mileage interaction term. The effect of fuel sulfur level is not significant at a =
0.10 when no reduction estimate is provided. For THC bag 1 and CH4 bag 1, because the effect of clean-out was
not statistically significant, the reduction estimates are based on the estimates of least squares means.
Major findings from this study include:
• Reversible sulfur poisoning is occurring in the in-use fleet of Tier 2 vehicles and has a
measureable effect on emissions of NOX, hydrocarbons, and other pollutants of interest.
• The effectiveness of high speed/load procedures in restoring catalyst efficiency is limited
when operating on higher sulfur fuel.
• Reducing fuel sulfur levels from 28 to 5 ppm produces significant reductions in
emissions of NOX, hydrocarbons, and other pollutants of interest from a broad range of
in-use Tier 2 vehicles.
• Bag 2 NOx effects, which are relatively large but based on low emission levels, were
found to be robust to sensitivity analyses related to influential vehicles and measurement
uncertainty at low emission levels.
-------
DRAFT DOCUMENT
Sulfur in gasoline has long been found to reduce the conversion efficiency of automotive
three-way catalysts, with some studies suggesting an increase in catalyst sensitivity (in terms of
percent conversion efficiency) to sulfur as tailpipe emission levels decrease.3 The Tier 2 light
duty emission standards recognized the importance of sulfur and required lower gasoline sulfur
levels in accordance with the relative level of the new emission standards. Since that time, little
work has been published on the effect of further reductions of fuel sulfur level, especially on in-
use vehicles that may have some degree of catalyst deterioration due to real world operation.
This study investigates the effects of lower sulfur gasoline on in-use Tier 2 vehicles that have
been operated on commercial gasoline (with relatively higher sulfur levels) under real world
driving conditions.
Aftertreatment systems utilize catalysts containing precious metals and metal oxides to
selectively oxidize hydrocarbons and reduce nitrogen oxides in the exhaust gases. Sulfur oxides
from fuel combustion preferentially bind to active sites in the catalyst, inhibiting their ability to
participate in the intended conversion reactions (sulfur contamination or poisoning). The amount
of sulfur retained by the catalyst is a function of the type and arrangement of active materials and
coatings used within the catalyst, its operating temperature, as well as the concentration of sulfur
oxides and the air-to-fuel ratio of the exhaust gas.4'5 Modern vehicle engines operate with rich-
lean oscillations that maintain the proper oxidation-reduction condition of the catalyst, but under
typical driving conditions there is a nonzero equilibrium level of sulfur retained. Regularly
operating the catalyst at a high temperature under net reducing conditions can release much the
sulfur oxides from the catalyst (Figure 2-1), and could minimize the effects of fuel sulfur on
catalyst efficiency. However, producing these conditions at sustained and/or regular intervals
would accelerate thermal degradation of the catalyst and may also raise other challenges for
emissions control and fuel economy.
-------
DRAFT DOCUMENT
Figure 2-1 Catalyst-sulfur interaction by temperature and A/F ratio (Adapted from
information from Ford Motor Company)
Sulfate reduction &
release asS02, H2S
S02 adsorption &
reduction to elemental S
(reversible poisoning)
900°C
Sulfur oxidation Ssulfate
decomposition
SOOT
S02 release & conversion
toS03, H2S04
S02 adsorption &
oxidation to S03, S04
Additionally, it is not always possible to maintain these catalyst temperatures (e.g., cold
weather, idles), and the rich air-to-fuel ratios necessary can result in increased PM, NMOG and
CO emissions, therefore reducing fuel sulfur levels has been the primary regulatory mechanism
to minimize sulfur contamination of the catalyst and ensure optimum emissions performance
over the useful life of a vehicle.
In 2005, EPA and several automakers jointly conducted a program that examined the
effects of sulfur, benzene, and volatility on gaseous emissions from a fleet of nine Tier 2
compliant vehicles (referred to as the Mobile Source Air Toxics or MSAT study6). The fuel
matrix consisted of four non-ethanol fuels blended in a stepwise manner, starting with a
purchased base fuel containing the lowest levels of the three properties of interest. The base fuel
-------
DRAFT DOCUMENT
contained 6 ppm sulfur, and after additions of butane (to arrive at 9 psi RVP) and benzene (to
arrive at 1.0 vol. %), sulfur was increased to 32 ppm using a small amount of doping agent
containing a sulfur compound. Thus, the sulfur effect could be deduced by comparing emission
results between the final fuel and the one just before it in the blending sequence. The test
vehicles were broken-in production samples owned by the manufacturers, which had emission
catalysts and oxygen sensors bench-aged to the equivalent of 120,000 miles. The dataset
consisted of tests performed at four different labs, with three manufacturers testing two of their
own vehicles, and EPA testing the remaining three. Tests conducted on high-sulfur fuel were
preceded by a low-speed cruise to simulate sulfur loading, and tests on low-sulfur fuels were
preceded by a high-speed/load cycle used in previous programs performed by the Coordinating
Research Council.?
Mixed model analyses were performed on the data using vehicle and lab as random
effects and fuel properties as fixed effects. Reductions in FTP-weighted emissions for 6
compared to 32 ppm sulfur were 33 percent for NOx, 11 percent for THC, 17 percent for CO,
and 32 percent for methane (all statistically significant at a = 0.10). These data suggested that
there were likely to be significant emission reductions possible with further reductions in
gasoline sulfur level.
A study published in 2011 by Umicore Autocat USA describes tests performed on a
model year 2009 Chevrolet Malibu calibrated to PZEV levels (20 mg/mi NOx), intended to
examine the hypothesis that very-low emitting vehicles are more sensitive to fuel sulfur content.
The study used two test fuels, at 3 and 33 ppm sulfur, combined with FTP, US06, and a high
speed/load clean-out cycle similar to one in the MSAT study described above, to examine the
impact of sulfur on the underfloor catalyst efficiency during repeated FTP tests. The authors
observed consistent degradation of catalyst performance when repeat FTPs were performed
using the 33 ppm fuel, versus none with the 3 ppm test fuel. Emission measurements showed a
NOx reduction of 41 percent between averages of three replicate FTPs on low versus high sulfur
level. 8
-------
DRAFT DOCUMENT
The MSAT study described in the previous section motivated the design of a new study
to further investigate the effects of lower sulfur gasoline on a larger number of vehicles.
Specifically, we wanted to assess what the benefits of sulfur control might be under in-use
driving conditions for typical modern vehicles, as previous studies have generally not provided
data that can be utilized for this type of analysis in a straightforward manner. Many studies have
designs that look for a change in emissions at a single point in time shortly after a fuel sulfur
change and/or a clean-out cycle, while in fact catalysts of in-use vehicles are normally operating
at a point somewhere between being clean of sulfur (e.g., following high speed, high load and
therefore high catalyst temperature operation) and fully loaded with sulfur (e.g., following low
speed, low load, cool catalyst temperature operation). Others have measured effects of aging
catalysts on different fuel sulfur levels, which do not give any information on what happens
when fuel sulfur is changed partway through a vehicle's life. Thus, the design of this study
sought to assess the impact of reversible sulfur loading on in-use Tier 2 vehicles recruited from
their owners, as well as the emission performance after a change to lower sulfur fuel and
subsequent mileage accumulation.
This section outlines the hypotheses and theoretical bases for the specific procedures that
follow in the testing section.
In order to characterize the effect of sulfur on the in-use fleet, vehicles were recruited
from their owners and tested in EPA's National Vehicle and Fuel Emission Laboratory (NVFEL)
in Ann Arbor, Michigan. Given this arrangement, it was not feasible to directly measure sulfur
loading in the catalysts themselves by means that involved damaging or destroying them.
Instead, the behavior of emissions during various test procedures was used as a proxy for sulfur
loading.
10
-------
DRAFT DOCUMENT
The program design used two fuels differing only in sulfur level, with the higher level of
30 ppm chosen to match the current average sulfur limit promulgated along with the Tier 2
vehicle standards. All other fuel properties were controlled to be representative of conventional
commercial gasoline (see Table 5-1).'
The level of reversible in-use sulfur storage (or loading) and release within an exhaust
catalyst system can be assessed by measuring emissions from the vehicle as received, performing
a high-speed, high-load clean-out cycle, then measuring emissions again. This change in
emissions represents the amount of reversible deactivation relative to a "clean" baseline state.
Prior to the start of this test program, we consulted previous studies as well as vehicle calibration
and emission controls experts to select an appropriate clean-out cycle for this test program. An
example of such a cycle is presented in Appendix C of the CRC E-60 emissions study report.9
The US06 certification cycle, while not as extreme as the E-60 cycle, does provide similar
conditions that are favorable for desulfurization (rich-lean switching at high temperatures) and is
commonly used in the auto industry to desulfurize catalysts prior to emissions testing. Though
the E-60 cycle would be preferred, there were concerns that it would cause undesirable wear and
tear on recruited vehicles. As a compromise, two back-to-back US06 cycles were selected for
use in this test program.
The level of sulfur loading in a vehicle catalyst is primarily a function of the fuel sulfur
level and catalyst temperature, the latter of which is a function of vehicle and catalyst design and
driver behavior. A vehicle with a relatively high exhaust temperature at the catalyst location,
and/or significant excess loading of certain platinum group metals (PGM) and other active
materials in the catalyst may be relatively insensitive to sulfur loading regardless of driver
behavior. These effects of vehicle design were accounted for in the study by recruiting several
1 At the time this study was designed, a non-oxygenated gasoline blend was selected as being representative of in-
use fuel as ethanol blends were not as ubiquitous as they are today. However, since sulfur's effect on (non-sulfur)
emissions is understood to occur solely in the catalyst, we do not anticipate meaningful interaction with ethanol.
Moreover, since sulfur was varied independently of other fuel properties, we do not anticipate that the relative
difference in emission results would have been adversely affected by this design choice.
11
-------
DRAFT DOCUMENT
different makes and models covering a range of engine and vehicle sizes. We attempted to
account for the effects of the owner's behavior by testing several samples of the same make and
model.
Following vehicle operation that raises catalyst temperature and removes sulfur,
accumulation immediately resumes toward a higher equilibrium level as temperature declines
under more typical (milder) driving conditions. This loading continues over a period of time
with vehicle operation, and can be observed as an increase in emissions (sometimes referred to as
"NOx creep"). To capture this effect, repeated emission tests were performed following a
cleanout at two different fuel sulfur levels. Given that the exhaust stream contains different
concentrations of sulfur according to the fuel sulfur level, the rate of reloading is expected to be
lower for the lower sulfur fuel. Additionally, if an equilibrium loading (stable level of
emissions) were to be reached after many miles of driving, this procedure investigates whether
this equilibrium loading is also lower for the lower sulfur fuel.
The test fleet was chosen to be representative of latest-technology light duty vehicles
being sold at the time the program was launched. The study did not attempt to analyze or model
details of aftertreatment design specific to each vehicle model such as catalyst position, precious
metal types and quantities used, or related engine control strategies such as timing advance at
cold start or fuel cut during deceleration. While these things undoubtedly influence the behavior
of emissions and may interact with the fuel sulfur effects being investigated, including them in
an analysis requires correctly assessing and parameterizing them for all vehicles in the study.
Instead this program's aim was to characterize overall effects on emission inventories by
observing the aggregate behavior of a representative fleet of vehicles.
12
-------
DRAFT DOCUMENT
In terms of emissions standards, the test fleet was to conform on average to Tier 2 Bin 5
exhaust levels and employ a variety of emission control technologies. These goals were
achieved by including a range of vehicle sizes, engine displacements, and manufacturers. Engine
family sales data obtained from EPA certification and Wards databases were analyzed to
generate a list of high-sales vehicles as candidates for inclusion. Grouping sales data by engine
family allowed additional transparency and flexibility in choosing test vehicles that represent a
wider group with identical powertrains without targeting one specific make and model. The
resulting target list of 19 vehicle models for recruitment is shown in Table 4-1.
Table 4-1 Test Vehicles Recruited
Make
GM
GM
GM
GM
Toyota
Toyota
Toyota
Toyota
Ford
Ford
Ford
Ford
Chrysler
Chrysler
Chrysler
Honda
Honda
Honda
Nissan
Brand
Chevrolet
Chevrolet
Saturn
Chevrolet
Toyota
Toyota
Toyota
Toyota
Ford
Ford
Ford
Ford
Dodge
Dodge
Jeep
Honda
Honda
Honda
Nissan
Model
Cobalt
Impala FFV
Outlook
Silverado FFV
Corolla
Camry
Sienna
Tundra
Focus
Taurus
Explorer
F150 FFV
Caliber
Caravan FFV
Liberty
Civic
Accord
Odyssey
Altima
Program
ID
CCOB
CIMP
SOUT
CSIL
TCOR
TCAM
TSIE
TTUN
FFOC
FTAU
FEXP
F150
DCAL
DCAR
JLffi
HCIV
HACC
HODY
NALT
Engine
Size
2.2L 14
3.5LV6
3.6LV6
5.3L V8
1.8LI4
2.4L 14
3.5LV6
4.0L V6
2.0L 14
3.5LV6
4.0L V6
5.4L V8
2.4L 14
3.3LV6
3.7LV6
1.8LI4
2.4L 14
3.5LV6
2.5L 14
Engine Family
8GMXV02.4025
8GMXV03.9052
8GMXT03.6151
8GMXT05.3373
8TYXV01.8BEA
8TYXV02.4BEA
8TYXT03.5BEM
8TYXT04.0AES
8FMXV02.0VD4
8FMXV03.5VEP
8FMXT04.03DB
8FMXT05.44HF
8CRXB02.4MEO
8CRXT03.3NEP
8CPJCT03.7NEO
8HNXV01.8LKR
8HNXV02.4TKR
8HNXT03.54KR
8NSXV02.5G5A
Tier 2
Cert Bin
5
5
5
5
5
5
5
5
4
5
4
8
5
8
5
5
5
5
5
The aforementioned 19 makes/models selected for recruitment were targeted to have a
mileage between 12,000 and 40,000 miles and an age of less than three years old. The
recruitment process used the State of Michigan vehicle owner registration database to identify
candidate vehicle owners within 50 miles of NVFEL. Solicitations were sent to owners of
13
-------
DRAFT DOCUMENT
candidate vehicles to find willing participants who would allow their vehicles to be held and
tested for up to six weeks, in exchange for cash and/or leaner vehicle. If an owner agreed to
participate, the vehicle was scheduled for testing.
Vehicles were driven to the test facility either by their owners or NVFEL personnel sent
to retrieve them. Drivers were instructed to avoid hard accelerations and high speeds in an effort
to preserve the "as-received" state of the catalyst. Once in custody, vehicles were subjected to
visual inspection for obvious damage or malfunction. The condition of the MIL (malfunction
indicator, or "check engine", lamp) was also noted, and if illuminated, the vehicle was not
accepted for testing.
Upon receipt of the vehicle at NVFEL, the existing fuel was drained and a small sample
was analyzed for its sulfur content to screen for vehicles that had been operating on unusually
high or low sulfur levels. Federal fuel sulfur standards applicable in the recruiting area specify a
30 ppm annual refinery average with an 80 ppm per-gallon cap. Figure 4-1 shows the
distribution of fuel sulfur levels found in the tanks of test vehicles "as received". The mean of
the fuel samples for which data were available was 25 ppm with standard deviation of 11.0. Fuel
in the recruiting area typically contains 10% ethanol.
All vehicles were inspected for minor fuel and exhaust leaks, and were repaired, as
needed. However, damage to the catalyst, muffler, or sections of exhaust pipe disqualified the
vehicle from inclusion in the program. Fuel leaks were not repaired unless they presented a
safety issue. All repairs were documented. Gas caps and other fuel systems components were
visually inspected.
14
-------
DRAFT DOCUMENT
Figure 4-1 Fuel sulfur levels for test vehicles as received
<10 10-20 20-30 30-40 40-50
Fuel Sulfur(ppm)
50-60
>60
5. Test Fuel Specs and Procurement
A non-ethanol, low-sulfur, certification-grade gasoline was purchased in bulk from a
commercial fuel blender (Haltermann Solutions, Houston, TX) and split into two underground
fuel tanks prior to start of the program. One tank was spiked with a small amount (on the order
of a liter of volume) of dibutyl disulfide to increase the sulfur content from the as-received 5
ppm to 28 ppm. Due to the small volume of sulfur agent added, changes in other fuel properties
were assumed to be negligible. Other properties of the test fuels are listed in Table 5-1, and fall
in the range of what is encountered in typical commercial gasoline in the vehicle recruiting area.
Relative to national average 2010 summer conventional gasoline (much of which contained
ethanol) as reported in the Auto Alliance U.S fuel survey database, these test fuels are somewhat
higher in aromatics (by about 6 vol. %), somewhat lower in olefins (by about 6 vol. %),
somewhat higher in T50 (by about 20°F), and slightly lower in T90 (by about 3°F).
15
-------
DRAFT DOCUMENT
Table 5-1 Test Fuel Properties
Fuel Property
Sulfur
Benzene
Total Aromatics
Olefins
Saturates
Oxygenates
T50
T90
RVP
ASTM Method
D2622
D5769
D5769
D1319
D1319
D5599
D86
D86
D5191
Low S Test Fuel
5 ppm
0.34 Vol. %
31.2 Vol. %
0.5 Vol. %
68.3 Vol. %
0.0 Vol. %
22 FF
317°F
9.0 psi
High S Test Fuelf
28 ppm
0.34 Vol. %
3 1.2 Vol. %
0.5 Vol. %
68.3 Vol. %
0.0 Vol. %
22 FF
317°F
9.0 psi
^Sulfur content was confirmed for the higher-sulfur test fuel, while other properties were assumed to be the same as
the base fuel given the small amount of dopant added.
Fuel drains were generally done via the fuel rail by energizing the vehicle's pump. Next,
the key was turned to the "on" position, without starting the vehicle, for 30 seconds before
keying off. This allowed the vehicle to register the new fuel level to trigger any fast fuel trim
learning procedures the vehicle may have.11 Next, the vehicle was filled to 50 percent capacity
with test fuel. This fuel drain and fill process was then repeated a second time to ensure that the
fuel system had been flushed fully prior to testing.
Following the fuel change the vehicle was moved to the dynamometer to conduct an LA4
preconditioning drive cycle.111 At the completion of this prep cycle, the vehicle was idled in park
11 Fuel trim refers to the ability of the engine control system to adapt to a change in fuel properties, which may affect
vehicle or emissions performance. Some vehicles may accelerate this "learning" process if a fuel change is detected.
111 The LA4 drive cycle covers the first 7.5 miles of the FTP test cycle, consisting of typical urban driving behavior.
For more details on these test cycles see 40 CFR Part 86.
16
-------
DRAFT DOCUMENT
for two minutes before engine shutdown, a step intended to ensure fuel trim data was updated for
idle operation. The vehicle was then moved to a storage ("soak") area using an electric towing
device for a 12-36 hour holding period at approximately 70°F prior to testing.
Vehicles recruited for testing in this program fell into one of two test groups: "Long Test
Procedure" vehicles or "Short Test Procedure" vehicles. For a given vehicle model class (or
family ID), one of the five vehicles was tested under the Long procedure while the other four
were tested under the Short procedure. The vehicle undergoing the Long procedure was selected
at random from willing participants in a given vehicle class. The Long and Short procedures are
shown in Figure 6-1 and are discussed in greater detail below. They are identical in structure for
the first six emission tests.
17
-------
DRAFT DOCUMENT
Figure 6-1 Original Short (S) and Long (L) Procedure flow chart
Fuel change to high-sulfur (28 ppm) test fuel
Pre-clean-out (as-received) replicate
FTPs at 28 ppm S
Sulfur clean-out cycle, 2 x US06 at 28 ppm S
Post-clean-out replicate FTPs at 28 ppm S
Add'l replicates for long-procedure vehicle at 28 ppm
S, alternating cold start FTPs for data collection
(numbered circles) with unsampled hot start FTPs
used for mileage accumulation (blank circles)
Fuel change to low-sulfur (5 ppm) test fuel
Sulfur clean-out cycle, 2 x US06 at 5 ppm S
Data collection at 5 ppm S, alternating cold and hot
start FTPs as above
J
uj
o:
o:
Q.
o:
S
to
UJ
to
S
S
O
o
1
Data used for "clean-out effect"
J
Data used for "sulfur level effect"
18
-------
DRAFT DOCUMENT
All vehicles were tested over three cold-start FTPs with high sulfur fuel to assess "as
received" emissions levels prior to performing a sulfur clean-out cycle (two back-to-back US06
cycles) to remove sulfur accumulated in-use from the catalyst.1V Emissions were then sampled
again over three cold-start FTPs on the same high sulfur test fuel to assess the effect of the sulfur
clean-out cycle on emissions. This data, collected on all vehicles, was used to determine the
"clean-out" effect and is referred to as the clean-out dataset in the analysis section of this report.
Testing of a short procedure vehicle was complete at this point and the vehicle was returned to
its owner.
Long procedure vehicles continued testing for an additional length of time to determine
the long term effects of both high and low sulfur fuel on emissions. Beginning with the fourth
post-cleanout FTP test on high sulfur fuel, the test sequence began to alternate between sampled,
cold-start FTPs and un-sampled hot-start FTPs (immediately following the cold-start test) in
order to accumulate additional miles between emission tests. Following the sixth sampled FTP,
the fuel was changed to low sulfur test fuel and a clean-out cycle was run to remove accumulated
sulfur from the catalyst and establish a new "cleaned-out" emissions baseline on low sulfur test
fuel. Six additional FTPs (with un-sampled FTPs in-between) were conducted on low sulfur
fuel. Together with the six high sulfur fuel tests, this data is used to determine the "sulfur level
effect," which is the assessment of high and low sulfur fuel use starting from an "unloaded"
catalyst.
Vehicles tested on the Long procedure were run for approximately 100 miles of operation
on each fuel following a cleanout. Mid-way through the test program, however, we were
concerned that 100 miles on each fuel may not be a sufficient amount of time to fully re-load the
catalyst to a level that is representative of relatively mild in-use driving. Therefore, we modified
the Long procedure to incorporate several 50-mile on-road mileage accumulation intervals in-
between emission tests on each fuel. This modified Long procedure, called the L/M procedure,
allowed us to accumulate an additional 100 miles on each fuel without increasing the number of
1V A cold-start FTP refers to the FTP drive cycle being performed after the vehicle has been sitting with no engine
starts for between 12-36 hours at approximately 70°F. This is useful because emissions are generally highest in the
first few minutes of cold engine operation. A hot-start FTP refers to a test driven within a short time after a previous
test, such that the engine and aftertreatment are still well above room temperature. Hot-start tests were only used for
mileage accumulation and did not have emissions sampled.
19
-------
DRAFT DOCUMENT
sampled emission tests. A flowchart of the modified Long procedure (L/M) is shown in Figure
6-2.
Figure 6-2 Modified Long Procedure (L/M) flowchart
Fuel change to high-sulfur (28 ppm) test fuel
Pre-clean-out ("as-recieved") replicate FTPs
at 28 ppm S
Sulfur clean-out cycle, 2 x US06 at 28 ppm S
Post-clean-out replicates at 28 ppm S
-50 Miles on-road mileage accumulation
Add'l replicates of cold start FTPs at 28 ppm S
-50 Miles on-road mileage accumulation
Add'l replicates of cold start FTPs at 28 ppm S
Fuel change to low-sulfur (5 ppm) test fuel
Sulfur clean-out cycle, 2 x US06 at 5 ppm S
Low sulfur replicates of cold start FTPs at 5 ppm S
-50 Miles on-road mileage accumulation
Low sulfur replicates of cold start FTPs at 5 ppm S
-50 Miles on-road mileage accumulation
Low sulfur replicates of cold start FTPs at 5 ppm S
§
o:
O
to
uj
I
8
I
o
1
Data used for "clean-out effect"
J
Data used for "sulfur level effect"
On-road mileage accumulation for the L/M procedure was performed over a pre-defined
driving route using the same fuel used for emissions testing (either high or low sulfur). The
route used was selected for its resemblance to the FTP in terms of speed and load distribution,
20
-------
DRAFT DOCUMENT
and was used as a way to rapidly accumulate miles over a representative test cycle without
occupying a chassis dynamometer. Refer to Appendix A for more details on this drive route.
Approximately halfway through the data collection, the Short procedure was extended to
include a second clean-out and subsequent set of emission tests performed on low sulfur fuel.
This change was made after a mid-point analysis of available data suggested that sulfur level had
an effect on emissions starting immediately after the cleanout for some vehicles/ Figure 6-3
shows the modified Short procedure containing two fuel sulfur levels, and the change in the
number of vehicles providing the "sulfur level" data can be seen in Table 7-7 starting with
Family ID N513.
Figure 6-3 Modified Short procedure
Fuel change to high-sulfur (28 ppm) test fuel
Pre-clean-out (as-received) replicate
FTPs at 28 ppm S
Sulfur clean-out cycle, 2 x US06 at 28 ppm S
Post-clean-out replicate FTPs at 28 ppm S
Fuel change to low-sulfur (5 ppm) test fuel
Sulfur clean-out cycle, 2 x US06 at 5 ppm S
Low-sulfur replicates for modified S procedure, FTPs
at 5 ppm S
Data used for "clean-out effect"
Q.
to
Data used for "sulfur level effect"
This information became the basis for the "clean-out at 5ppm" conclusions.
21
-------
DRAFT DOCUMENT
The data generated by this test program will be discussed as three distinct but overlapping
datasets: "clean-out at 28 ppm" data, "clean-out at 5 ppm" data, and "sulfur level" data. The
"clean-out at 28 ppm" data consists of the measurements from original and modified 'short'
procedures; "clean-out at 5 ppm" data consists of a subset of original and modified 'long'
procedures and modified 'short procedures; "sulfur level" data consists of the complete data
from original and modified 'long' procedures and modified 'short' procedures. All vehicles
included in the "sulfur level" dataset had a subset of their tests included in the "clean-out"
datasets, by design.
Pollutants included in the analysis were total hydrocarbons (THC) as reported by the FID
analyzer, carbon monoxide (CO), oxides of nitrogen (NOX), methane (CH4), as well as
particulate matter (PM) mass. The following calculated emissions were also included: non-
methane hydrocarbons (NMHC, defined as THC minus methane) and oxides of nitrogen plus
non-methane organic gases (NOX + NMOG), a measurement relevant in regulatory contexts/1
Each bag, 'Bag 1 minus Bag 3', and the composites from the FTP test cycle were analyzed
separately. The first bag captures the initial "cold start", meaning the emissions produced when
the vehicle is started after cooling to room temperature overnight. The "hot stabilized" operation
is captured in the second bag (a portion of the test that begins after approximately 8 minutes of
driving), and the emissions from "hot-restart" are measured in the third bag (a repeat of the cold
start bag, but after the vehicle has been turned off for only 10 minutes). The 'Bag 1 minus Bag
3' emission was used to isolate cold-start emissions for each FTP test.
The statistical methodologies described in the following section were applied consistently
in the analysis of all pollutants and all bags. However, the analysis of nitrogen oxides (NOX)
from Bag 2 is presented in greater detail to assist the reader in understanding the analytical
V1 NMOG refers to non-methane organic gases. The flame ionization analyzer used to produce the THC result
produces an inaccurate measurement of oxygenated emission species, which can be corrected using speciated results
for those compounds along with their analyzer response factors. The ratio of NMOG/NMHC can be predicted with
good accuracy using the fuel ethanol content (zero in this case), which was the approach used here to generate the
NMOG results. This method is described in Sluder S.C, West B.H., "NMOG Emissions Characterizations and
Estimation for Vehicles Using Ethanol-Blended Fuels". Oak Ridge National Laboratories, 2011. Publication
number ORNL/TM-2011/461.
22
-------
DRAFT DOCUMENT
approaches and illustrate the statistical methods being used. The final model results from all
other pollutants are also presented in this report.
The goals of the statistical analyses of the "clean-out" and "sulfur level" data were to
assess the reversible sulfur loading in the catalysts of the in-use fleet, to examine the differences
in the effectiveness of the clean-out cycle by comparing emissions immediately following the
clean-out procedure at two sulfur levels, and to characterize the effects of fuel sulfur level on
emissions as mileage was accumulated.
Prior to proceeding with the statistical analyses, the issues associated with the dataset,
namely, very low emission measurements and outliers were examined. The following sections
describe how these issues were addressed.
The graphical examination of both the "clean-out" and the "sulfur level" dataset revealed
the presence of very low emission measurements from some pollutants and bags including NOX
Bag 2. Since uncertainty associated with these low measurements could potentially affect the
outcome of the analysis, it was important to understand the measurement process and evaluate
the impact of any uncertainty.
During emission testing, the vehicle exhaust stream is collected and diluted with
background air to avoid water condensation and other stability problems. A small sample of this
mixture flows into a collection bag for analysis after the test. The concentration of emission
species in the bag is determined by flowing the contents through a properly-calibrated gas
analyzer and reading the result. This method provides a time-weighted result via physical
integration of the emission stream produced over the course of a transient driving cycle.
The measurement process has uncertainty associated with it as a result of the physics of
mixing and sampling from a gas stream as well as noise in analyzer components such as
23
-------
DRAFT DOCUMENT
optoelectronic detectors and signal amplifiers. This means that repeated measurements taken
under identical process conditions will produce a range of results, their average being the true
(intended) response of the instrument and the range around it representing the measurement
variability.
For the analyzers used in this program, the size of the measurement error is expected to
increase relative to the measured value as the concentration decreases. Moreover, the dilute bag
method used here requires measurement of concentrations in both the sample and background
bags, followed by a subtraction between the two, such that the net result contains variability from
two measurements. To assess whether these issues may affect this dataset, we examined plots of
the measured concentrations for each test by vehicle by pollutant and bag. Figure 7-1 shows the
Bag 2 NOX dataset for the vehicles providing the "sulfur level" data, which contains a number of
very low values, as well as tests where sample and background are of similar magnitude (the
vehicle codes refer to the Family IDs listed in Table 7-7; additional plots are available in
Appendix B). Given these findings, we performed sensitivity analyses to evaluate the impact of
these low emission measurements on the study results (presented in Section 7.3.4).
Figure 7-1 Bag 2 NOX concentration measurements by vehicle
X
0
.a
Q.
Q.
•
: ; * 5 i • k . . • :
% . i * . •
4 > *• * •
f * '° s . i °° « 1 * »" . j '• •
°&%«,3» o^J.0* o?S° fc • •
°f*%°\%Sl°o5^*** j
0 y . "- « 00 = Oo o ) „ ,
? . t° « 5 8= «
•o
•
o
o
I 2 S i 3 S 3 S S S ? 2 1 § 5 3 3 g
lsl|llllll|i|||iii
Bag 2 NOx concentration o Background . sample, Hig
« *
'
.« » 8
• • *
° 1 °
Q8 • S
• ^
§ s s § ?
1 i 1 1 1
n Sulfur • Sample, Low Sulfur
24
-------
DRAFT DOCUMENT
When a dilute emission measurement is lower than the measured background level, the
net result is reported as zero (this calculation is performed on a test-by-test basis). However,
since it is unlikely that tailpipe emissions are truly zero during a test, it was assumed that a zero
is a result of the actual emission level being smaller than the sum of the measurement errors
occurring on the sample and background measurements. The emission level was thus considered
to be below the limit of quantification (LOQ), a level below which we are not confident in the
accuracy of quantitative value.
In this situation, the data point can be left as zeroes, deleted, or replaced with an imputed
value. However, because it was necessary to apply log-transformation, as described in more
detail in Section 7.2, the zero values could not be left as is in the data. Table 7-1 summarizes the
number of measurements with zero values and the percentages in parenthesis for each dataset.
Given that observations below the LOQ appear to be randomly distributed across sulfur levels
and vehicles, and since excluding such observations would result in reduced sample size, less
statistical power, and larger standard errors,10 they were imputed in the current analysis.
Since an imputation method involving each vehicle's own longitudinal data would be
superior to methods using no information about the vehicle,11 a commonly-used single-
imputation method, using half the minimum of a valid measurement from a given mileage bin for
the vehicle with zero values, was performed. This imputation method recognized the fact that
emission measurements below the limit of quantification must be smaller than any quantified
value.
Although using vehicle-specific imputation minimizes the likelihood of artificially
reducing the natural variance of the data, there is a potential to inflate the reliability estimates as
the number of imputed values increase. However, since the number of measurements with
imputed values are less than 20 percent (Table 7-1), we can expect good estimates of the
reliability of measures.12 Nonetheless, it is important to determine the effect of these imputed
values on the resulting test statistics and corresponding conclusions. Thus, the results from the
statistical analysis with and without the imputed values were compared once the model was
finalized to assess the potential for introducing bias. This is discussed further in Section 7.3.4.
25
-------
DRAFT DOCUMENT
Table 7-1 Number of measurements with zero values
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Clean-out at 28 ppm data (N = 479)
NOX
0
32 (6.7%)
0
0
0
THC
1 (0.2%)
6(1.3%)
1 (0.2%)
1 (0.2%)
1 (0.2%)
CO
0
33 (6.9%)
21 (4.4%)
0
0
NMHC
1 (0.2%)
32 (6.7%)
35 (7.3%)
1 (0.2%)
1 (0.2%)
CH4
1 (0.2%)
4 (0.8%)
1 (0.2%)
1 (0.2%)
1 (0.2%)
PM
1 (0.2%)
2 (0.4%)
1 (0.2%)
2 (0.4%)
1 (0.2%)
Clean- out at 5 ppm data (N = 132)
NOX
0
14 (10.6%)
2
0
0
THC
0
2(1.5%)
0
0
0
CO
0
3 (2.3%)
1 (0.8%)
0
0
NMHC
0
5 (3.8%)
8(6.1%)
0
0
CH4
0
3 (2.3%)
0
0
0
PM
0
0
0
0
0
Sulfur level data (N = 228) j
NOX
0
18 (7.9%)
3(1.3%)
0
7(3.1%)
THC
0
2 (0.9%)
0
0
0
CO
0
8 (3.5%)
3 (1.3%)
0
1 (0.4%)
NMHC
0
9(3.9%)
6 (2.8%)
0
0
CH4
0
3 (1.3%)
0
0
0
PM
0
2 (0.9%)
0
0
15 (6.6%)
f The sulfur level data for NMHC Bag 3 had 215 measurements.
7.1.2. Detection of outliers
Next, before proceeding to the full analysis, preliminary models were fit to detect
outliers. The residual plots were visually inspected for outlying observations and the outliers
were identified using the screening criterion value of ±3.5 for the externally studentized
residuals. Generally, one can expect about 95% of the externally studentized residuals to be
within ±3.5 standard deviations, and applying the ±3.5 criterion to detect potential outliers has
been widely used in statistics. When the outlying observation represented an actual
measurement, it was examined to assess its validity. Since none of the outliers representing
actual measurements showed a clear indication of measurement error, it was assumed that the
outlying observations were real and thus they were included in the dataset for the analysis.
However, there were instances where a very low imputed value was identified as an outlier. In
26
-------
DRAFT DOCUMENT
such instances, the imputed values were removed from the dataset. Table 7-2 summarizes the
number of outliers as well as the number of imputed measurements that were removed in
parenthesis.
Table 7-2 Number of outliers
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Clean-out at 28 p
NOX
2(0)
3(1)
2(0)
3(0)
10(0)
THC
6(1)
3(2)
2(0)
3(0)
5(1)
CO
2(0)
6(3)
4(2)
2(0)
2(0)
am data (N = 479)
NMHC
6(1)
1(0)
4(0)
5(0)
6(1)
CH4
2(1)
6(1)
1(0)
0(0)
5(1)
PM
2(0)
5(0)
2(0)
1(0)
7(1)
Clean-out at 5 ppm data (N = 132)
NOX
1(0)
0(0)
2(0)
1(0)
3(0)
THC
1(0)
0(0)
1(0)
0(0)
1(0)
CO
1(0)
3(1)
1(0)
1(0)
2(0)
NMHC
2(0)
1(0)
1(0)
1(0)
1(0)
CH4
1(0)
1(0)
0(0)
0(0)
1(0)
PM
1(0)
1(0)
0(0)
1(0)
3(0)
Sulfur level data (N = 228)1
NOX
0(0)
0(0)
1(0)
3(0)
2(1)
THC
4(0)
1(0)
2(0)
2(0)
2(0)
CO
2(0)
4(1)
2(2)
2(0)
2(0)
NMHC
4(0)
3(1)
2(2)
2(0)
2(0)
CH4
2(0)
2(2)
1(0)
0(0)
1(0)
PM
2(0)
1(0)
1(0)
2(0)
4(0)
f The sulfur level data for NMHC Bag 3 had 215 measurements.
7.2. Modeling Methodology
The following section describes the statistical approaches and the model fitting
methodologies applied in the analysis of all three datasets - "clean-out at 28 ppm", "clean-out at
5 ppm", and "sulfur level" data.
First, the emission measurements were log-transformed. In the current study, the
distributions of emissions exhibited positive skewness (log-normal), and thus, transforming
emission measurements by the natural logarithm was necessary to stabilize the variance, to
obtain a linear relationship between the mean of the dependent variable and the fixed and
27
-------
DRAFT DOCUMENT
random effects, and to normalize the distribution of the residual. The log-transformation of
emission measurements has been well-established in previous studies analyzing vehicle
emissions data.13'14'15
Both the "clean-out" and "sulfur level" data are a classic example of a "repeated
measures data" where multiple measurements were taken from a single vehicle at different
accumulated mileages. The conventional methods for analyzing "repeated measures data" are
the univariate and multivariate analysis of variance (discussed further in Appendix D).
However, the linear mixed model was selected for the analyses of both "clean-out" and "sulfur
level" data for the following reasons: The mixed model approach uses generalized least squares
to estimate the fixed effects, which is considered superior to the ordinary least squares used by
the univariate and multivariate procedure.15 It is a more robust and flexible procedure in
modeling the covariance structures for repeated measures data and better accounts for within-
vehicle mileage-dependent correlations.13'14 In addition, the mixed model is capable of including
vehicles with missing data and handling irregularly spaced measurements.
The MIXED procedure in the SAS 9.2 software package was used to fit the model. The
mixed model is represented in Equation 7-1 as:
Yt = Xifi + Z-Ui + £i Equation 7-1
where ft and ut are fixed and random effects parameters, respectively, and e, is the random
residuals. The mixed model accounts for correlation in the data through the inclusion of random
effects and modeling of the covariance structure, ft represents parameters that are the same for
all vehicles and ut represents parameters that are allowed to vary over vehicles reflecting the
natural heterogeneity in the vehicle fleet. In other words, the model considers the differences in
the effect of sulfur level on emission for each vehicle. The distributional assumptions for the
mixed model are: ut is normal with mean 0 and variance G,; et is normal with mean 0 and
variance R,; the random components ut and st are independent.
In developing the mixed model, a top-down model fitting strategy, similar to previously
established methods,16'17 was used. The first step was to start with a saturated model, including
28
-------
DRAFT DOCUMENT
all candidate fixed effects to ensure unbiasedness of the fixed effect estimates. Next, we selected
a model with the most optimal covariance structure, which specifies the variation between
vehicles as well as the covariation between emission measurements at different accumulated
mileages on the same vehicle. Finally, the fixed effects portion of the model was reduced to fit
the final model.
The main objective of analyzing the "clean-out at 28 ppm" data was to assess the in-use
reversible sulfur loading in the catalysts at the fuel sulfur level of 28 ppm. In addition, the
results from the "clean-out" data were used to supplement the "sulfur level" data in estimating
the differences of in-use sulfur loading between high and low fuel sulfur levels, as described in
Section 7.3.3.
The "clean-out at 28 ppm" data consists of the as-received emission measurements (pre-
cleanout) and the measurements after the back-to-back US06 cycles (post-cleanout) at the fuel
sulfur level of 28 ppm from original and modified 'short' procedures. The change in emissions
between pre- and post-cleanout represents the amount of reversible in-use sulfur loading at 28
ppm. The "clean-out at 28 ppm" data at the time of the analysis comprised 17 vehicle families,
for a total of 81 unique vehicles. Vehicles from the same engine family had the same engine size,
vehicle configuration, and weight. The average starting odometer reading of 81 unique vehicles
was 32,508 ± 6,164 miles. Additional details of this test fleet are shown in Table 7-3. A total of
479 measurements were taken - 242 pre-cleanout and 237 post-cleanout measurements.
29
-------
DRAFT DOCUMENT
Table 7-3 Description of vehicles in the "clean-out at 28 ppm data"
Vehicle
Family ID
M500
M501
M502
M503
M504
M505
M506
M507
M508
M509
N510
N511
N512
N513
N514
N515
N520
Make
Toyota
Ford
Dodge
Honda
Saturn
Chevrolet
Nissan
Ford
Dodge
Chevrolet
Toyota
Chevrolet
Jeep
Ford
Honda
Ford
Toyota
Model
Corolla
Explorer
Caliber
Odyssey
Outlook
Silverado
Altima
Taurus
Caravan
Impala
Sienna
Cobalt
Liberty
Focus
Civic
F150
Tacoma
Model
Year
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2008
2008
2008
2008
2008
2009
Tier 2
Cert Bin
5
4
5
5
5
5
5
5
8
5
5
5
5
4
5
8
5
Number of
Vehicles
5
5
5
5
5
5
5
5
5
5
5
6
5
5
O
5
2
Average Starting
Odometer
32,520
33,548
31,131
35,893
35,700
37,342
32,227
29,407
34,317
26,126
30,939
28,106
27,530
26,843
32,869
31,662
28,837
The box-plot of the "clean-out at 28 ppm" data (Figure 7-2) for log-transformed NOX bag
2 shows the spread of the data for each vehicle family by clean-out status. The diamond and the
line inside the box represent the mean and the median, respectively. The box represents the
interquartile range between 25* and 75* percentile and the error bars show the full data range.
The box-plot suggests that the variance in the pre-cleanout group tends to be larger than
the variance observed in the post-cleanout group. Generally, there is a tendency for the pre-
cleanout measurements, representing the as-received sulfur level, to have higher NOX than the
post-cleanout measurements. In addition, the plot illustrates that the emission profiles are
substantially different between vehicle families. Some of the vehicle families also show the
presence of within-family variability, as demonstrated by each error bar, reflecting the influence
of individual driving patterns on in-use sulfur loading. These findings from the graphical
examination of the data assisted in selecting the appropriate statistical model for the analysis.
30
-------
DRAFT DOCUMENT
Figure 7-2 Box-plot of vehicle families by pre- and post-cleanout at 28ppm
Bag 2)
-4-
-6-
2 -*-\
3f
-10-
-12-
I I
i ' i I I
i i r
i i i i r
Vehicle
| D PRE D POST |
The dependent variable (7,) of the mixed model (Equation 7-1) was the natural logarithm
of emissions. The fixed effect (X,) included in the model was the cleanout status (pre- and post-
cleanout), vehicle type (car vs. trucks), and the interaction term. The random effects (Z,) were
each vehicle family in the study assuming that vehicles from the same vehicle family have
similar emission profiles. For the degrees of freedom, the Satterthwaite approximation was used
for tests of fixed effects.
18
First, a test for the significance of the between-family variation was performed for
inclusion of the random intercept for each family using the likelihood ratio test. For all
pollutants and bags, the result was significant and the random intercept for each family was
included for all subsequent models. The significance of between family-variation was observed
graphically in Figure 7-2 and is also supported by recent automotive emissions studies.19'20
31
-------
DRAFT DOCUMENT
The covariance structure was estimated using restricted, or residual, maximum likelihood
(REML)21 and selected by comparing the Schwarz Bayesian Criterion (BIC) values for various
potential covariance structures. BIC, considered a consistent criterion, accounts for the total
number of observations in the dataset and has generally been known to perform relatively better
for small size settings.22'23'24
In modeling the covariance structures, the variance associated with the pre-cleanout was
permitted to differ from the variance associated with the post-cleanout. As an initial step, a
saturated model with fixed effects was attempted to be fit with "unstructured" (UN) covariance,
which makes no assumptions regarding equal variances or correlations. One advantage of
starting with the unstructured covariance is the fact that it allows selection of the most optimal
covariance structure based on the data by visually examining the matrix for patterns. However,
the model failed to converge and it was not possible to fit an unstructured covariance matrix. It
may be explained by the fact that UN structure requires estimation of a large number of variance
and covariance parameters, resulting in computational problems.
Since the "clean-out" data involved three repeat tests of FTP for both pre- and post-
cleanout without mileage accumulation, the covariance matrix of compound symmetry (CS), in
Equation 7-2, was selected, which assumes that the measurements from the same vehicle have
the homogeneous variance and the correlation among measurements is constant regardless of
how far apart the measurements are.
tj -(- (7^ (7^ (7^ o-^
°12 °2 + °12 °12 °12 r r, 0
2 2222 Equation 7-2
tJi2 tJi2 a^2 a2 + a^2.
The covariance structure was modeled using a combination of compound symmetry
structure within families and a random effect between families. This combination structure
specifies an inter-vehicle random effect of differences between families, and a correlation
structure within families that are the same between emission measurements. It implies that the
only aspect of the covariance between repeated measures is due to the vehicle contribution.
32
-------
DRAFT DOCUMENT
Following the selection of structures for the random effects and the covariance structure
for the residuals, the fixed effects in the model were tested using the approximate F-test with the
Satterthwaite degrees of freedom. A non-significant effect was removed from the model and the
reduced model was re-fit and re-tested until all fixed effects in the model were significant. The
significance level of 10 percent (a = 0.1) was used to test the null hypothesis. Statistical
hierarchy was kept in removing insignificant fixed parameters - first order terms were included
when the second order term was significant.
We then performed the likelihood ratio test between the reference and the nested models
using Maximum Likelihood (ML)21'24 as the estimation method to examine whether the model
can be reduced further without influencing the model fit. To perform the likelihood ratio test, the
-2 Res Log Likelihood Scores from two separate models were considered and the chi square test
statistic was computed by subtracting the -2 Res Log Likelihood from the model with more
parameters from the model with fewer. The Chi-Square test statistic had the degrees of freedom
equal to the difference of the number of parameters. When this test statistic was significant, the
model with the greater number of parameters was chosen. However, when the test statistic was
not significant, there is no significant difference in the fit of the two models and the model with
the fewer number of parameters was selected. Once the fixed effects were finalized based on the
chi square test, no further simplification of the fixed-effects component of the model was
performed. For all pollutants and bags, only the cleanout effect was significant - the emissions
did not differ significantly between cars and trucks.
The change in emissions between pre- (as-received level) and post-cleanout at 28 ppm
was estimated using the differences of least squares means from the final model. The Tukey-
Kramer adjustment was used in calculating the p-values for the least squares means. The
difference of least squares means between pre- and post-cleanout were reverse-transformed to
grams per mile space to estimate the percent reduction in emissions. Table 7-4 shows the
reductions in emissions for all pollutants between pre- and post-cleanout at the fuel sulfur level
of 28 ppm, reflecting the level of reversible sulfur loading in catalysts. The statistical analysis of
the "clean-out at 28 ppm" showed highly significant reductions in emissions for several
33
-------
DRAFT DOCUMENT
pollutants including NOX and hydrocarbons. The findings here suggest there is a degradation of
emission performance for Tier 2 vehicles in-use due to sulfur loading in the catalyst, and it has a
measureable effect on aftertreatment performance.
Table 7-4 Average emission reduction post-cleanout vs. pre-cleanout using 28 ppm test fuel
Bagl
Bag 2
Bag3
FTP Composite
Bag 1 - Bag 3
NOX
(p-value)
-
31.9%
(0.0009)
38.3%
(O.OOOl)
11.4%
(O.OOOl)
-
THC
(p-value)
-
16.5%
(0.0024)
21.4%
(O.OOOl)
4.1%
(0.0187)
-
CO
(p-value)
4.7%
(0.0737)
-
19.5%
(0.0011)
7.6%
(0.0008)
4.2%
(0.0714)
NMHC
(p-value)
-
17.8%
(0.0181)
27.8%
(O.OOOl)
3.0%
(0.0751)
-
CH4
(p-value)
-
15.3%
(0.0015)
12.0%
(O.OOOl)
6.9%
(0.0003)
-
PM
(p-value)
15.4%
(< 0.0001)
-
24.5%
(O.OOOl)
13.7%
(O.OOOl)
-
The clean-out effect is not significant at a = 0.10 when no reduction estimate is provided.
7.3.2. Effect of clean-out at 5 ppm
To study the differences in the effectiveness of the clean-out procedure between 28 ppm
and 5 ppm fuel sulfur levels, a dataset was constructed using the subset of "sulfur level" data
which include the measurements from a subset of original and modified 'long' procedures and
modified 'short' procedure with mileage accumulation less than 50 miles for both sulfur levels.
This dataset essentially represents the first three repeat tests of FTP from each sulfur level
following a high-speed/load "clean-out" procedure consisting of two back-to-back US06 cycles.
The dataset included 17 vehicle families, 23 individual vehicles with 132 observations -
68 measurements from clean-out at 28 ppm and 64 measurements from clean-out at 5 ppm. The
average starting odometer reading of 23 unique vehicles was 31,869 ± 6,572 miles. Additional
details of the test fleet are shown in Table 7-5.
34
-------
DRAFT DOCUMENT
Table 7-5 Description of vehicles in the "clean-out at 5 ppm data"
Vehicle
Family
ID
M500
M501
M502
M503
M504
M505
M506
M507
M508
M509
N510
N511
N512
N513
N514
N515
N520
Vehicle ID
0003
0023
0026
0194
0021
0031
0123
0148
0075
0046
0264
0179
0107
0089, 0178
0010,0101
0006, 0007,
0074, 0165
0011,0022
Make
Toyota
Ford
Dodge
Honda
Saturn
Chevrolet
Nissan
Ford
Dodge
Chevrolet
Toyota
Chevrolet
Jeep
Ford
Honda
Ford
Toyota
Model
Corolla
Explorer
Caliber
Odyssey
Outlook
Silverado
Altima
Taurus
Caravan
Impala
Sienna
Cobalt
Liberty
Focus
Civic
F150
Tacoma
Model
Year
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2008
2008
2008
2008
2008
2009
Tier 2
Cert Bin
5
4
5
5
5
5
5
5
8
5
5
5
5
4
5
8
5
Number of
Vehicles
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
4
2
Average
Starting
Odometer
33,122
27,562
29,097
35,816
43,733
27,891
39,936
28,802
41,117
37,734
38,464
38,722
24,614
24,726
30,091
29,738
28,897
Figure 7-3 shows the box-plot of log-transformed emissions from Bag 2 NOX by vehicle
and by clean-out sulfur level at 28 ppm and 5 ppm. The diamond and the line inside the box
represent the mean and the median, respectively. The box represents the interquartile range
between 25th and 75th percentile and the error bars show the full data range. The data generally
shows that the NOX emissions from clean-out at 28 ppm fuel sulfur are higher compared to the
emissions from clean-out at 5 ppm, suggesting that the fuel sulfur level might affect the
effectiveness of clean-out cycle in reducing sulfur loading in the catalyst. Furthermore, the data
illustrates that there are significant between- and within-vehicle variability.
35
-------
DRAFT DOCUMENT
Figure 7-3 Box-plot of vehicle emissions by clean-out sulfur level at 28 ppm and 5 ppm
Bag 2)
-4-
-6-
-10-
^^^&^^
•o,>,-^Jk %,\!^,-% •^V--?A3.,.c
Vehicle
D Clean-out at 28ppm D Clean-out at ^ppm
The statistical approach described in Section 7.3.1 was applied in modeling the fixed
effects and the covariance structure. Table 7-6 summarizes the percent reduction in emissions
from 28 ppm to 5 ppm fuel sulfur level only for the first three test replicates following the clean-
out cycle. The results from bag 2 NOX, for example, shows that immediately after the clean-out,
a reduction of 47 percent was achieved by lowering the fuel sulfur level. This suggests that the
effectiveness of high-temperature and high-speed cleanout cycle to reduce catalyst sulfur loading
is limited by the level of sulfur in the fuel. Also of note here is the absence of any statistically
significant PM reductions for these procedures; there is more discussion of PM effects in 7.3.3.
36
-------
DRAFT DOCUMENT
Furthermore, the reduction in emissions from cleanout shown in Table 7-4 would likely
be larger if the low sulfur test fuel at 5 ppm had been used for the cleanout procedure and the
tests that immediately followed the as-received baseline emissions. This is demonstrated in
Figure 7-4, which compares the pre-cleanout (as-received) emissions level to the emissions level
after the clean-out using 28 ppm and 5 ppm fuel sulfur. The data generally shows that the lower
fuel sulfur level increases the effectiveness of the cleanout cycle. In order to eliminate the
vehicle variability, only the pre-cleanout results from 23 unique vehicles that received the
cleanout at both 28 ppm and 5 ppm are included (rather than comparing to the pre-cleanout
results from 81 vehicles in Section 7.3.1).
Figure 7-4 Box-plot of vehicle emissions by pre-cleanout and
post-cleanout at 28 ppm and 5 ppm (NOX Bag 2)
-4-
-6-
&
Z
s
-10-
in, - - . ^ «.--.•
'4
* V '*
V
Vehicle
| D Pre-Cleanout B Clean-out at 28ppm B Clean-out at 5ppm |
37
-------
DRAFT DOCUMENT
Table 7-6 Percent reduction in emissions from 28 ppm to 5 ppm
for the first three repeat FTP tests immediately following clean-out
Bagl
Bag 2
Bag3
FTP
Composite
Bag 1 - Bag 3
NOX
(p-value)
5.9%
(0.0896)
47.3%
(0.0010)
51.2%
(O.OOOl)
17.7%
(0.0001)
_*
THC
(p-value)
5.4%
(0.0118)
40.2%
(O.OOOl)
35.0%
(O.OOOl)
11.2%
(O.OOOl)
_*
CO
(p-value)
7.3%
(0.0023)
*
10.1%
(0.0988)
8.3%
(0.0003)
5.8%
(0.0412)
NMHC
(p-value)
4.6%
(0.0465)
34.4%
(0.0041)
45.0%
(O.OOOl)
8.8%
(0.0003)
_*
CH4
(p-value)
11.1%
(O.OOOl)
53.6%
(O.OOOl)
25.4%
(O.OOOl)
21.4%
(O.OOOl)
_*
PM*
-
-
-
-
-
1 Sulfur level not significant at a = 0.10.
7.3.3. Effect of Sulfur level
The "sulfur level" data represents the emission measurements from the repeated FTP test
cycles following the clean-out. This dataset provided key information for assessing the in-use
effect of sulfur level on emissions over time as vehicles operate on fuels with different sulfur
levels. The dataset is comprised of emission measurements from two different fuel sulfur levels
at 28 ppm and 5 ppm, following vehicle operation that removed sulfur from the catalyst (clean-
out procedure). The "sulfur level" data included all measurements from vehicles tested on both
sulfur levels, for a total of 23 unique vehicles from 17 vehicle families (Table 7-7). The average
starting odometer of 23 unique vehicles was 31,869 ± 6,572 miles. A total of 228 measurements
were taken from Long (S/L and L/M) and modified Short procedures - 114 measurements each
for both high and low fuel sulfur levels.
38
-------
DRAFT DOCUMENT
Table 7-7 Description of vehicles in the "sulfur level" data
Vehicle
Family
ID
M500
M501
M502
M503
M504
M505
M506
M507
M508
M509
N510
N511
N512
N513
N514
N515
N520
Vehicle ID
0003
0023
0026
0194
0021
0031
0123
0148
0075
0046
0264
0179
0107
0089, 0178
0010,0101
0006, 0007,
0074, 0165
0011,0022
Make
Toyota
Ford
Dodge
Honda
Saturn
Chevrolet
Nissan
Ford
Dodge
Chevrolet
Toyota
Chevrolet
Jeep
Ford
Honda
Ford
Toyota
Model
Corolla
Explorer
Caliber
Odyssey
Outlook
Silverado
Altima
Taurus
Caravan
Impala
Sienna
Cobalt
Liberty
Focus
Civic
F150
Tacoma
Model
Year
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2007
2008
2008
2008
2008
2008
2009
Tier 2
Cert Bin
5
4
5
5
5
5
5
5
8
5
5
5
5
4
5
8
5
Number of
Vehicles
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
4
2
Average
Starting
Odometer
33,122
27,562
29,097
35,816
43,733
27,891
39,936
28,802
41,117
37,734
38,464
38,722
24,614
24,726
30,091
29,738
28,897
The box-plot of the log-transformed emissions from Bag 2 NOX "sulfur level" data
(Figure 7-5) shows the spread of the data for each vehicle and sulfur level across all mileages.
The diamond and the line inside the box represent the mean and the median, respectively. The
box represents the interquartile range between 25th and 75th percentile and the error bars show the
full data range. Generally, there is a tendency for the vehicles running on high sulfur fuel to emit
more NOX than the vehicles running on low sulfur fuel. However, the effect of operation on
higher sulfur fuel certainly varies by vehicle. For example, Toyota Corolla, Ford Focus, and
Toyota Tacoma clearly show a large effect of fuel sulfur level on emissions while the effect is
more marginal for Nissan Altima and Toyota Sienna. Furthermore, the "sulfur level" data
illustrates the presence of substantial between-vehicle variability, and suggesting that each
vehicle be considered as a random effect in constructing a statistical model. Although some of
the vehicle families had multiple vehicles (i.e., vehicle family IDs N513, N514, N515, and
N520), we did not consider each family as random effects since the vehicles from the same
39
-------
DRAFT DOCUMENT
family had markedly different emission profiles, suggestive of considerable between-vehicle
variability even within the same family.
Also, the increase in emissions as vehicles accumulate mileage for the high sulfur level is
more significant compared to the low sulfur level, contributing to the increased variance for
some vehicles, and suggests that the rate of sulfur loading might be different for the two fuel
sulfur levels. This is shown in Figure 7-6, which shows the log-transformed emissions from
individual vehicles by sulfur level. Thus, the interaction between sulfur level and the
accumulated mileage was included in the statistical modeling of the data. These findings from
the graphical examination of the data assisted in selecting the statistical modeling approach
discussed in this section.
Figure 7-5 Box-plot of individual vehicles by sulfur level (NOY Bag 2)
|
-4-
-6-
-8-
-10-
0
•/*• v*> ft //
\v^
^
. J. 1
I I I I I I I I I I I I I I I I I I I I I
Sj_ Sj j> c\ A,. X- Si r*. ?• r\ s, x< s. Xf A- /:• A> S.* /:* ^ ?•
^^^9'^^'^^^
•%,* •* '&*& %v ^
«&
» '4
'%'%
xx ^
Vehicle
| D High Sulfur at 28ppm D Low Siilfij al jppm |
40
-------
DRAFT DOCUMENT
We refrained from looking at the simple descriptive statistics, such as the mean and the
standard deviation, to assess the relationship between the sulfur level and emissions even as a
preliminary step, because reaching any conclusion from them can potentially be very misleading
due to the presence of repeated measurements, and both between-vehicle and within-vehicle
variability. In addition, the mileage accumulations varied from vehicle to vehicle, and simple
descriptive statistics would not be able to accurately capture the significant amount of variability
inherent in the dataset.
Figure 7-6 Log-transformed emissions from individual vehicles by sulfur level (NOY Bag 2)
X
O
-4-
-10-
-4-
-10-
-4-
-10-
-4-
-10-
vehlD=0003
0 +
°o+°o
+
vehlD=0022
oo
o
vehlD=0075
*^o + +
vehlD=0165
*;
vehlD=0006
o
°0+
vehlD=0023
fV%
vehlD=0089
%
*+
vehlD=0178
OO
vehlD=0007
c° o°
+
vehlD=0026
:-o^^+
veil ID = 01 01
$j
vehlD=0179
o -f %
Ot- O
+ + -H-
vehlD=0010
OJO
vehlD=0031
o
^ +++
vehlD=0107
n? ++ ^
vehlD=0194
S3tofC^ +
vehlD=0011
cP
^+
vehlD=0046
+
.0 + + o
o o
vehlD=0123
flt» SS-B+
vehlD=0264
vehlD=0021
#
vehlD= 0074
*
vehlD=0148
0 0 0
0+++
n i i i i i i i i i i i i i i i i r
0 100 200 0 100 200 0 100 200 0 100 200 0 100 200 0 100 200
miles
sulfur o High + Low
In analyzing the "sulfur level" data, a similar top-down model fitting statistical approach
applied to the "clean-out" data, as described in Section 7.3.1, was applied to characterize the
effects of fuel sulfur level on emissions as a function of accumulated mileages since cleanout.
The dependent variable (7,) was the natural logarithm of emissions. The fixed effects (JQ
included in the model were sulfur level, accumulated mileage, vehicle type, and the interaction
41
-------
DRAFT DOCUMENT
terms. The random effects (Z,) were each vehicle in the study. The likelihood ratio test for the
significance of between-vehicle variation was statistically significant for all pollutants and bags,
and thus, the random intercept for each vehicle was included in the model. The significance of
the between-vehicle variation was observed graphically in Figure 7-5.
All of the measurements on the same vehicle will have the same between-vehicle errors;
their within-vehicle errors will differ, and can be correlated within a vehicle. The measurements
from the same vehicle are correlated simply because they share common characteristics. Also,
measurements on the same vehicle close in time are often more highly correlated than
measurements far apart in time as observed in Figure 7-6 - the covariation within vehicles. Both
within- and between-vehicle errors are assumed independent from vehicle to vehicle. Since
measurements on different vehicles are assumed independent, the structure refers to the
covariance pattern of measurements on the same subject. For most of these structures, the
covariance between two measurements on the same vehicle depends only on the differences in
mileage accumulation between measurements, and the variance is constant over mileage.
The covariance structure was modeled by first fitting the "unstructured" (UN) covariance
matrix with a saturated model including all fixed effects. For similar reasons provided in the
analysis of the "clean-out" data, the unstructured model failed to converge. Next, since
emissions were measured irregularly, where the mileage intervals between measurements are
more or less unique to each vehicle, the spatial covariance structure, which allows for a
continuous representation of mileage, was fit. However, the model failed to converge for the
spatial covariance matrix as well. Thus, we proceeded to fit the compound symmetry (CS)
which specifies that measurements at all mileage have the same variance, and that all
measurements on the same vehicle have the same correlation. The BIC value for the compound
symmetry was 803.36.
Lastly, the first-order autoregressive structure was modeled. It assumes that the variances
are homogeneous and the correlations decline exponentially with time, i.e., the variability in
measured emissions is constant regardless of mileage for each vehicle and the measurements
next to each other are more correlated than the measurements further apart. The BIC value for
42
-------
DRAFT DOCUMENT
the first-order autoregressive structure was 764.90. Based on the BIC value, the first-order
autoregressive structure (Equation 7-3) was selected as the covariance matrix.
= Var(et) =
a2 <72p a2p2 - tfV1
<72p a2 a2p ...
-------
DRAFT DOCUMENT
Table 7-8 Type 3 Tests of Fixed Effects (NO* Bag2)
Model 1
Model 2
Model 3
Model 4
Effect1
slevel
miles
vehclass
slevel * miles
miles * vehclass
slevel
miles
vehclass
slevel * miles
slevel
miles
slevel * miles
slevel
miles
NumDF
1
1
1
1
1
1
1
1
1
1
1
1
1
1
DenDF
93.8
197
23.1
85.3
199
94.4
161
21
85.9
94.5
161
86.1
68
167
F Value
5.32
0.85
0.11
2.36
0.00
5.35
7.67
0.11
2.40
5.38
7.69
2.38
19.25
8.31
Pr>F*
0.0232
0.3579
0.7443
0.1279
0.9591
0.0229
0.0063
0.7468
0.1247
0.0226
0.0062
0.1263
< 0.0001
0.0045
slevel = sulfur level (high and low); miles = accumulated mileage since clean-out;
vehclass = vehicle types (car and truck); Pr > F represents the p-value associated with the F
statistic;
Then, the likelihood ratio test using maximum likelihood was performed to examine if
the model can be reduced further without compromising the model fit. For example, comparing
model 4 and 5 (Table 7-9), since the result of the likelihood ratio test was significant, we
concluded that accumulated mileage indeed has an effect on Bag 2 NOX, after controlling for
sulfur level, and thus, model 4 was selected as the final model.
Table 7-9 Likelihood ratio test for Bag 2 NOX model
Model 4
Model 5
Fixed effects in model
slevel, miles
slevel
-2 Res Log Likelihood
740.0
737.2
p-value (x )
0.04713
The final NOX Bag 2 model (model 4) had sulfur level and mileage terms as the fixed
effects. Thus, the model finds that there is a statistically significant difference in emissions from
high and low fuel sulfur levels that did not differ between vehicle types (car vs. truck) since
sulfur level and vehicle type interaction term was not significant. Also, since the mileage term is
significant, it can be concluded that the mileage accumulation after the clean-out increases
emissions independent of the fuel sulfur level. However, since the sulfur level and the
accumulated mileage interaction term was not significant, the model suggests that the rate of
44
-------
DRAFT DOCUMENT
sulfur loading does not vary by accumulated mileages after the clean-out between high and low
fuel sulfur levels. In other words, the effect of high fuel sulfur on Bag 2 NOX exists immediately
after clean-out and remains essentially constant on a percentage basis, during subsequent driving
of a vehicle.
Figure 7-7 shows the data vs. predicted plots based on the final model for NOX Bag 2.
There are two paired plots next to each other with the same vehicle ID showing emissions from
both high and low sulfur. There are some instances (e.g., vehicle ID 0046) where the model
overestimates the effect of sulfur by over-predicting high sulfur and under-predicting low sulfur.
In contrast, there are other instances (e.g., vehicle ID 0178) where the model underestimates the
effect of sulfur by under-predicting high sulfur and over-predicting low sulfur. However, this is
to be expected given the variability in the emission testing. In general, the model predictions are
in agreement with the data.
45
-------
DRAFT DOCUMENT
Figure 7-7 Data vs. predicted (log-transformed Bag 2 NOY)
-4-
-10-
-4-
X
0 -10-
^
o:
_Q
-4-
-10-
-4-
-10-
Data vs Predicted
velilD=0003
sulfur = High
ft3» ?•
vehlD=0010
sulfur = High
m
velllD=0022
sulfur = High
o o
-!«•
vetllD=0031
sulfur = High
af** °
i i i
0 100 200
vehlD=0003
sulfur = Low
o
+ » 0 i +
o
vehlD = 0010
sulfur = Low
o
•*
vehlD = OQ22
sulfur = Low
SS
vehlD = 0031
sulfur = Low
o
«•+»•»
vehlD = 0006
sulfur = High
$
vehlD = 0011
sulfur= High
CD
+H-
o
vehlD = 0023
sulfur= High
**f
veh!D = 0046
sulfur = High
:* - i
0 0
vehlD= 0006
sulfur= Low
«e
o
vehlD=0011
sulfur = Low
%
vehlD= 0023
sulfur = Low
+ SS04J
vehlD=0046
sulfur = Low
£ +» •>•
I I II I II I I
0 100 200 0 100 200 0 100 200
miles
| group o Data + Model
vehlD=0007
sulfur = High
cP cP
^ «- -H-
o
vehlD=0021
sulfur= High
dP «S *
vehlD= 0026
sulfur = High
^S-
vehlD = 0074
sulfur = High
»
vehlD = 0007
sulfur = Low
- * S
vehlD = 0021
sulfur = Low
5J *? *b
vehID = 0026
sulfur = Low
sV»*
vehlD = 0074
sulfur = Low
0
^
i i ii i i
0 100 200 0 100 200
-4-
-10-
-4-
X
0 -1Q-
r-j
O)
-4-
-10-
-4-
-10-
Data vs Predicted
vehlD=0075
sulfur= High
rf*
vehlD= 0107
sulfur = High
o
vehlD=0165
sulfur = High
.,
vehlD=0194
sulfur = High
_M
i i i
0 100 200
vehlD = 00/5
sulfur = Low
,MM.
vehlD = 0107
sulfur = Low
S" *•* **
vehID = 0165
sulfur = Low
*
veh!D = 0194
sulfur = Low
......
vehlD = 0089
sulfur = Hiah
S*
vehlD = 0123
sulfur = High
*ff* * 9 4
vehlD = 0178
sulfur = High
00 °~l
+ -H- +*
O
vehlD=0264
sulfur= High
w , *
vehlD= 0089
sulfur = Low
m
vehlD=0123
sulfur = Low
!,„»,
vehlD= 0178
sulfur = Low
9 c? ?
vehlD=0264
sulfur = Low
¥¥ w ,
i i ii i ii i i
0 100 200 0 100 200 0 100 200
miles
iroup o Data + Model
vehlD= 0101
sulfur= High
*
vehlD=0148
sulfur = High
„. ? ?.?
o
vehlD= 0179
sulfur = High
vehlD = 0101
sulfur = Low
A
vehlD = 0148
sulfur = Low
-
vehlD = 0179
sulfur = Low
, s «
i i ii i i
0 100 200 0 100 200
46
-------
DRAFT DOCUMENT
Furthermore, the one-to-one plot of data vs. model predictions in Figure 7-8 shows that
the points generally lie close to the 1:1 line and has the adjusted R-square of 0.72, demonstrating
reasonable accuracy in model predictions for Bag 2 NOX.
03
"03
Q
-12 :
I I
-12
Figure 7-8 Data vs. predicted (log-transformed NOX Bag 2)
-10 -9 -8 -7 -6
Model Prediction
• • • High Sulfur • • • Low Sulfur
-5
-4
-3
Figure 7-9 presents the model predictions for individual vehicle by sulfur level for Bag 2
NOX in grams/mile. The model-predicted values depicted in this figure were generated from
Model 4 in Table 7-8 for NOX bag 2. The model for Bag 2 NOX shows that sulfur loading
resumes immediately after the clean-out (at or near zero accumulated mileage) with high fuel
sulfur level emitting significantly higher emissions compared to low fuel sulfur level. In
addition, the sulfur loading continues with vehicle operation, causing an increase in emissions.
The rate of sulfur loading was not statistically different between high and low sulfur levels on a
percentage basis.
47
-------
DRAFT DOCUMENT
Figure 7-9 Model predictions for individual vehicle by sulfur level (NOX Bag 2)
0.020-
0.015-
0.010-
0.005-
0.000-
„ 0.020-
-£ 0.015-
x 0.010-
o
Z 0.005-
0.000-
0.020-
0.015-
0.010-
0.005-
o.ooo-
Bag2: High vs Low Sulfur
vehlD=0003
+ + -H- + +
verilD = 0046
0,0 ++°° + + *
vehlD= 0148
0 °° °
vehlD = 0007
« 9? S
vehlD = 0075
o
cP°°°
++++++
vehlD = 0194
m-A +
vehlD=0021
00
OO 4. ++
++
vehlD=0101
&
vehlD= 0264
0°
0°
+ + "
vehlD = 0022
m
vehlD = 0107
w fp &
vehlD=0031
0
rrT++-
vehlD = 0123
o
^+++++
i i ii i ii i i i ii i i
D 100 200 0 100 200 0 100 200 0 100 200 0 100 200
miles
| sulfur o High + Low
0.0008-
0.0004-
0.0000-
-f, 0.0008-
0 0.0004-
0.0000-
Bag2: High vs Low Sulfur
verilD=0006
000
+++
vehlD= 0074
B¥
vehlD = 0010
0°°
+++
vehlD = 0089
-H-+
vehlD = 0011
ccP
+++
vehlD = 0165
»°
veh!D = 0023
(JJOOO 0
++++++
vehlD=0178
0°
+
vehlD=0026
5=?°+°+ + +
vehlD=0179
0°
0°
0 ++
i i ii i ii i i i ii i i
0 100 200 0 100 200 0 100 200 0 100 200 0 1DO 200
miles
sulfur o High + Low
48
-------
DRAFT DOCUMENT
Table 7-10 summarizes the final models that were selected for all pollutants and bags,
applying the same statistical methodology described for Bag 2 NOX. For models without the
sulfur level and mileage interaction term, the change in emissions from reducing the fuel sulfur
from 28 ppm to 5 ppm was estimated using the differences of least squares means from the final
model, adjusting for other effects in the model. The Tukey-Kramer adjustment was used in
calculating the p-values for the least squares means. The differences of least squares means
between high and low fuel sulfur level were reverse-transformed to estimate the percent
reduction in emissions (Table 7-11). When the sulfur level and mileage interaction term is not
significant, the percent differences in emissions between high and low fuel sulfur levels stay
constant across accumulated mileage after clean-out (the sulfur loading curves for high and low
sulfur are parallel) and thus, using the least squares means to quantify the reduction in emissions
without considering the as-received in-use sulfur loading was sufficient.
49
-------
DRAFT DOCUMENT
Table 7-10 Final models of all pollutants
Pollutant
NOX
THC
CO
NMHC
CH4
PM
Bag
Bagl
Bag 2
Bag3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Bagl
Bag 2
Bag 3
FTP Composite
Bag 1 - Bag 3
Fixed Effects7
slevel, miles
slevel, miles
slevel, miles
slevel, miles, slevel * miles
slevel
slevel, miles, slevel * miles
slevel, miles
slevel, miles
slevel, miles, slevel * miles
slevel
slevel, miles, slevel * miles
slevel
slevel, miles
slevel, miles, slevel * miles
slevel
slevel, miles
slevel, miles, slevel * miles
slevel, miles
slevel, miles, slevel * miles
slevel
slevel, miles, slevel * miles
slevel, miles
slevel, miles
slevel, miles, slevel * miles
slevel, miles
-
-
-
-
miles
slevel = sulfur level (high and low); miles = accumulated mileage since clean-out;
However, for models with significant sulfur level and mileage interaction term, the sulfur
effect does depend on mileage accumulation and the rate of sulfur loading is different for high
and low sulfur levels. Thus, the in-use emission level upon arrival prior to clean-out was
necessary to accurately estimate the percent differences in emissions between high and low fuel
sulfur levels for in-use operation. Without comparing the high sulfur loading curve to the level
of as-received loading, the calculated emissions difference between high and low sulfur may be
arbitrarily high or low based on the point at which the comparison is made. Comparing
50
-------
DRAFT DOCUMENT
emissions immediately after clean-out, for example, may not accurately represent the role of
sulfur accumulation in the real world. Conversely, comparing emissions after a large number of
miles accumulated on the FTP cycle (relatively mild driving) may overestimate the effect of
loading. Comparing just the average results of high sulfur to the average results of low sulfur
would bias the results depending on the number of tests included in the analysis.
Therefore, for cases where the sulfur level and mileage interaction term (slevel * miles)
was significant, the in-use emission level upon arrival (pre-cleanout) from the "clean-out"
dataset was projected out to the sulfur level curve and the differences in emissions at the mileage
where the two lines intersect (in-use equivalent loading) were estimated as the in-use sulfur level
effect. For example, the NOX FTP composite model had a significant sulfur level and mileage
interaction term and thus, the sulfur level curves diverge as the vehicle accumulates mileage.
The mileage at which the pre-cleanout level intersects the high sulfur curve was used to estimate
the percent reduction in emissions from high to low fuel sulfur levels as illustrated in Figure
7-10.
51
-------
DRAFT DOCUMENT
Figure 7-10 Estimating in-use sulfur effect based on clean-out (NOX FTP composite)
0.02
In-use equivalent loading
Pre-cleanout at 28 ppm
(as-received level)
*<•
Post-cleanout 3t 28 ppm
Post-cleanout at 5 ppm
I n-use su Ifu r level effect
-High Sulfur
•Low Sulfur
clean-out 0 50 100 150 200
Miles since clean-out procedure
Table 7-11 summarizes the percent reduction in emissions from 28 ppm to 5 ppm fuel
sulfur for all pollutants and all bags. For all models except CO Bag 1, CO Bagl-Bag3, CH4
Bagl-Bag3, and PM, the reduction estimates are statistically significant at a = 0.05; CO Bag 1,
CO Bagl-Bag3, and CH4 Bagl-Bag3 are significant at a = 0.10. The results suggest that
significant reduction in emissions can be achieved by reducing the fuel sulfur levels from 28 to 5
ppm in the in-use fleet of Tier 2 vehicles.
Unlike the gaseous pollutants, there was no effect of sulfur level found for PM. A likely
explanation is that the majority of PM as measured in this program (that is, from normal-emitting
Tier 2 vehicles operated at low and moderate loads) was soot produced shortly after cold start
(bag I).25 Once formed in the combustion chamber, oxidation of soot requires high
temperatures, lean air-fuel conditions, and residence time. Modern gasoline vehicles are
calibrated to operate at or very near a stoichiometric fuel/air mixture over all operating modes,
and therefore destruction of soot by the catalyst is minimal regardless of its relative efficiency.
52
-------
DRAFT DOCUMENT
As a result, sulfur would not be expected to have a significant effect on directly-emitted PM
(other than very small amounts of sulfate). The clean-out effect on PM was observed in the
initial portion of the procedures but not later, suggesting again that it was not likely an effect of
sulfur on catalyst efficiency but something else related to release and accumulation of material
typically measured as PM. Since there were no analyses of PM composition in this program, we
are not able to draw more definite conclusions here.
Table 7-11 Estimated reduction in in-use emissions from 28 ppm to 5 ppm fuel sulfur
Bagl
Bag 2
Bag3
FTP
Composite
Bag 1 - Bag 3
NOX
(p-value)
10.7%
(0.0033)
59.2%
(< 0.0001)
62.1%
(< 0.0001)
23.0%f
(0.0180)
t
THC
(p-value)
8.5%T
(0.0382)
48.8%
(< 0.0001)
40.2%
(< 0.0001)
13.0%f
(0.0027)
5.2%
(0.0063)
CO
(p-value)
7.5%T
(0.0552)
t
20.1%
(< 0.0001)
11.9%f
(0.0378)
4.3%
(0.0689)
NMHC
(p-value)
7.5%
(< 0.0001)
44.8%T
(0.0260)
49.9%
(< 0.0001)
10.6%T
(0.0032)
5.1%
(0.0107)
CH4
(p-value)
13.9%T
(< 0.0001)
49.9%
(< 0.0001)
29.2%
(< 0.0001)
25.8%f
(< 0.0001)
4.6%
(0.0514)
NOX+NMOG
(p-value)
N/A
N/A
N/A
17.3%
(0.0140)
N/A
PM*
-
-
-
-
-
r Model with significant sulfur and mileage interaction term. * Sulfur level not significant at a = 0.10. For THC bag 1 and CH4
bag 1, because the effect of clean-out was not statistically significant, the reduction estimates are based on the estimates of least
squares means.
7.3.4. Sensitivity Analysis
A series of sensitivity analyses of the "sulfur level" data was performed to address some
of the issues that might affect the mixed model results. They include the impacts of: low
concentration measurements, censoring of measurements with zero values, and influential
vehicles. The sensitivity analyses were conducted only for Bag 2 NOX, since above mentioned
issues pertain the most to Bag 2 NOX. For example, Bag 2 NOX showed a higher percentage of
measurements with zero values than most other pollutant and bag combinations, as illustrated in
Table 7-1.
53
-------
DRAFT DOCUMENT
Effect of low concentration measurements
The issue of measurements with very low concentration from Bag 2 NOX has been
discussed in 7.1.1. To address uncertainty of measurements from these very low-emitting
vehicles, we performed sensitivity analyses using two measurement concentration screening
levels: 100 ppb (based on the lower end of the instrument manufacturer's stated calibration range
for the emission analyzer), and 50 ppb (chosen at half the former limit). In each analysis,
vehicles with all sample measurements falling below the screening level were removed, and
models were refit. Results of these sensitivity analyses are provided in Table 7-12.
Table 7-12 Results of sensitivity modeling analysis (NOX Bag 2)
Model Description
Final NOX bag 2 model
50 ppb vehicle screen
100 ppb vehicle screen
Vehicles
23
17
11
Observations
228
174
120
Model Estimate of
Bag 2 NOX Reduction
59.2%
60.5%
70.2%
In each of these sensitivity analyses, the sulfur level effect remained highly significant
with p-value <0.0001, suggesting a meaningful sulfur effect exists regardless of removal of the
lowest-emitting vehicles. Thus, we conclude that the sulfur effect is considerably larger than the
uncertainty or error associated with the measurements.
Effect of use of imputed values
In order to assess the impact of replacing censored values, models with and without
imputed values for Bag 2 NOX were compared in Table 7-13. For the model without imputed
values, the mixed model was re-fit using a new dataset with all imputed values removed,
consisting only of actual measurements. Based on the examination of the estimates of fixed
effects and the standard errors from both models, we concluded that the imputed values did not
significantly bias the results. The percent reduction in emissions from 28 ppm to 5 ppm fuel
sulfur level was changed from 59.2% in model with imputed values to 57.8% in model without
them.
54
-------
DRAFT DOCUMENT
Table 7-13 Impact of imputed values on final model (NOy Bag 2)
Model with imputed values
Model without imputed values
Effect
slevel
miles
slevel
miles
Estimate
-0.8953
0.0047
-0.8618
0.0046
StdErr
0.2040
0.0016
0.2001
0.0016
DF
68
167
64.1
157
tValue
-4.39
2.88
-4.31
2.82
p-value
< 0.0001
0.0045
< 0.0001
0.0055
Effect of influential vehicles
As an additional test of robustness, we also looked at the impact of removing the
influential vehicles from the dataset. Influence can be broadly defined as the ability of a single or
multiple vehicles to affect the resulting outcome through the presence or absence in the data.
The influential vehicles can be identified by examining the restricted likelihood distance (RLD),
which is calculated after an iterative process of refitting the model with and without each vehicle.
Figure 7-11 shows the restricted likelihood distance from the influential diagnostics
where vehicle IDs 0007, 0046, and 0178 can be considered influential vehicles affecting both the
fixed effects and covariance parameter estimates based on Cook's D and COVRATIO estimates.
Although we do not have specific grounds for excluding these vehicles from the mixed model
analysis since the measurements from these vehicles did not fall into the category of neither low
concentration measurements nor the outlying observations, these influential vehicles were
removed and the model for Bag 2 NOX was refit to examine the impacts from these vehicles.
55
-------
DRAFT DOCUMENT
Figure 7-11 Influential diagnostics for NOX bag 2
2.0-
1.5-
1.0-
0.5-
0.0
Restricted Likelihood Distance
°r °r °r °r °r °r °r
Deleted vehID
The resulting model showed that the percent reduction in emissions from 28 ppm to 5
ppm was 50.9 percent, compared to the reduction of 59.2 percent from the final model. This
analysis demonstrated that even when the influential vehicles are removed from the analysis, the
reduction in emissions from reducing the fuel sulfur level from 28 ppm to 5 ppm is still highly
significant with p-value <0.0001.
8. Summary and Conclusions
This study assessed the emission reductions expected from in-use Tier 2 light duty
vehicles with reduction in gasoline sulfur content. The test fleet consisted of light-duty cars and
trucks chosen to be representative of high sales models covering a range of types and sizes. Test
fuels were two non-ethanol gasolines with properties typical of certification fuel, one at a sulfur
content of 5 ppm and the other at 28 ppm.
56
-------
DRAFT DOCUMENT
Using the high-sulfur test fuel, emissions data were collected from vehicles in their as-
received state, and then following a high-speed/load "clean-out" procedure consisting of two
back-to-back US06 cycles to examine the existence of reversible sulfur loading in the in-use
fleet. In addition, the differences in the effectiveness of the clean-out procedure at reducing
emissions between the two fuel sulfur levels were assessed. Lastly, a representative subset of
vehicles performed additional test replicates alternated with mileage accumulation on both high
and low sulfur test fuels. This dataset was used to assess the differences in the rate of sulfur
reloading after clean-out as a function of fuel sulfur level. Major findings from this study
include:
• Reversible sulfur loading is occurring in the in-use fleet of Tier 2 vehicles and has a
measureable effect on emissions of NOX and other pollutants of interest. For example, by
performing a clean-out cycle on 28 ppm fuel, FTP composite NOX was reduced by 11%,
NMHC by 3%. A PM reduction of 14% was found, but it is not clear that it was a sulfur
effect given other results in the program.
• The effectiveness of high speed/load procedures in restoring catalyst efficiency is limited
when using higher sulfur fuel. Comparing emissions immediately following the clean-
out procedure on 5 vs. 28 ppm fuel, FTP composite NOX emissions were 18% lower,
NMHC 9% lower, and CO 8% lower.
• Reducing fuel sulfur levels from 28 to 5 ppm is expected to achieve significant reductions
in emissions of NOX, hydrocarbons, and other pollutants of interest in the in-use fleet.
For example, controlling for sulfur loading effects by mileage where appropriate, FTP
composite NOX was 23% lower, NOX+NMOG 17% lower, and CO 12% lower. The
following sensitivity analyses were performed for Bag 2 NOx and suggested the
magnitude and statistical significance of the results are robust:
o With and without data from very-low-emitting vehicles, which may contain larger
measurement error than data from vehicles with higher emission levels
o With and without inclusion of imputed values used in place of zero measurements
o With and without data from vehicles determined to be most influential on model
fitting
57
-------
DRAFT DOCUMENT
The overall results of this study are in agreement with other studies conducted using low
sulfur gasoline in Tier 2 vehicles. The magnitude of NOX and HC reductions found in this study
when switching from 28 ppm to 5 ppm fuel are consistent with those found in other studies done
by the US EPA and automobile and catalyst manufacturers.26'27'28
58
-------
DRAFT DOCUMENT
9.
1. The Regulatory Impact Analysis for the Tier2/Sulfur Final Rule, Chapter III and
Appendix B, EPA 420-R-99-023.
2. The Regulatory Impact Analysis for the Control of Hazardous Air Pollutants from Mobile
Sources Final Rule, Chapter 6, EPA 420-R-07-002.
3. Hochhauser A.M. (2008). Review of Prior Studies of Fuel Effects on Vehicle Emissions,
Report number E-84. Coordinating Research Council, Alpharetta, GA. Available at
www.crcao.org.
4. Heck R., Farrauto R.J., Gulati, S.T. (2009). Catalytic Air Pollution Control: Commercial
Technology (3rd Ed.). John Wiley & Sons: Hoboken, NJ.
5. Eastwood P. (2000) Critical Topics in Exhaust Gas Aftertreatment (EngineeringDesign).
Research Studies Press Ltd: Hertfordshire, England.
6. The Regulatory Impact Analysis for the Tier2/Sulfur Final Rule, Chapter III and
Appendix B, EPA 420-R-99-023.
7. Durbin T.D., et al. (2003). The Effect of Fuel Sulfur on NH3 and Other Emissions from
2000-2001 Model Year Vehicles, Appendix C. Report number E-60. Coordinating
Research Council, Alpharetta, GA. Available at www.crcao.org.
8. Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx Emissions from a
PZEV4 Cylinder Application. SAE 2011 World Congress Paper 2011-01-0300. SAE
International: Warrendale, PA.
9. Durbin T.D., et al. (2003). The Effect of Fuel Sulfur on NH3 and Other Emissions from
2000-2001 Model Year Vehicles, Report number E-60. Coordinating Research Council,
Alpharetta, GA. Available at www.crcao.org.
10. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the
behavioral sciences (2nd Ed.). Hillsdale, NJ: Lawrence Erlbaum.
11. Engels, J.M.; Diehr, P. (2003). Imputation of missing longitudinal data: a comparison of
methods. Journal of Clinical Epidemiology, 56 (2003), 968-976.
12. Downey R.G., & King C.V. (1998). Missing data in Likert ratings: a comparison of
replacement methods. J Gen Psychol. 125:175-191.
13. Stephens, R.D. (1994). Remote sensing data and a potential model of vehicle exhaust
emissions. Journal of the Air Waste Management Association, 44:1284-1292.
59
-------
DRAFT DOCUMENT
14. Holmen, B.A., & Niemeier, D.A. (1998). Characterizing the effects of driver variability
on real-world vehicle emissions. Transportation Research Part D: Transport and
Environment, 3(2), 117-128.
15. Beydoun, M., & Guldmann, J. (2006). Vehicle characteristics and emissions: Logit and
regression analyses of I/M data from Massachusetts, Maryland, and Illinois.
Transportation Research Part D: Transport and Environment, 11(1), 59-76.
16. Diggle PJ. (1988). An approach to the analysis of repeated measures. Biometrics,
44:959-971.
17. Wolfmger, R.D. (1993). Covariance structure selection in general mixed models.
Communications in Statistics, Simulation and Computation, 22(4): 1079-1106.
18. Keselman, H.J., Algina, J., Kowalchuk, R.K., & Wolfmger, R.D. (1999). A comparison
of recent approaches to the analysis of repeated measurements. British Journal of
Mathematical and Statistical Psychology, 52:63-78.
19. Deaton, M.I., & Winebrake, JJ. (2000). The Use of Mixed Effects ANCOVA to
Characterize Vehicle Emissions Profiles. J. Transp. Stat. 3:49-64.
20. Winebrake, J.J., Deaton, M.L., Coburn, T.C., & Kelly, KJ. (2000). Statistical Analysis of
Emissions and Deterioration Rates from In-Use, High Mileage CNG and Gasoline Ford
Crown Victoria Taxicabs; SAE Technical Paper Series No. 2000-01-1959; Society of
Automotive Engineers: Warrendale, PA.
21. Verbeke, G., & Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data,
New York: Springer-Verlag.
22. Bozdogan, H. (1987). Model selection and Akaike's information criterion (AIC): the
general theory and its analytical extensions. Psychometrika, 52:345-370.
23. Little, R.C., Milliken, G.A., Stroup, W.W. & Wolfmger, R.D. (1996). SAS system for
mixed models, Gary, NC: SAS Institute.
24. Wolfmger, R.D. (1993). Covariance structure selection in general mixed models.
Communications in Statistics, Simulation and Computation, 22(4): 1079-1106.
25. "Analysis of Particulate Matter Emissions from Light-Duty Gasoline Vehicles in Kansas
City" (2008), Chapter 8. EPA420-R-08-010.
26. Regulatory Impact Analysis for the Control of Hazardous Air Pollutants from Mobile
Sources Final Rule, Chapter 6. EPA 420-R-07-002.
60
-------
DRAFT DOCUMENT
27. Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx Emissions from a
PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011-01-0300. SAE
International: Warrendale, PA.
28. Shapiro, E. (2009). National Clean Gasoline, An Investigation of Costs and Benefits.
Published by the Alliance of Automobile Manufacturers, Washington, DC.
61
------- |